Creating simulated search environments has been of a significant interest in information retrieval, in both general and biomedical search domains. Existing collections include modest number of queries and are constructed by manually evaluating retrieval results. In this work we propose leveraging MeSH term assignments for creating synthetic test beds. We select a suitable subset of MeSH terms as queries, and utilize MeSH term assignments as labels for retrieval evaluation. Using well studied retrieval functions, we show that their performance on the proposed data is consistent with similar findings in previous work. We further use the proposed retrieval evaluation framework to better understand how to combine heterogeneous sources of textual information.
展开▼
