Neural embedding-based specificity metrics for pre-retrieval query performance prediction
Document Type
Article
Source of Publication
Information Processing and Management
Publication Date
7-1-2020
Abstract
© 2020 Elsevier Ltd In information retrieval, the task of query performance prediction (QPP) is concerned with determining in advance the performance of a given query within the context of a retrieval model. QPP has an important role in ensuring proper handling of queries with varying levels of difficulty. Based on the extant literature, query specificity is an important indicator of query performance and is typically estimated using corpus-specific frequency-based specificity metrics However, such metrics do not consider term semantics and inter-term associations. Our work presented in this paper distinguishes itself by proposing a host of corpus-independent specificity metrics that are based on pre-trained neural embeddings and leverage geometric relations between terms in the embedding space in order to capture the semantics of terms and their interdependencies. Specifically, we propose three classes of specificity metrics based on pre-trained neural embeddings: neighborhood-based, graph-based, and cluster-based metrics. Through two extensive and complementary sets of experiments, we show that the proposed specificity metrics (1) are suitable specificity indicators, based on the gold standards derived from knowledge hierarchies (Wikipedia category hierarchy and DMOZ taxonomy), and (2) have better or competitive performance compared to the state of the art QPP metrics, based on both TREC ad hoc collections namely Robust’04, Gov2 and ClueWeb’09 and ANTIQUE question answering collection. The proposed graph-based specificity metrics, especially those that capture a larger number of inter-term associations, proved to be the most effective in both query specificity estimation and QPP. We have also publicly released two test collections (i.e. specificity gold standards) that we built from the Wikipedia and DMOZ knowledge hierarchies.
DOI Link
ISSN
Publisher
Elsevier Ltd
Volume
57
Issue
4
First Page
102248
Disciplines
Computer Sciences
Keywords
Ad hoc retrieval, Neural embeddings, Performance prediction
Scopus ID
Recommended Citation
Bagheri, Ebrahim; Arabzadeh, Negar; Zarrinkalam, Fattane; Jovanovic, Jelena; and Al-Obeidat, Feras, "Neural embedding-based specificity metrics for pre-retrieval query performance prediction" (2020). All Works. 2491.
https://zuscholars.zu.ac.ae/works/2491
Indexed in Scopus
yes
Open Access
no