TY - GEN
T1 - Automated evaluation of search engine performance via implicit user feedback
AU - Sharma, Himanshu
AU - Jansen, Bernard J.
PY - 2005
Y1 - 2005
N2 - Measuring the information retrieval effectiveness of Web search engines can be expensive if human relevance judgments are required to evaluate search results. Using implicit user feedback for search engine evaluation provides a cost and time effective manner of addressing this problem. Web search engines can use human evaluation of search results without the expense of human evaluators. An additional advantage of this approach is the availability of real time data regarding system performance. Wecapture user relevance judgments actions such as print, save and bookmark, sending these actions and the corresponding document identifiers to a central server via a client application. We use this implicit feedback to calculate performance metrics, such as precision. We can calculate an overall system performance metric based on a collection of weighted metrics.
AB - Measuring the information retrieval effectiveness of Web search engines can be expensive if human relevance judgments are required to evaluate search results. Using implicit user feedback for search engine evaluation provides a cost and time effective manner of addressing this problem. Web search engines can use human evaluation of search results without the expense of human evaluators. An additional advantage of this approach is the availability of real time data regarding system performance. Wecapture user relevance judgments actions such as print, save and bookmark, sending these actions and the corresponding document identifiers to a central server via a client application. We use this implicit feedback to calculate performance metrics, such as precision. We can calculate an overall system performance metric based on a collection of weighted metrics.
KW - implicit user feedback
KW - search engine evaluation
UR - http://www.scopus.com/inward/record.url?scp=84879848684&partnerID=8YFLogxK
U2 - 10.1145/1076034.1076172
DO - 10.1145/1076034.1076172
M3 - Conference contribution
AN - SCOPUS:84879848684
SN - 1595930345
SN - 9781595930347
T3 - SIGIR 2005 - Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval
SP - 649
EP - 650
BT - SIGIR 2005 - Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval
T2 - 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2005
Y2 - 15 August 2005 through 19 August 2005
ER -