An analysis of searchers' perceptions of sponsored and non-sponsored links using nested design

This paper examines the effects of sponsored links' reputation. We pursued this study through a mature design of experiment method - the nested design - in order to investigate this fast growing segment of online advertising. In this study, we firstly analyze the adaptability of the nested design to find out whether the reputation of sponsored links will bias searchers. We then carefully contrived a user study to investigate this phenomenon. We used 56 participants engaged in six ecommerce Web searching tasks. The tasks were extracted from the transaction log of a Web search engine so that they could represent actual ecommerce searching information needs. We controlled the study by switching non-sponsored and sponsored links on half of the tasks for each participant. By doing this, we could use the nested design statistical method to detect the impact of the sponsored links' reputation on the final evaluations of these results. During the study, we collected 2,435 interactions with links from search engine results page and 961 utterance evaluations of these links. The results indicate that the reputation or the location of the sponsored links will show strong significance on the final evaluation of the search engine performance. We give some perspectives and discussion which could benefit search engine providers to improve their services.