Whose relevance? Web search engines as multisided relevance machines

Aleksandra Urman R. Ulloa M. Makhortykh

11 Mei 2020

Access to accurate and up-to-date information is essential for individual and collective decision making, especially at times of emergency. On February 26, 2020, two weeks before the World Health Organization (WHO) officially declared the COVID-19’s emergency a “pandemic,” we systematically collected and analyzed search results for the term “coronavirus” in three languages from six search engines. We found that different search engines prioritize specific categories of information sources, such as government-related websites or alternative media. We also observed that source ranking within the same search engine is subjected to randomization, which can result in unequal access to information among users.

Sebastian Schultheiß D. Lewandowski

12 September 2022

To gain a better understanding of user knowledge and perspectives of search engines, a fruitful approach are representative online surveys. In 2020, we conducted an online survey with a sample representative of the German online population aged 16 through 69 ( N = 2,012). The online survey included 12 search engine-related sections. The questions cover topics such as usage behavior, self-assessed search engine literacy, trust in search engines, knowledge of ads and search engine optimization (SEO), ability to distinguish ads from organic results, assessments and opinions regarding SEO, and personalization of search results. SEO is the specific focus of the survey, as it was conducted as part of the SEO Effect project, dealing with issues such as the role of SEO from the user perspective. This data set contains complete data from the online survey. On the one hand, the data set will allow further analyses, and, on the other hand, comparisons with follow-up studies.

Kevin Matthe Caramancion

11 Januari 2024

This study embarked on a comprehensive exploration of user preferences between Search Engines and Large Language Models (LLMs) in the context of various information retrieval scenarios. Conducted with a sample size of 100 internet users (N=100) from across the United States, the research delved into 20 distinct use cases ranging from factual searches, such as looking up COVID-19 guidelines, to more subjective tasks, like seeking interpretations of complex concepts in layman's terms. Participants were asked to state their preference between using a traditional search engine or an LLM for each scenario. This approach allowed for a nuanced understanding of how users perceive and utilize these two predominant digital tools in differing contexts. The use cases were carefully selected to cover a broad spectrum of typical online queries, thus ensuring a comprehensive analysis of user preferences. The findings reveal intriguing patterns in user choices, highlighting a clear tendency for participants to favor search engines for direct, fact-based queries, while LLMs were more often preferred for tasks requiring nuanced understanding and language processing. These results offer valuable insights into the current state of digital information retrieval and pave the way for future innovations in this field. This study not only sheds light on the specific contexts in which each tool is favored but also hints at the potential for developing hybrid models that leverage the strengths of both search engines and LLMs. The insights gained from this research are pivotal for developers, researchers, and policymakers in understanding the evolving landscape of digital information retrieval and user interaction with these technologies.

Sebastian Schultheiß D. Lewandowski Sebastian Sünkler

24 Januari 2020

The purpose of this paper is (1) to show which topics are especially fruitful for researchers interested in user behavior in commercial search engines, (2) to help researchers decide which data to collect and to what extent. We classify potential areas for IIR research along two dimensions, namely the type of interaction data used (small-scale or large-scale), and whether search engine companies are likely to publish research on the topic chosen (likely or unlikely). This results in a framework consisting of five areas, which are further detailed. In the second part of the paper, we present some empirical studies showing how researchers could approach relevant topics where no results from the search engine providers themselves are published. We also show how researchers can improve the evidential value of their work by going from small-scale to at least medium-scale studies.

S. Geiss Birgit Stark M. Steiner + 1 lainnya

24 Juni 2020

ABSTRACT Search engines are important political news sources and should thus provide users with diverse political information – an important precondition of a well-informed citizenry. The search engines’ algorithmic content selection strongly influences the diversity of the content received by the users – particularly since most users highly trust search engines and often click on only the first result. A widespread concern is that users are not informed diversely by search engines, but how far this concern applies has hardly been investigated. Our study is the first to investigate content diversity provided by five search engines on ten current political issues in Germany. The findings show that sometimes even the first result is highly diverse, but in most cases, more results must be considered to be informed diversely. This unreliability presents a serious challenge when using search engines as political news sources. Our findings call for media policy measures, for example in terms of algorithmic transparency.