Comparing Traditional and LLM-based Search for Image Geolocation
arXiv:2401.10184 · doi:10.1145/3627508.3638305
Abstract
Web search engines have long served as indispensable tools for information retrieval; user behavior and query formulation strategies have been well studied. The introduction of search engines powered by large language models (LLMs) suggested more conversational search and new types of query strategies. In this paper, we compare traditional and LLM-based search for the task of image geolocation, i.e., determining the location where an image was captured. Our work examines user interactions, with a particular focus on query formulation strategies. In our study, 60 participants were assigned either traditional or LLM-based search engines as assistants for geolocation. Participants using traditional search more accurately predicted the location of the image compared to those using the LLM-based search. Distinct strategies emerged between users depending on the type of assistant. Participants using the LLM-based search issued longer, more natural language queries, but had shorter search sessions. When reformulating their search queries, traditional search participants tended to add more terms to their initial queries, whereas participants using the LLM-based search consistently rephrased their initial queries.
References in corpus (9)
- Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Emergent Abilities of Large Language Models
- What Makes a Good Conversation? Challenges in Designing Truly Conversational Agents
- PlaNet - Photo Geolocation with Convolutional Neural Networks
- How Context Affects Language Models' Factual Predictions
- How to Query Language Models?
- Starting Conversations with Search Engines -- Interfaces that Elicit Natural Language Queries
- Prompt Middleware: Mapping Prompts for Large Language Models to UI Affordances
- Bridging the Gulf of Envisioning: Cognitive Design Challenges in LLM Interfaces