31 citations · 74 across the 27 of their papers we have counts for
31 papers
TeleTune: Evolving Agent Skills From Offline Telemetry
Justin Chih-Yao Chen, Elias Stengel-Eskin, Yan Chen +6
Computer-use agents need to capture procedural knowledge of how people use software. User telemetry offers a scalable source of this knowledge. However, learning reusable skills fr…
From Images to Tasks: Characterizing Multimodal LLM Interactions in the Wild
Jinyi Ye, Scott Counts, Gaurav Verma +2
Multimodal large language models (LLMs) increasingly integrate vision and text, yet how people use them in natural settings remains underexplored. We seek to answer the question: w…
Abstracting Cross-Domain Action Sequences into Interpretable Workflows
Gaurav Verma, Scott Counts
Sequential or time-stamped interaction logs provide objective records of digital application usage, yet their granularity and noise often obscure meaningful insights into people's…
A Framework for Situating Innovations, Opportunities, and Challenges in Advancing Vertical Systems with Large AI Models
Gaurav Verma, Jiawei Zhou, Mohit Chandra +2
Large artificial intelligence (AI) models have garnered significant attention for their remarkable, often "superhuman", performance on standardized benchmarks. However, when these…
Managing Project Teams in an Online Class of 1000+ Students
Nazanin Tabatabaei Anaraki, Taneisha Ng, Gaurav Verma +6
Team projects in Computer Science (CS) help students build collaboration skills, apply theory, and prepare for real-world software development. Online classes present unique opport…
AdaptAgent: Adapting Multimodal Web Agents with Few-Shot Learning from Human Demonstrations
Gaurav Verma, Rachneet Kaur, Nishan Srishankar +3
State-of-the-art multimodal web agents, powered by Multimodal Large Language Models (MLLMs), can autonomously execute many web tasks by processing user instructions and interacting…
UniGuard: Towards Universal Safety Guardrails for Jailbreak Attacks on Multimodal Large Language Models
Sejoon Oh, Yiqiao Jin, Megha Sharma +4
Multimodal large language models (MLLMs) have revolutionized vision-language understanding but remain vulnerable to multimodal jailbreak attacks, where adversarial inputs are metic…
Lived Experience Not Found: LLMs Struggle to Align with Experts on Addressing Adverse Drug Reactions from Psychiatric Medication Use
Mohit Chandra, Siddharth Sriraman, Gaurav Verma +5
Adverse Drug Reactions (ADRs) from psychiatric medications are the leading cause of hospitalizations among mental health patients. With healthcare systems and online communities fa…
Adversarial Text Rewriting for Text-aware Recommender Systems
Sejoon Oh, Gaurav Verma, Srijan Kumar
Text-aware recommender systems incorporate rich textual features, such as titles and descriptions, to generate item recommendations for users. The use of textual features helps mit…
A Community-Centric Perspective for Characterizing and Detecting Anti-Asian Violence-Provoking Speech
Gaurav Verma, Rynaa Grover, Jiawei Zhou +4
Violence-provoking speech -- speech that implicitly or explicitly promotes violence against the members of the targeted community, contributed to a massive surge in anti-Asian crim…
Cross-Modal Projection in Multimodal LLMs Doesn't Really Project Visual Attributes to Textual Space
Gaurav Verma, Minje Choi, Kartik Sharma +3
Multimodal large language models (MLLMs) like LLaVA and GPT-4(V) enable general-purpose conversations about images with the language modality. As off-the-shelf MLLMs may have limit…
MM-Soc: Benchmarking Multimodal Large Language Models in Social Media Platforms
Yiqiao Jin, Minje Choi, Gaurav Verma +2
Social media platforms are hubs for multimodal information exchange, encompassing text, images, and videos, making it challenging for machines to comprehend the information or emot…
Better to Ask in English: Cross-Lingual Evaluation of Large Language Models for Healthcare Queries
Yiqiao Jin, Mohit Chandra, Gaurav Verma +3
Large language models (LLMs) are transforming the ways the general public accesses and consumes information. Their influence is particularly pronounced in pivotal sectors like heal…
A Survey on the Role of Crowds in Combating Online Misinformation: Annotators, Evaluators, and Creators
Bing He, Yibo Hu, Yeon-Chang Lee +3
Online misinformation poses a global risk with significant real-world consequences. To combat misinformation, current research relies on professionals like journalists and fact-che…
Adversarial Robustness of Prompt-based Few-Shot Learning for Natural Language Understanding
Venkata Prabhakara Sarath Nookala, Gaurav Verma, Subhabrata Mukherjee +1
State-of-the-art few-shot learning (FSL) methods leverage prompt-based fine-tuning to obtain remarkable results for natural language understanding (NLU) tasks. While much of the pr…
Cross-Modal Attribute Insertions for Assessing the Robustness of Vision-and-Language Learning
Shivaen Ramshetty, Gaurav Verma, Srijan Kumar
The robustness of multimodal deep learning models to realistic changes in the input text is critical for their applicability to important tasks such as text-to-image retrieval and…
Large-Scale Text Analysis Using Generative Language Models: A Case Study in Discovering Public Value Expressions in AI Patents
Sergio Pelaez, Gaurav Verma, Barbara Ribeiro +1
Labeling data is essential for training text classifiers but is often difficult to accomplish accurately, especially for complex and abstract concepts. Seeking an improved method,…
Learning the Visualness of Text Using Large Vision-Language Models
Gaurav Verma, Ryan A. Rossi, Christopher Tensmeyer +2
Visual text evokes an image in a person's mind, while non-visual text fails to do so. A method to automatically detect visualness in text will enable text-to-image retrieval and ge…
Robustness of Fusion-based Multimodal Classifiers to Cross-Modal Content Dilutions
Gaurav Verma, Vishwa Vinay, Ryan A. Rossi +1
As multimodal learning finds applications in a wide variety of high-stakes societal tasks, investigating their robustness becomes important. Existing work has focused on understand…
Overcoming Language Disparity in Online Content Classification with Multimodal Learning
Gaurav Verma, Rohit Mujumdar, Zijie J. Wang +2
Advances in Natural Language Processing (NLP) have revolutionized the way researchers and practitioners address crucial societal problems. Large language models are now the standar…
Minority Stress Experienced by LGBTQ Online Communities during the COVID-19 Pandemic
Yunhao Yuan, Gaurav Verma, Barbara Keller +1
The COVID-19 pandemic has disproportionately impacted the lives of minorities, such as members of the LGBTQ community (lesbian, gay, bisexual, transgender, and queer) due to pre-ex…
Characterizing, Detecting, and Predicting Online Ban Evasion
Manoj Niverthi, Gaurav Verma, Srijan Kumar
Moderators and automated methods enforce bans on malicious users who engage in disruptive behavior. However, malicious users can easily create a new account to evade such bans. Pre…
DRAG: Director-Generator Language Modelling Framework for Non-Parallel Author Stylized Rewriting
Hrituraj Singh, Gaurav Verma, Aparna Garimella +1
Author stylized rewriting is the task of rewriting an input text in a particular author's style. Recent works in this area have leveraged Transformer-based language models in a den…
Incorporating Stylistic Lexical Preferences in Generative Language Models
Hrituraj Singh, Gaurav Verma, Balaji Vasan Srinivasan
While recent advances in language modeling have resulted in powerful generation models, their generation style remains implicitly dependent on the training data and can not emulate…