5 citations · 20 across the 10 of their papers we have counts for
10 papers
FactLens: Benchmarking Fine-Grained Fact Verification
Kushan Mitra, Dan Zhang, Sajjadur Rahman +1
Large Language Models (LLMs) have shown impressive capability in language generation and understanding, but their tendency to hallucinate and produce factually incorrect informatio…
A Blueprint Architecture of Compound AI Systems for Enterprise
Eser Kandogan, Sajjadur Rahman, Nikita Bhutani +11
Large Language Models (LLMs) have showcased remarkable capabilities surpassing conventional NLP challenges, creating opportunities for use in production use cases. Towards this goa…
CMDBench: A Benchmark for Coarse-to-fine Multimodal Data Discovery in Compound AI Systems
Yanlin Feng, Sajjadur Rahman, Aaron Feng +2
Compound AI systems (CASs) that employ LLMs as agents to accomplish knowledge-intensive tasks via interactions with tools and data retrievers have garnered significant interest wit…
MEGAnno+: A Human-LLM Collaborative Annotation System
Hannah Kim, Kushan Mitra, Rafael Li Chen +2
Large language models (LLMs) can label data faster and cheaper than humans for various NLP tasks. Despite their prowess, LLMs may fall short in understanding of complex, sociocultu…
Knowledge Acquisition and Integration with Expert-in-the-loop
Sajjadur Rahman, Frederick Choi, Hannah Kim +2
Constructing and serving knowledge graphs (KGs) is an iterative and human-centered process involving on-demand programming and analysis. In this paper, we present Kyurem, a program…
Reasoning Capacity in Multi-Agent Systems: Limitations, Challenges and Human-Centered Solutions
Pouya Pezeshkpour, Eser Kandogan, Nikita Bhutani +3
Remarkable performance of large language models (LLMs) in a variety of tasks brings forth many opportunities as well as challenges of utilizing them in production settings. Towards…
Characterizing Large Language Models as Rationalizers of Knowledge-intensive Tasks
Aditi Mishra, Sajjadur Rahman, Hannah Kim +2
Large language models (LLMs) are proficient at generating fluent text with minimal task-specific supervision. Yet, their ability to provide well-grounded rationalizations for knowl…
Towards Transparent, Reusable, and Customizable Data Science in Computational Notebooks
Frederick Choi, Sajjadur Rahman, Hannah Kim +1
Data science workflows are human-centered processes involving on-demand programming and analysis. While programmable and interactive interfaces such as widgets embedded within comp…
Towards Multifaceted Human-Centered AI
Sajjadur Rahman, Hannah Kim, Dan Zhang +2
Human-centered AI workflows involve stakeholders with multiple roles interacting with each other and automated agents to accomplish diverse tasks. In this paper, we call for a holi…
Leam: An Interactive System for In-situ Visual Text Analysis
Sajjadur Rahman, Peter Griggs, Çağatay Demiralp
With the increase in scale and availability of digital text generated on the web, enterprises such as online retailers and aggregators often use text analytics to mine and analyze…