11 papers
LDDR: Linear-DPP Dynamic-Resolution Frame Sampling for Video MLLMs
Jingfeng Chen, Jiawen Qian, Wendi Deng +5
Multimodal large language models, when used for long video understanding, require selecting an informative frame set from massive and redundant candidates. Limited by the model's v…
Document-as-Image Representations Fall Short for Scientific Retrieval
Ghazal Khalighinejad, Raghuveer Thirukovalluru, Alexander H. Oh +1
Many recent document embedding models are trained on document-as-image representations, embedding rendered pages as images rather than the underlying source. Meanwhile, existing be…
Text-Guided Semantic Image Encoder
Raghuveer Thirukovalluru, Xiaochuang Han, Bhuwan Dhingra +2
Image encoders, a fundamental component of vision-language models (VLMs), are typically pretrained independently before being aligned with a language model. This standard paradigm…
InData: Towards Secure Multi-Step, Tool-Based Data Analysis
Karthikeyan K, Raghuveer Thirukovalluru, Bhuwan Dhingra +1
Large language model agents for data analysis typically generate and execute code directly on databases. However, when applied to sensitive data, this approach poses significant se…
Additive Large Language Models for Semi-Structured Text
Karthikeyan K, Raghuveer Thirukovalluru, David Carlson
Large Language Models have advanced clinical text classification, but their opaque predictions remain a critical barrier to practical adoption in research and clinical settings whe…
ClinStructor: AI-Powered Structuring of Unstructured Clinical Texts
Karthikeyan K, Raghuveer Thirukovalluru, David Carlson
Clinical notes contain valuable, context-rich information, but their unstructured format introduces several challenges, including unintended biases (e.g., gender or racial bias), a…
Breaking the Batch Barrier (B3) of Contrastive Learning via Smart Batch Mining
Raghuveer Thirukovalluru, Rui Meng, Ye Liu +7
Contrastive learning (CL) is a prevalent technique for training embedding models, which pulls semantically similar examples (positives) closer in the representation space while pus…
Atomic Consistency Preference Optimization for Long-Form Question Answering
Jingfeng Chen, Raghuveer Thirukovalluru, Junlin Wang +2
Large Language Models (LLMs) often produce factoid hallucinations - plausible yet incorrect answers. A common mitigation strategy is model alignment, which improves factual accurac…
GenEOL: Harnessing the Generative Power of LLMs for Training-Free Sentence Embeddings
Raghuveer Thirukovalluru, Bhuwan Dhingra
Training-free embedding methods directly leverage pretrained large language models (LLMs) to embed text, bypassing the costly and complex procedure of contrastive learning. Previou…
Atomic Self-Consistency for Better Long Form Generations
Raghuveer Thirukovalluru, Yukun Huang, Bhuwan Dhingra
Recent work has aimed to improve LLM generations by filtering out hallucinations, thereby improving the precision of the information in responses. Correctness of a long-form respon…
Calibrating Long-form Generations from Large Language Models
Yukun Huang, Yixin Liu, Raghuveer Thirukovalluru +2
To enhance Large Language Models' (LLMs) reliability, calibration is essential -- the model's assessed confidence scores should align with the actual likelihood of its responses be…