8 citations · 17 across the 13 of their papers we have counts for
16 papers
Beyond Episodic Evaluation: Memory Architectural Bottlenecks in Sequential Embodied Question Answering
Zikui Cai, Kaushal Janga, Tan Dat Dao +15
Embodied question answering (EQA) is traditionally evaluated under an episodic formulation, where agents solve each task independently and reset internal state between episodes. Ho…
Flash-BoN: Instant Drafts for Inference-Time Scaling in Diffusion Models
Ruchit Rawal, Reza Shirkavand, Sayak Paul +5
Inference-time scaling for text-to-image generation has progressed from simple Best-of- (BoN) sampling to guided search methods that verify and steer candidate trajectories at i…
When AI Benchmarks Plateau: A Systematic Study of Benchmark Saturation
Mubashara Akhtar, Anka Reuel, Prajna Soni +34
Artificial intelligence benchmarks are an important mechanism to measure model progress and guide deployment decisions. However, benchmarks quickly "saturate", making it difficult…
Benchmarking Correctness and Security in Multi-Turn Code Generation
Ruchit Rawal, Jeffrey Yang Fan Chiang, Chihao Shen +4
AI coding assistants powered by large language models (LLMs) have transformed software development, significantly boosting productivity. While existing benchmarks evaluate the corr…
ARGUS: Hallucination and Omission Evaluation in Video-LLMs
Ruchit Rawal, Reza Shirkavand, Heng Huang +2
Video large language models have not yet been widely deployed, largely due to their tendency to hallucinate. Typical benchmarks for Video-LLMs rely simply on multiple-choice questi…
Hints Help Finding and Fixing Bugs Differently in Python and Text-based Program Representations
Ruchit Rawal, Victor-Alexandru Pădurean, Sven Apel +2
With the recent advances in AI programming assistants such as GitHub Copilot, programming is not limited to classical programming languages anymore--programming tasks can also be e…
CinePile: A Long Video Question Answering Dataset and Benchmark
Ruchit Rawal, Khalid Saifullah, Miquel Farré +4
Current datasets for long-form video understanding often fall short of providing genuine long-form comprehension challenges, as many tasks derived from these datasets can be succes…
Perturbed examples reveal invariances shared by language models
Ruchit Rawal, Mariya Toneva
The rapid growth in natural language processing (NLP) research has led to numerous new models, outpacing our understanding of how they compare to established ones. One major reason…
DAD++: Improved Data-free Test Time Adversarial Defense
Gaurav Kumar Nayak, Inder Khatri, Shubham Randive +2
With the increasing deployment of deep neural networks in safety-critical applications such as self-driving cars, medical imaging, anomaly detection, etc., adversarial robustness h…
What Happens During Finetuning of Vision Transformers: An Invariance Based Investigation
Gabriele Merlin, Vedant Nanda, Ruchit Rawal +1
The pretrain-finetune paradigm usually improves downstream performance over training a model from scratch on the same task, becoming commonplace across many areas of machine learni…
Robust Few-shot Learning Without Using any Adversarial Samples
Gaurav Kumar Nayak, Ruchit Rawal, Inder Khatri +1
The high cost of acquiring and annotating samples has made the `few-shot' learning problem of prime importance. Existing works mainly focus on improving performance on clean data a…
Data-free Defense of Black Box Models Against Adversarial Attacks
Gaurav Kumar Nayak, Inder Khatri, Ruchit Rawal +1
Several companies often safeguard their trained deep models (i.e., details of architecture, learnt weights, training details etc.) from third-party users by exposing them only as b…
DE-CROP: Data-efficient Certified Robustness for Pretrained Classifiers
Gaurav Kumar Nayak, Ruchit Rawal, Anirban Chakraborty
Certified defense using randomized smoothing is a popular technique to provide robustness guarantees for deep neural networks against l2 adversarial attacks. Existing works use thi…
Holistic Approach to Measure Sample-level Adversarial Vulnerability and its Utility in Building Trustworthy Systems
Gaurav Kumar Nayak, Ruchit Rawal, Rohit Lal +2
Adversarial attack perturbs an image with an imperceptible noise, leading to incorrect model prediction. Recently, a few works showed inherent bias associated with such attack (rob…
MMD-ReID: A Simple but Effective Solution for Visible-Thermal Person ReID
Chaitra Jambigi, Ruchit Rawal, Anirban Chakraborty
Learning modality invariant features is central to the problem of Visible-Thermal cross-modal Person Reidentification (VT-ReID), where query and gallery images come from different…
Climate Adaptation: Reliably Predicting from Imbalanced Satellite Data
Ruchit Rawal, Prabhu Pradhan
The utility of aerial imagery (Satellite, Drones) has become an invaluable information source for cross-disciplinary applications, especially for crisis management. Most of the map…