4 citations · 9 across the 9 of their papers we have counts for
11 papers
SiliconBench: Speed, Memory, and Fidelity for LLM Serving on Unified-Memory Desktops
Ranran Haoran Zhang, Aysa Xuemo Fan, David Munhá Correia +2
Concurrent local LLM serving on unified-memory desktops must preserve memory headroom and output fidelity, which speed-only rankings overlook. We introduce SiliconBench, which eval…
SIMMER: Benchmarking Latent Failures in LLM Executable Planning with a World Model
Xiaoxin Lu, Ranran Haoran Zhang, Rui Zhang
Large language models (LLMs) are increasingly deployed as planners for autonomous agents in household environments. While existing benchmarks evaluate whether LLM-generated plans e…
Ran Score: a LLM-based Evaluation Score for Radiology Report Generation
Ran Zhang, Yucong Lin, Zhaoli Su +13
Chest X-ray report generation and automated evaluation are limited by poor recognition of low-prevalence abnormalities and inadequate handling of clinically important language, inc…
Correctness Forensics for Batch Speculative Decoding: Diagnosing the Ragged Tensor Problem
Ranran Haoran Zhang, Soumik Dey, Ashirbad Mishra +3
Inference optimizations are routinely evaluated by throughput alone, without verifying output correctness. We conduct a forensic analysis of batch speculative decoding and find tha…
Enhance Multimodal Consistency and Coherence for Text-Image Plan Generation
Xiaoxin Lu, Ranran Haoran Zhang, Yusen Zhang +1
People get informed of a daily task plan through diverse media involving both texts and images. However, most prior research only focuses on LLM's capability of textual plan genera…
HRScene: How Far Are VLMs from Effective High-Resolution Image Understanding?
Yusen Zhang, Wenliang Zheng, Aashrith Madasu +14
High-resolution image (HRI) understanding aims to process images with a large number of pixels, such as pathological images and agricultural aerial images, both of which can exceed…
VisOnlyQA: Large Vision Language Models Still Struggle with Visual Perception of Geometric Information
Ryo Kamoi, Yusen Zhang, Sarkar Snigdha Sarathi Das +2
Large Vision Language Models (LVLMs) have achieved remarkable performance in various vision-language tasks. However, it is still unclear how accurately LVLMs can perceive visual in…
From Lazy to Prolific: Tackling Missing Labels in Open Vocabulary Extreme Classification by Positive-Unlabeled Sequence Learning
Ranran Haoran Zhang, Bensu Uçar, Soumik Dey +3
Open-vocabulary Extreme Multi-label Classification (OXMC) extends traditional XMC by allowing prediction beyond an extremely large, predefined label set (typically to $10^{1…
Evaluating LLMs at Detecting Errors in LLM Responses
Ryo Kamoi, Sarkar Snigdha Sarathi Das, Renze Lou +12
With Large Language Models (LLMs) being widely used across various tasks, detecting errors in their responses is increasingly crucial. However, little research has been conducted o…
Unified Low-Resource Sequence Labeling by Sample-Aware Dynamic Sparse Finetuning
Sarkar Snigdha Sarathi Das, Ranran Haoran Zhang, Peng Shi +2
Unified Sequence Labeling that articulates different sequence labeling problems such as Named Entity Recognition, Relation Extraction, Semantic Role Labeling, etc. in a generalized…
Exploring the Potential of Large Language Models in Generating Code-Tracing Questions for Introductory Programming Courses
Aysa Xuemo Fan, Ranran Haoran Zhang, Luc Paquette +1
In this paper, we explore the application of large language models (LLMs) for generating code-tracing questions in introductory programming courses. We designed targeted prompts fo…