3 citations · 10 across the 14 of their papers we have counts for
14 papers
Video Models Can Reason with Verifiable Rewards
Tinghui Zhu, Sheng Zhang, James Y. Huang +5
Video diffusion models have made rapid progress in perceptual realism and temporal coherence, but they remain primarily optimized for plausible generation rather than verifiable re…
Be My Eyes: Extending Large Language Models to New Modalities Through Multi-Agent Collaboration
James Y. Huang, Sheng Zhang, Qianchu Liu +5
Large Language Models (LLMs) have demonstrated remarkable capabilities in challenging, knowledge-intensive reasoning tasks. However, extending LLMs to perceive and reason over a ne…
OmniStruct: Universal Text-to-Structure Generation across Diverse Schemas
James Y. Huang, Wenxuan Zhou, Nan Xu +5
The ability of Large Language Models (LLMs) to generate structured outputs that follow arbitrary schemas is crucial to a wide range of downstream tasks that require diverse structu…
MetaScale: Test-Time Scaling with Evolving Meta-Thoughts
Qin Liu, Wenxuan Zhou, Nan Xu +5
One critical challenge for large language models (LLMs) for making complex reasoning is their reliance on matching reasoning patterns from training data, instead of proactively sel…
MuirBench: A Comprehensive Benchmark for Robust Multi-image Understanding
Fei Wang, Xingyu Fu, James Y. Huang +18
We introduce MuirBench, a comprehensive benchmark that focuses on robust multi-image understanding capabilities of multimodal LLMs. MuirBench consists of 12 diverse multi-image tas…
mDPO: Conditional Preference Optimization for Multimodal Large Language Models
Fei Wang, Wenxuan Zhou, James Y. Huang +4
Direct preference optimization (DPO) has shown to be an effective method for large language model (LLM) alignment. Recent works have attempted to apply DPO to multimodal scenarios…
Offset Unlearning for Large Language Models
James Y. Huang, Wenxuan Zhou, Fei Wang +4
Despite the strong capabilities of Large Language Models (LLMs) to acquire knowledge from their training corpora, the memorization of sensitive information in the corpora such as c…
Contrastive Instruction Tuning
Tianyi Lorena Yan, Fei Wang, James Y. Huang +5
Instruction tuning has been used as a promising approach to improve the performance of large language models (LLMs) on unseen tasks. However, current LLMs exhibit limited robustnes…
DeAL: Decoding-time Alignment for Large Language Models
James Y. Huang, Sailik Sengupta, Daniele Bonadiman +6
Large Language Models (LLMs) are nowadays expected to generate content aligned with human preferences. Current work focuses on alignment at model training time, through techniques…
Robust Natural Language Understanding with Residual Attention Debiasing
Fei Wang, James Y. Huang, Tianyi Yan +2
Natural language understanding (NLU) models often suffer from unintended dataset biases. Among bias mitigation methods, ensemble-based debiasing methods, especially product-of-expe…
Bridging Continuous and Discrete Spaces: Interpretable Sentence Representation Learning via Compositional Operations
James Y. Huang, Wenlin Yao, Kaiqiang Song +3
Traditional sentence embedding models encode sentences into vector representations to capture useful properties such as the semantic similarity between sentences. However, in addit…
Parameter-Efficient Tuning with Special Token Adaptation
Xiaocong Yang, James Y. Huang, Wenxuan Zhou +1
Parameter-efficient tuning aims at updating only a small subset of parameters when adapting a pretrained model to downstream tasks. In this work, we introduce PASTA, in which we on…
Unified Semantic Typing with Meaningful Label Inference
James Y. Huang, Bangzheng Li, Jiashu Xu +1
Semantic typing aims at classifying tokens or spans of interest in a textual context into semantic categories such as relations, entity types, and event types. The inferred labels…
Disentangling Semantics and Syntax in Sentence Embeddings with Pre-trained Language Models
James Y. Huang, Kuan-Hao Huang, Kai-Wei Chang
Pre-trained language models have achieved huge success on a wide range of NLP tasks. However, contextual representations from pre-trained models contain entangled semantic and synt…