11 papers
LogicTrack: Auditing Reasoning Trajectories of Large Language Models with Formal Logic Solvers
Jingyu Hu, Shu Yang, Weiru Liu +1
Chain-of-Thought (CoT) reasoning has been shown to improve the performance of large language models (LLMs), yet existing optimization methods largely rely on outcome-based feedback…
Measuring Judgment Quality in Natural-Language Explanations: Evidence from Forecasting Tournaments
Christopher W. Karvetski, Sheldon S. Huang, Simas Kučinskas +4
Decision-makers routinely rely on expert judgments accompanied by written explanations, yet explanation quality is difficult to measure at scale. Forecasting tournaments offer a na…
What a One-Hour Interview Does Not Tell an LLM About You: Interviews Elicit Traits, Not Decision Policies
Ruoxi Su, Yuhan Liu, Jingyu Hu
Large language models are increasingly asked to stand in for particular people, and interviewing the person is the natural way to tell a model who they are. We test what a one-hour…
StakeBench: Evaluating Language Understanding Grounded in Market Commitment
Yunhua Pei, Jingyu Hu, Yiwei Shi +3
Existing financial NLP benchmarks often rely on labels supplied by outside observers, measuring how language is perceived rather than what speakers have committed to in the market.…
Influencing LLM Multi-Agent Dialogue via Policy-Parameterized Prompts
Hongbo Bo, Jingyu Hu, Weiru Liu
Large Language Models (LLMs) have emerged as a new paradigm for multi-agent systems. However, existing research on the behaviour of LLM-based multi-agents relies on ad hoc prompts…
MONICA: Real-Time Monitoring and Calibration of Chain-of-Thought Sycophancy in Large Reasoning Models
Jingyu Hu, Shu Yang, Xilin Gong +3
Large Reasoning Models (LRMs) suffer from sycophantic behavior, where models tend to agree with users' incorrect beliefs and follow misinformation rather than maintain independent…
Mitigating Degree Bias Adaptively with Hard-to-Learn Nodes in Graph Contrastive Learning
Jingyu Hu, Hongbo Bo, Jun Hong +2
Graph Neural Networks (GNNs) often suffer from degree bias in node classification tasks, where prediction performance varies across nodes with different degrees. Several approaches…
Fine-Grained Interpretation of Political Opinions in Large Language Models
Jingyu Hu, Mengyue Yang, Mengnan Du +1
Studies of LLMs' political opinions mainly rely on evaluations of their open-ended responses. Recent work indicates that there is a misalignment between LLMs' responses and their i…
Large Vision-Language Model Alignment and Misalignment: A Survey Through the Lens of Explainability
Dong Shu, Haiyan Zhao, Jingyu Hu +4
Large Vision-Language Models (LVLMs) have demonstrated remarkable capabilities in processing both visual and textual information. However, the critical challenge of alignment betwe…
ProxiMix: Enhancing Fairness with Proximity Samples in Subgroups
Jingyu Hu, Jun Hong, Mengnan Du +1
Many bias mitigation methods have been developed for addressing fairness issues in machine learning. We found that using linear mixup alone, a data augmentation technique, for bias…
Strategic Demonstration Selection for Improved Fairness in LLM In-Context Learning
Jingyu Hu, Weiru Liu, Mengnan Du
Recent studies highlight the effectiveness of using in-context learning (ICL) to steer large language models (LLMs) in processing tabular data, a challenging task given the structu…