40 citations · 100 across the 68 of their papers we have counts for
18 papers · 1 filter
Unveiling Performance Challenges of Large Language Models in Low-Resource Healthcare: A Demographic Fairness Perspective
Yue Zhou, Barbara Di Eugenio, Lu Cheng
This paper studies the performance of large language models (LLMs), particularly regarding demographic fairness, in solving real-world healthcare tasks. We evaluate state-of-the-ar…
Evaluating LLMs Capabilities Towards Understanding Social Dynamics
Anique Tahir, Lu Cheng, Manuel Sandoval +3
Social media discourse involves people from different backgrounds, beliefs, and motives. Thus, often such discourse can devolve into toxic interactions. Generative Models, such as…
Different Horses for Different Courses: Comparing Bias Mitigation Algorithms in ML
Prakhar Ganesh, Usman Gohar, Lu Cheng +1
With fairness concerns gaining significant attention in Machine Learning (ML), several bias mitigation techniques have been proposed, often compared against each other to find the…
From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge
Dawei Li, Bohan Jiang, Liangjie Huang +10
Assessment and evaluation have long been critical challenges in artificial intelligence (AI) and natural language processing (NLP). Traditional methods, usually matching-based or s…
Towards Trustworthy Knowledge Graph Reasoning: An Uncertainty Aware Perspective
Bo Ni, Yu Wang, Lu Cheng +2
Recently, Knowledge Graphs (KGs) have been successfully coupled with Large Language Models (LLMs) to mitigate their hallucinations and enhance their reasoning capability, such as i…
Self-correction is Not An Innate Capability in Language Models
Guangliang Liu, Zimo Qi, Xitong Zhang +2
Although there has been growing interest in the self-correction capability of Large Language Models (LLMs), there are varying conclusions about its effectiveness. Prior research ha…
Threshold Filtering Packing for Supervised Fine-Tuning: Training Related Samples within Packs
Jiancheng Dong, Lei Jiang, Wei Jin +1
Packing for Supervised Fine-Tuning (SFT) in autoregressive models involves concatenating data points of varying lengths until reaching the designed maximum length to facilitate GPU…
LLM Uncertainty Quantification through Directional Entailment Graph and Claim Level Response Augmentation
Longchao Da, Tiejin Chen, Lu Cheng +1
The Large language models (LLMs) have showcased superior capabilities in sophisticated tasks across various domains, stemming from basic question-answer (QA), they are nowadays use…
ConU: Conformal Uncertainty in Large Language Models with Correctness Coverage Guarantees
Zhiyuan Wang, Jinhao Duan, Lu Cheng +6
Uncertainty quantification (UQ) in natural language generation (NLG) tasks remains an open challenge, exacerbated by the closed-source nature of the latest large language models (L…
Direct-Inverse Prompting: Analyzing LLMs' Discriminative Capacity in Self-Improving Generation
Jihyun Janice Ahn, Ryo Kamoi, Lu Cheng +2
Mainstream LLM research has primarily focused on enhancing their generative capabilities. However, even the most advanced LLMs experience uncertainty in their outputs, often produc…
LLMs Assist NLP Researchers: Critique Paper (Meta-)Reviewing
Jiangshu Du, Yibo Wang, Wenting Zhao +37
This work is motivated by two key trends. On one hand, large language models (LLMs) have shown remarkable versatility in various generative tasks such as writing, drawing, and ques…
Long-Term Fairness Inquiries and Pursuits in Machine Learning: A Survey of Notions, Methods, and Challenges
Usman Gohar, Zeyu Tang, Jialu Wang +4
The widespread integration of Machine Learning systems in daily life, particularly in high-stakes domains, has raised concerns about the fairness implications. While prior works ha…
Deconstructing The Ethics of Large Language Models from Long-standing Issues to New-emerging Dilemmas: A Survey
Chengyuan Deng, Yiqun Duan, Xin Jin +15
Large Language Models (LLMs) have achieved unparalleled success across diverse language modeling tasks in recent years. However, this progress has also intensified ethical concerns…
Assessing Empathy in Large Language Models with Real-World Physician-Patient Interactions
Man Luo, Christopher J. Warren, Lu Cheng +2
The integration of Large Language Models (LLMs) into the healthcare domain has the potential to significantly enhance patient care and support through the development of empathetic…
API Is Enough: Conformal Prediction for Large Language Models Without Logit-Access
Jiayuan Su, Jing Luo, Hongwei Wang +1
This study aims to address the pervasive challenge of quantifying uncertainty in large language models (LLMs) without logit-access. Conformal Prediction (CP), known for its model-a…
JORA: JAX Tensor-Parallel LoRA Library for Retrieval Augmented Fine-Tuning
Anique Tahir, Lu Cheng, Huan Liu
The scaling of Large Language Models (LLMs) for retrieval-based tasks, particularly in Retrieval Augmented Generation (RAG), faces significant memory constraints, especially when f…
Media Bias Matters: Understanding the Impact of Politically Biased News on Vaccine Attitudes in Social Media
Bohan Jiang, Lu Cheng, Zhen Tan +2
News media has been utilized as a political tool to stray from facts, presenting biased claims without evidence. Amid the COVID-19 pandemic, politically biased news (PBN) has signi…
Large Language Models for Data Annotation and Synthesis: A Survey
Zhen Tan, Dawei Li, Song Wang +7
Data annotation and synthesis generally refers to the labeling or generating of raw data with relevant information, which could be used for improving the efficacy of machine learni…