Publications (35)
GrowLength: Accelerating LLMs Pretraining by Progressively Growing Training Length
Hongye Jin, Xiaotian Han, Jingfeng Yang +3
The evolving sophistication and intricacies of Large Language Models (LLMs) yield unprecedented advancements, yet they simultaneously demand considerable computational resources an…
Toward Fast and Accurate Neural Discourse Segmentation
Yizhong Wang, Sujian Li, Jingfeng Yang
Discourse segmentation, which segments texts into Elementary Discourse Units, is a fundamental step in discourse analysis. Previous discourse segmenters rely on complicated hand-cr…
Improving Large Language Models Function Calling and Interpretability via Guided-Structured Templates
Hy Dang, Tianyi Liu, Zhuofeng Wu +9
Large language models (LLMs) have demonstrated strong reasoning and tool-use capabilities, yet they often fail in real-world tool-interactions due to incorrect parameterization, po…
Situated Natural Language Explanations
Zining Zhu, Haoming Jiang, Jingfeng Yang +6
Natural language is among the most accessible tools for explaining decisions to humans, and large pretrained language models (PLMs) have demonstrated impressive abilities to genera…
Multi-VALUE: A Framework for Cross-Dialectal English NLP
Caleb Ziems, William Held, Jingfeng Yang +3
Dialect differences caused by regional, social, and economic factors cause performance discrepancies for many groups of language technology users. Inclusive and equitable language…
Segment Anything without Supervision
XuDong Wang, Jingfeng Yang, Trevor Darrell
The Segmentation Anything Model (SAM) requires labor-intensive data labeling. We present Unsupervised SAM (UnSAM) for promptable and automatic whole-image segmentation that does no…
Enhancing User Intent Capture in Session-Based Recommendation with Attribute Patterns
Xin Liu, Zheng Li, Yifan Gao +5
The goal of session-based recommendation in E-commerce is to predict the next item that an anonymous user will purchase based on the browsing and purchase history. However, constru…
Thinking Preference Optimization
Wang Yang, Hongye Jin, Jingfeng Yang +2
Supervised Fine-Tuning (SFT) has been a go-to and effective method for enhancing long chain-of-thought (CoT) reasoning in relatively small LLMs by fine-tuning them with long CoT re…
Large Language Models in the Clinic: A Comprehensive Benchmark
Fenglin Liu, Zheng Li, Hongjian Zhou +16
The adoption of large language models (LLMs) to assist clinicians has attracted remarkable attention. Existing works mainly adopt the close-ended question-answering (QA) task with…
SUBS: Subtree Substitution for Compositional Semantic Parsing
Jingfeng Yang, Le Zhang, Diyi Yang
Although sequence-to-sequence models often achieve good performance in semantic parsing for i.i.d. data, their performance is still inferior in compositional generalization. Severa…
RRO: LLM Agent Optimization Through Rising Reward Trajectories
Zilong Wang, Jingfeng Yang, Sreyashi Nag +5
Large language models (LLMs) have exhibited extraordinary performance in a variety of tasks while it remains challenging for them to solve complex multi-step tasks as agents. In pr…
A survey of cross-lingual features for zero-shot cross-lingual semantic parsing
Jingfeng Yang, Federico Fancellu, Bonnie Webber
The availability of corpora to train semantic parsers in English has lead to significant advances in the field. Unfortunately, for languages other than English, annotation is scarc…
Can Language Models Follow Multiple Turns of Entangled Instructions?
Chi Han, Xin Liu, Haodong Wang +12
Despite significant achievements in improving the instruction-following capabilities of large language models (LLMs), the ability to process multiple potentially entangled or confl…
Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond
Jingfeng Yang, Hongye Jin, Ruixiang Tang +5
This paper presents a comprehensive and practical guide for practitioners and end-users working with Large Language Models (LLMs) in their downstream natural language processing (N…
Mutually-paced Knowledge Distillation for Cross-lingual Temporal Knowledge Graph Reasoning
Ruijie Wang, Zheng Li, Jingfeng Yang +4
This paper investigates cross-lingual temporal knowledge graph reasoning problem, which aims to facilitate reasoning on Temporal Knowledge Graphs (TKGs) in low-resource languages b…
END: Early Noise Dropping for Efficient and Effective Context Denoising
Hongye Jin, Pei Chen, Jingfeng Yang +11
Large Language Models (LLMs) have demonstrated remarkable performance across a wide range of natural language processing tasks. However, they are often distracted by irrelevant or…
CCGen: Explainable Complementary Concept Generation in E-Commerce
Jie Huang, Yifan Gao, Zheng Li +7
We propose and study Complementary Concept Generation (CCGen): given a concept of interest, e.g., "Digital Cameras", generating a list of complementary concepts, e.g., 1) Camera Le…
Scaling Laws for Predicting Downstream Performance in LLMs
Yangyi Chen, Binxuan Huang, Yifan Gao +3
Precise estimation of downstream performance in large language models (LLMs) prior to training is essential for guiding their development process. Scaling laws analysis utilizes th…
Tag-Enhanced Tree-Structured Neural Networks for Implicit Discourse Relation Classification
Yizhong Wang, Sujian Li, Jingfeng Yang +2
Identifying implicit discourse relations between text spans is a challenging task because it requires understanding the meaning of the text. To tackle this task, recent studies hav…
Language-Image Alignment with Fixed Text Encoders
Jingfeng Yang, Ziyang Wu, Yue Zhao +1
Currently, the most dominant approach to establishing language-image alignment is to pre-train text and image encoders jointly through contrastive learning, such as CLIP and its va…
Data Diversity Matters for Robust Instruction Tuning
Alexander Bukharin, Shiyang Li, Zhengyang Wang +6
Recent works have shown that by curating high quality and diverse instruction tuning datasets, we can significantly improve instruction-following capabilities. However, creating su…
Inductive or Deductive? Rethinking the Fundamental Reasoning Abilities of LLMs
Kewei Cheng, Jingfeng Yang, Haoming Jiang +9
Reasoning encompasses two typical types: deductive reasoning and inductive reasoning. Despite extensive research into the reasoning capabilities of Large Language Models (LLMs), mo…
QUBRIC: Co-Designing Queries and Rubrics for RL Beyond Verifiable Rewards
Rongzhi Zhang, Rui Feng, Zhihan Zhang +8
Rubric-based RL is a promising route for extending reinforcement learning beyond verifiable rewards, yet existing methods optimize rubrics while treating the query distribution as…
HeaPA: Difficulty-Aware Heap Sampling and On-Policy Query Augmentation for LLM Reinforcement Learning
Weiqi Wang, Xin Liu, Binxuan Huang +13
RLVR has become a standard recipe for training LLMs on reasoning tasks with verifiable outcomes, but when rollout generation dominates the cost, efficiency hinges on which prompts…
Choice Fusion as Knowledge for Zero-Shot Dialogue State Tracking
Ruolin Su, Jingfeng Yang, Ting-Wei Wu +1
With the demanding need for deploying dialogue systems in new domains with less cost, zero-shot dialogue state tracking (DST), which tracks user's requirements in task-oriented dia…
TableFormer: Robust Transformer Modeling for Table-Text Encoding
Jingfeng Yang, Aditya Gupta, Shyam Upadhyay +3
Understanding tables is an important aspect of natural language understanding. Existing models for table understanding require linearization of the table structure, where row or co…
Hephaestus: Improving Fundamental Agent Capabilities of Large Language Models through Continual Pre-Training
Yuchen Zhuang, Jingfeng Yang, Haoming Jiang +16
Due to the scarcity of agent-oriented pre-training data, LLM-based autonomous agents typically rely on complex prompting or extensive fine-tuning, which often fails to introduce ne…
Adversarial Generative Flow Network for Solving Vehicle Routing Problems
Ni Zhang, Jingfeng Yang, Zhiguang Cao +1
Recent research into solving vehicle routing problems (VRPs) has gained significant traction, particularly through the application of deep (reinforcement) learning for end-to-end s…
LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning
Hongye Jin, Xiaotian Han, Jingfeng Yang +5
It is well known that LLMs cannot generalize well to long contexts whose lengths are larger than the training sequence length. This poses challenges when employing LLMs for process…
On the Security Vulnerabilities of Text-to-SQL Models
Xutan Peng, Yipeng Zhang, Jingfeng Yang +1
Although it has been demonstrated that Natural Language Processing (NLP) algorithms are vulnerable to deliberate attacks, the question of whether such weaknesses can lead to softwa…
Shopping MMLU: A Massive Multi-Task Online Shopping Benchmark for Large Language Models
Yilun Jin, Zheng Li, Chenwei Zhang +19
Online shopping is a complex multi-task, few-shot learning problem with a wide and evolving range of entities, relations, and tasks. However, existing models and benchmarks are com…
SeqZero: Few-shot Compositional Semantic Parsing with Sequential Prompts and Zero-shot Models
Jingfeng Yang, Haoming Jiang, Qingyu Yin +3
Recent research showed promising results on combining pretrained language models (LMs) with canonical utterance for few-shot semantic parsing. The canonical utterance is often leng…
MEMORYLLM: Towards Self-Updatable Large Language Models
Yu Wang, Yifan Gao, Xiusi Chen +9
Existing Large Language Models (LLMs) usually remain static after deployment, which might make it hard to inject new knowledge into the model. We aim to build models containing a c…
LR-CNN: Local-aware Region CNN for Vehicle Detection in Aerial Imagery
Wentong Liao, Xiang Chen, Jingfeng Yang +4
State-of-the-art object detection approaches such as Fast/Faster R-CNN, SSD, or YOLO have difficulties detecting dense, small targets with arbitrary orientation in large aerial ima…
Chinese Discourse Segmentation Using Bilingual Discourse Commonality
Jingfeng Yang, Sujian Li
Discourse segmentation aims to segment Elementary Discourse Units (EDUs) and is a fundamental task in discourse analysis. For Chinese, previous researches identify EDUs just throug…