6 citations · 34 across the 20 of their papers we have counts for
27 papers
Competence-Based Evaluation of Logical Reasoning in Large Language Models
Jize Jiang, Chengxiang Zhai
Recent advances in large language models (LLMs) have brought impressive performance across diverse tasks; however, shortcomings remain in logical reasoning, including inconsistenci…
Resisting Harmful Instructions through Training-Free Weight Editing
Ali Ebrahimpour-Boroojeny, Xiaocong Yang, Samuel Schapiro +4
Instruction-following language models can comply with requests that conflict with intended behaviors and safety alignment. We introduce a training-free weight-editing approach for…
Skip the Compressor: Coding Agents Fix More Issues When They Pick Whole Files
Xiaocong Yang, Rishit Chatterjee, Himanshu Udupi +2
A coding agent works on repositories that can be far larger than its context window can hold. Prompt compressors and retrievers offer solutions to this problem, but patching an iss…
Sparse Autoencoders Are a Shared Coordinate System for Symbolic Matching
Xiaocong Yang, Xuanyu Meng, Yuji Zhang +2
Large language models (LLMs) hold broad knowledge in one set of weights, but lack interpretability and trustworthy reasoning, especially in high-stakes use cases. A neuro-symbolic…
DAGFormer: Global Topology Prediction for Transformer Cross-Layer Connections
Xiaocong Yang, Yuren Hao, ChengXiang Zhai
We propose DAGFormer, a Transformer language model whose cross-layer connections are planned by a jointly trained topology predictor. Beyond the residual connections in a standard…
Language Models for Page-Level Layout Decisions in E-commerce Search
Varun Joshi, Eva C. Song, ChengXiang Zhai
E-commerce search pages are critical touchpoints for millions of online shoppers. While traditional search engines return a ranked list of results, modern E-commerce search pages i…
C-PMI: Conditional Pointwise Mutual Information for Turn-level Dialogue Evaluation
Liliang Ren, Mankeerat Sidhu, Qi Zeng +3
Existing reference-free turn-level evaluation metrics for chatbots inadequately capture the interaction between the user and the system. Consequently, they often correlate poorly w…
Sparse Modular Activation for Efficient Sequence Modeling
Liliang Ren, Yang Liu, Shuohang Wang +3
Recent hybrid models combining Linear State Space Models (SSMs) with self-attention mechanisms have demonstrated impressive results across a range of sequence modeling tasks. Howev…
Quick Dense Retrievers Consume KALE: Post Training Kullback Leibler Alignment of Embeddings for Asymmetrical dual encoders
Daniel Campos, Alessandro Magnani, ChengXiang Zhai
In this paper, we consider the problem of improving the inference latency of language model-based dense retrieval systems by introducing structural compression and model size asymm…
To Asymmetry and Beyond: Structured Pruning of Sequence to Sequence Models for Improved Inference Efficiency
Daniel Campos, ChengXiang Zhai
Sequence-to-sequence language models can be used to produce abstractive summaries which are coherent, relevant, and concise. Still, model sizes can make deployment in latency-sensi…
oBERTa: Improving Sparse Transfer Learning via improved initialization, distillation, and pruning regimes
Daniel Campos, Alexandre Marques, Mark Kurtz +1
In this paper, we introduce the range of oBERTa language models, an easy-to-use set of language models which allows Natural Language Processing (NLP) practitioners to obtain betwee…
Competence-Based Analysis of Language Models
Adam Davies, Jize Jiang, ChengXiang Zhai
Despite the recent successes of large, pretrained neural language models (LLMs), comparatively little is known about the representations of linguistic structure they learn during p…
Learning by Applying: A General Framework for Mathematical Reasoning via Enhancing Explicit Knowledge Learning
Jiayu Liu, Zhenya Huang, Chengxiang Zhai +1
Mathematical reasoning is one of the crucial abilities of general artificial intelligence, which requires machines to master mathematical logic and knowledge from solving problems.…
When to Use What: An In-Depth Comparative Empirical Analysis of OpenIE Systems for Downstream Applications
Kevin Pei, Ishan Jindal, Kevin Chen-Chuan Chang +2
Open Information Extraction (OpenIE) has been used in the pipelines of various NLP tasks. Unfortunately, there is no clear consensus on which models to use in which tasks. Muddying…
Language Model Pre-Training with Sparse Latent Typing
Liliang Ren, Zixuan Zhang, Han Wang +3
Modern large-scale Pre-trained Language Models (PLMs) have achieved tremendous success on a wide range of downstream tasks. However, most of the LM pre-training objectives only foc…
Analogy Generation by Prompting Large Language Models: A Case Study of InstructGPT
Bhavya Bhavya, Jinjun Xiong, Chengxiang Zhai
We propose a novel application of prompting Pre-trained Language Models (PLMs) to generate analogies and study how to design effective prompts for two task settings: generating a s…
CONCRETE: Improving Cross-lingual Fact-checking with Cross-lingual Retrieval
Kung-Hsiang Huang, ChengXiang Zhai, Heng Ji
Fact-checking has gained increasing attention due to the widespread of falsified information. Most fact-checking approaches focus on claims made in English only due to the data sca…
Incorporating Task-specific Concept Knowledge into Script Learning
Chenkai Sun, Tie Xu, ChengXiang Zhai +1
In this paper, we present Tetris, a new task of Goal-Oriented Script Completion. Unlike previous work, it considers a more realistic and general setting, where the input includes n…
Sparse*BERT: Sparse Models Generalize To New tasks and Domains
Daniel Campos, Alexandre Marques, Tuan Nguyen +2
Large Language Models have become the core architecture upon which most modern natural language processing (NLP) systems build. These models can consistently deliver impressive acc…
DeepQAMVS: Query-Aware Hierarchical Pointer Networks for Multi-Video Summarization
Safa Messaoud, Ismini Lourentzou, Assma Boughoula +4
The recent growth of web video sharing platforms has increased the demand for systems that can efficiently browse, retrieve and summarize video content. Query-aware multi-video sum…
Towards Dark Jargon Interpretation in Underground Forums
Dominic Seyler, Wei Liu, XiaoFeng Wang +1
Dark jargons are benign-looking words that have hidden, sinister meanings and are used by participants of underground forums for illicit behavior. For example, the dark term "rat"…
Multi-task Learning for Multilingual Neural Machine Translation
Yiren Wang, ChengXiang Zhai, Hany Hassan Awadalla
While monolingual data has been shown to be useful in improving bilingual neural machine translation (NMT), effectively and efficiently leveraging monolingual data for Multilingual…
AutoML to Date and Beyond: Challenges and Opportunities
Shubhra Kanti Karmaker Santu, Md. Mahadi Hassan, Micah J. Smith +3
As big data becomes ubiquitous across domains, and more and more stakeholders aspire to make the most of their data, demand for machine learning tools has spurred researchers to ex…
Towards a Soft Faceted Browsing Scheme for Information Access
Yinan Zhang, Parikshit Sondhi, Anjan Goswami +1
Faceted browsing is a commonly supported feature of user interfaces for access to information. Existing interfaces generally treat facet values selected by a user as hard filters a…