4 papers
DNAMotifTokenizer: Towards Biologically Informed Tokenization of Genomic Sequences
Xiaoxiao Zhou, Zihan Wang, Jingbo Shang +1
DNA language models have advanced genomics, but their downstream performance varies widely due to differences in tokenization, pretraining data, and architecture. We argue that a m…
Order Matters: Rethinking Prompt Construction in In-Context Learning
Warren Li, Yiqian Wang, Zihan Wang +1
In-context learning (ICL) enables large language models to perform new tasks by conditioning on a sequence of examples. Most prior work reasonably and intuitively assumes that whic…
Model-diff: A Tool for Comparative Study of Language Models in the Input Space
Weitang Liu, Yuelei Li, Ying Wai Li +2
Comparing two (large) language models (LMs) side-by-side and pinpointing their prediction similarities and differences on the same set of inputs are crucial in many real-world scen…
Multi-step Problem Solving Through a Verifier: An Empirical Analysis on Model-induced Process Supervision
Zihan Wang, Yunxuan Li, Yuexin Wu +4
Process supervision, using a trained verifier to evaluate the intermediate steps generated by a reasoner, has demonstrated significant improvements in multi-step problem solving. I…