39 citations · 88 across the 8 of their papers we have counts for
8 papers
SciCode: A Research Coding Benchmark Curated by Scientists
Minyang Tian, Luyu Gao, Shizhuo Dylan Zhang +27
Since language models (LMs) now outperform average humans on many challenging tasks, it has become increasingly difficult to develop challenging, high-quality, and realistic evalua…
CodeMind: Evaluating Large Language Models for Code Reasoning
Changshu Liu, Yang Chen, Reyhaneh Jabbarvand
Large Language Models (LLMs) have been widely used to automate programming tasks. Their capabilities have been evaluated by assessing the quality of generated code through tests or…
Can Transformers Learn to Solve Problems Recursively?
Shizhuo Dylan Zhang, Curt Tigges, Stella Biderman +2
Neural networks have in recent years shown promise for helping software engineers write programs and even formally verify them. While semantic information plays a crucial part in t…
Getting More out of Large Language Models for Proofs
Shizhuo Dylan Zhang, Talia Ringer, Emily First
Large language models have the potential to simplify formal theorem proving and make it more accessible. But how to get the most out of these models is still an open question. To a…
Large Language Models are Edge-Case Fuzzers: Testing Deep Learning Libraries via FuzzGPT
Yinlin Deng, Chunqiu Steven Xia, Chenyuan Yang +3
Deep Learning (DL) library bugs affect downstream DL applications, emphasizing the need for reliable systems. Generating valid input programs for fuzzing DL libraries is challengin…
Making Large Language Models Better Reasoners with Step-Aware Verifier
Yifei Li, Zeqi Lin, Shizhuo Zhang +4
Few-shot learning is a challenging task that requires language models to generalize from limited examples. Large language models like GPT-3 and PaLM have made impressive progress i…
GraphPrompt: Graph-Based Prompt Templates for Biomedical Synonym Prediction
Hanwen Xu, Jiayou Zhang, Zhirui Wang +5
In the expansion of biomedical dataset, the same category may be labeled with different terms, thus being tedious and onerous to curate these terms. Therefore, automatically mappin…
Pre-training Co-evolutionary Protein Representation via A Pairwise Masked Language Model
Liang He, Shizhuo Zhang, Lijun Wu +10
Understanding protein sequences is vital and urgent for biology, healthcare, and medicine. Labeling approaches are expensive yet time-consuming, while the amount of unlabeled data…