8 citations · 36 across the 17 of their papers we have counts for
19 papers
ProductWebBench: Evaluating Website Continuity Beyond Task Completion
Jiajun Wu, Jian Yang, Yaxin Du +9
Frontend coding agents are usually graded on one question: did the requested target get built. Real engineering changes carry a second obligation, because the rest of the site has…
IFEvalCode: Controlled Code Generation
Jian Yang, Wei Zhang, Shukai Liu +9
Code large language models (Code LLMs) have made significant progress in code generation by translating natural language descriptions into functional code; however, real-world appl…
Multilingual Multimodal Software Developer for Code Generation
Linzheng Chai, Jian Yang, Shukai Liu +12
The rapid advancement of Large Language Models (LLMs) has significantly improved code generation, yet most models remain text-only, neglecting crucial visual aids like diagrams and…
OpenCoder: The Open Cookbook for Top-Tier Code Large Language Models
Siming Huang, Tianhao Cheng, J. K. Liu +16
Large language models (LLMs) for code have become indispensable in various domains, including code generation, reasoning tasks and agent systems. While open-access code LLMs are in…
MdEval: Massively Multilingual Code Debugging
Shukai Liu, Linzheng Chai, Jian Yang +15
Code large language models (LLMs) have made significant progress in code debugging by directly generating the correct code based on the buggy code snippet. Programming benchmarks,…
M2rc-Eval: Massively Multilingual Repository-level Code Completion Evaluation
Jiaheng Liu, Ken Deng, Congnan Liu +13
Repository-level code completion has drawn great attention in software engineering, and several benchmark datasets have been introduced. However, existing repository-level code com…
In-Context Code-Text Learning for Bimodal Software Engineering
Xunzhu Tang, Liran Wang, Yonghui Liu +6
Bimodal software analysis initially appeared to be within reach with the advent of large language models. Unfortunately, the complex interplay of natural language text and code in…
FuzzCoder: Byte-level Fuzzing Test via Large Language Model
Liqun Yang, Jian Yang, Chaoren Wei +13
Fuzzing is an important dynamic program analysis technique designed for finding vulnerabilities in complex software. Fuzzing involves presenting a target program with crafted malic…
TableBench: A Comprehensive and Complex Benchmark for Table Question Answering
Xianjie Wu, Jian Yang, Linzheng Chai +10
Recent advancements in Large Language Models (LLMs) have markedly enhanced the interpretation and processing of tabular data, introducing previously unimaginable capabilities. Desp…
Raw Text is All you Need: Knowledge-intensive Multi-turn Instruction Tuning for Large Language Model
Xia Hou, Qifeng Li, Jian Yang +8
Instruction tuning as an effective technique aligns the outputs of large language models (LLMs) with human preference. But how to generate the seasonal multi-turn dialogues from ra…
UniCoder: Scaling Code Large Language Model via Universal Code
Tao Sun, Linzheng Chai, Jian Yang +6
Intermediate reasoning or acting steps have successfully improved large language models (LLMs) for handling various downstream natural language processing (NLP) tasks. When applyin…
McEval: Massively Multilingual Code Evaluation
Linzheng Chai, Shukai Liu, Jian Yang +15
Code large language models (LLMs) have shown remarkable advances in code understanding, completion, and generation tasks. Programming benchmarks, comprised of a selection of code c…
SEvenLLM: Benchmarking, Eliciting, and Enhancing Abilities of Large Language Models in Cyber Threat Intelligence
Hangyuan Ji, Jian Yang, Linzheng Chai +9
To address the increasing complexity and frequency of cybersecurity incidents emphasized by the recent cybersecurity threat reports with over 10 billion instances, cyber threat int…
xCoT: Cross-lingual Instruction Tuning for Cross-lingual Chain-of-Thought Reasoning
Linzheng Chai, Jian Yang, Tao Sun +8
Chain-of-thought (CoT) has emerged as a powerful technique to elicit reasoning in large language models and improve a variety of downstream tasks. CoT mainly demonstrates excellent…
MAC-SQL: A Multi-Agent Collaborative Framework for Text-to-SQL
Bing Wang, Changyu Ren, Jian Yang +8
Recent LLM-based Text-to-SQL methods usually suffer from significant performance degradation on "huge" databases and complex user questions that require multi-step reasoning. Moreo…
MT4CrossOIE: Multi-stage Tuning for Cross-lingual Open Information Extraction
Tongliang Li, Zixiang Wang, Linzheng Chai +8
Cross-lingual open information extraction aims to extract structured information from raw text across multiple languages. Previous work uses a shared cross-lingual pre-trained mode…
OWL: A Large Language Model for IT Operations
Hongcheng Guo, Jian Yang, Jiaheng Liu +13
With the rapid development of IT operations, it has become increasingly crucial to efficiently manage and analyze large volumes of data for practical applications. The techniques o…
QURG: Question Rewriting Guided Context-Dependent Text-to-SQL Semantic Parsing
Linzheng Chai, Dongling Xiao, Jian Yang +5
Context-dependent Text-to-SQL aims to translate multi-turn natural language questions into SQL queries. Despite various methods have exploited context-dependence information implic…
CQR-SQL: Conversational Question Reformulation Enhanced Context-Dependent Text-to-SQL Parsers
Dongling Xiao, Linzheng Chai, Qian-Wen Zhang +3
Context-dependent text-to-SQL is the task of translating multi-turn questions into database-related SQL queries. Existing methods typically focus on making full use of history cont…