Publications (80)
KET-QA: A Dataset for Knowledge Enhanced Table Question Answering
Mengkang Hu, Haoyu Dong, Ping Luo +2
Due to the concise and structured nature of tables, the knowledge contained therein may be incomplete or missing, posing a significant challenge for table question answering (Table…
DIGMN: Dynamic Intent Guided Meta Network for Differentiated User Engagement Forecasting in Online Professional Social Platforms
Feifan Li, Lun Du, Qiang Fu +4
User engagement prediction plays a critical role for designing interaction strategies to grow user engagement and increase revenue in online social platforms. Through the in-depth…
CoCoSoDa: Effective Contrastive Learning for Code Search
Ensheng Shi, Yanlin Wang, Wenchao Gu +5
Code search aims to retrieve semantically relevant code snippets for a given natural language query. Recently, many approaches employing contrastive learning have shown promising r…
TablePilot: Recommending Human-Preferred Tabular Data Analysis with Large Language Models
Deyin Yi, Yihao Liu, Lang Cao +4
Tabular data analysis is crucial in many scenarios, yet efficiently identifying the most relevant data analysis queries and results for a new table remains a significant challenge.…
Enabling and Analyzing How to Efficiently Extract Information from Hybrid Long Documents with LLMs
Chongjian Yue, Xinrun Xu, Xiaojun Ma +6
Large Language Models (LLMs) demonstrate exceptional performance in textual understanding and tabular reasoning tasks. However, their ability to comprehend and analyze hybrid text,…
GPT4Graph: Can Large Language Models Understand Graph Structured Data ? An Empirical Evaluation and Benchmarking
Jiayan Guo, Lun Du, Hengyu Liu +3
Large language models~(LLM) like ChatGPT have become indispensable to artificial general intelligence~(AGI), demonstrating excellent performance in various natural language process…
GBK-GNN: Gated Bi-Kernel Graph Neural Networks for Modeling Both Homophily and Heterophily
Lun Du, Xiaozhou Shi, Qiang Fu +4
Graph Neural Networks (GNNs) are widely used on a variety of graph-based machine learning tasks. For node-level tasks, GNNs have strong power to model the homophily property of gra…
Source Free Unsupervised Graph Domain Adaptation
Haitao Mao, Lun Du, Yujia Zheng +5
Graph Neural Networks (GNNs) have achieved great success on a variety of tasks with graph-structural data, among which node classification is an essential one. Unsupervised Graph D…
SECRET: Towards Scalable and Efficient Code Retrieval via Segmented Deep Hashing
Wenchao Gu, Ensheng Shi, Yanlin Wang +5
Code retrieval, which retrieves code snippets based on users' natural language descriptions, is widely used by developers and plays a pivotal role in real-world software developmen…
Tackling Long Code Search with Splitting, Encoding, and Aggregating
Fan Hu, Yanlin Wang, Lun Du +4
Code search with natural language helps us reuse existing code snippets. Thanks to the Transformer-based pretraining models, the performance of code search has been improved signif…
SuperRL: Reinforcement Learning with Supervision to Boost Language Model Reasoning
Yihao Liu, Shuocheng Li, Lang Cao +6
Large language models are increasingly used for complex reasoning tasks where high-quality offline data such as expert-annotated solutions and distilled reasoning traces are often…
TAP4LLM: Table Provider on Sampling, Augmenting, and Packing Semi-structured Data for Large Language Model Reasoning
Yuan Sui, Jiaru Zou, Mengyu Zhou +4
Table reasoning tasks have shown remarkable progress with the development of large language models (LLMs), which involve interpreting and drawing conclusions from tabular data base…
Reflection of Thought: Inversely Eliciting Numerical Reasoning in Language Models via Solving Linear Systems
Fan Zhou, Haoyu Dong, Qian Liu +3
Numerical reasoning over natural language has been a long-standing goal for the research community. However, cutting-edge language models have proven difficult to reliably generali…
MM-GNN: Mix-Moment Graph Neural Network towards Modeling Neighborhood Feature Distribution
Wendong Bi, Lun Du, Qiang Fu +3
Graph Neural Networks (GNNs) have shown expressive performance on graph representation learning by aggregating information from neighbors. Recently, some studies have discussed the…
SheetBrain: A Neuro-Symbolic Agent for Accurate Reasoning over Complex and Large Spreadsheets
Ziwei Wang, Jiayuan Su, Mengyu Zhou +7
Understanding and reasoning over complex spreadsheets remain fundamental challenges for large language models (LLMs), which often struggle with accurately capturing the complex str…
Text-to-Image Generation for Abstract Concepts
Jiayi Liao, Xu Chen, Qiang Fu +5
Recent years have witnessed the substantial progress of large-scale models across various domains, such as natural language processing and computer vision, facilitating the express…
CAST: Enhancing Code Summarization with Hierarchical Splitting and Reconstruction of Abstract Syntax Trees
Ensheng Shi, Yanlin Wang, Lun Du +4
Code summarization aims to generate concise natural language descriptions of source code, which can help improve program comprehension and maintenance. Recent studies show that syn…
TableSense: Spreadsheet Table Detection with Convolutional Neural Networks
Haoyu Dong, Shijie Liu, Shi Han +2
Spreadsheet table detection is the task of detecting all tables on a given sheet and locating their respective ranges. Automatic table detection is a key enabling technique and an…
Towards Efficient Fine-tuning of Pre-trained Code Models: An Experimental Study and Beyond
Ensheng Shi, Yanlin Wang, Hongyu Zhang +4
Recently, fine-tuning pre-trained code models such as CodeBERT on downstream tasks has achieved great success in many software testing and analysis tasks. While effective and preva…
FORTAP: Using Formulas for Numerical-Reasoning-Aware Table Pretraining
Zhoujun Cheng, Haoyu Dong, Ran Jia +4
Tables store rich numerical data, but numerical reasoning over tables is still a challenge. In this paper, we find that the spreadsheet formula, which performs calculations on nume…
LibDB: An Effective and Efficient Framework for Detecting Third-Party Libraries in Binaries
Wei Tang, Yanlin Wang, Hongyu Zhang +3
Third-party libraries (TPLs) are reused frequently in software applications for reducing development cost. However, they could introduce security risks as well. Many TPL detection…
A Unified and Fast Interpretable Model for Predictive Analytics
Yuanyuan Jiang, Rui Ding, Tianchi Qiao +3
Predictive analytics aims to build machine learning models to predict behavior patterns and use predictions to guide decision-making. Predictive analytics is human involved, thus t…
Learning Rate Perturbation: A Generic Plugin of Learning Rate Schedule towards Flatter Local Minima
Hengyu Liu, Qiang Fu, Lun Du +4
Learning rate is one of the most important hyper-parameters that has a significant influence on neural network training. Learning rate schedules are widely used in real practice to…
Not All Tokens Matter: Towards Efficient LLM Reasoning via Token Significance in Reinforcement Learning
Hanbing Liu, Lang Cao, Yuanyi Ren +5
Large language models (LLMs) show strong reasoning abilities but often produce unnecessarily long explanations that reduce efficiency. Although reinforcement learning (RL) has been…
Unveiling the Black Box of PLMs with Semantic Anchors: Towards Interpretable Neural Semantic Parsing
Lunyiu Nie, Jiuding Sun, Yanlin Wang +6
The recent prevalence of pretrained language models (PLMs) has dramatically shifted the paradigm of semantic parsing, where the mapping from natural language utterances to structur…
Jupiter: Enhancing LLM Data Analysis Capabilities via Notebook and Inference-Time Value-Guided Search
Shuocheng Li, Yihao Liu, Silin Du +7
Large language models (LLMs) have shown great promise in automating data science workflows, but existing models still struggle with multi-step reasoning and tool use, which limits…
CoCoST: Automatic Complex Code Generation with Online Searching and Correctness Testing
Xinyi He, Jiaru Zou, Yun Lin +4
Large Language Models have revolutionized code generation ability by converting natural language descriptions into executable code. However, generating complex code within real-wor…
PLOG: Table-to-Logic Pretraining for Logical Table-to-Text Generation
Ao Liu, Haoyu Dong, Naoaki Okazaki +2
Logical table-to-text generation is a task that involves generating logically faithful sentences from tables, which requires models to derive logical level facts from table records…
Table2Charts: Recommending Charts by Learning Shared Table Representations
Mengyu Zhou, Qingtao Li, Xinyi He +7
It is common for people to create different types of charts to explore a multi-dimensional dataset (table). However, to recommend commonly composed charts in real world, one should…
RACE: Retrieval-Augmented Commit Message Generation
Ensheng Shi, Yanlin Wang, Wei Tao +5
Commit messages are important for software development and maintenance. Many neural network-based approaches have been proposed and shown promising results on automatic commit mess…
PromptIntern: Saving Inference Costs by Internalizing Recurrent Prompt during Large Language Model Fine-tuning
Jiaru Zou, Mengyu Zhou, Tao Li +2
Recent advances in fine-tuning large language models (LLMs) have greatly enhanced their usage in domain-specific tasks. Despite the success, fine-tuning continues to rely on repeat…
TaCube: Pre-computing Data Cubes for Answering Numerical-Reasoning Questions over Tabular Data
Fan Zhou, Mengkang Hu, Haoyu Dong +3
Existing auto-regressive pre-trained language models (PLMs) like T5 and BART, have been well applied to table question answering by UNIFIEDSKG and TAPEX, respectively, and demonstr…
Vision Language Models for Spreadsheet Understanding: Challenges and Opportunities
Shiyu Xia, Junyu Xiong, Haoyu Dong +6
This paper explores capabilities of Vision Language Models on spreadsheet comprehension. We propose three self-supervised challenges with corresponding evaluation metrics to compre…
Hadamard Adapter: An Extreme Parameter-Efficient Adapter Tuning Method for Pre-trained Language Models
Yuyan Chen, Qiang Fu, Ge Fan +6
Recent years, Pre-trained Language models (PLMs) have swept into various fields of artificial intelligence and achieved great success. However, most PLMs, such as T5 and GPT3, have…
AnaMeta: A Table Understanding Dataset of Field Metadata Knowledge Shared by Multi-dimensional Data Analysis Tasks
Xinyi He, Mengyu Zhou, Mingjie Zhou +7
Tabular data analysis is performed every day across various domains. It requires an accurate understanding of field semantics to correctly operate on table fields and find common p…
Neuron Campaign for Initialization Guided by Information Bottleneck Theory
Haitao Mao, Xu Chen, Qiang Fu +3
Initialization plays a critical role in the training of deep neural networks (DNN). Existing initialization strategies mainly focus on stabilizing the training process to mitigate…
LUNA: Language Understanding with Number Augmentations on Transformers via Number Plugins and Pre-training
Hongwei Han, Jialiang Xu, Mengyu Zhou +3
Transformers are widely used in NLP tasks. However, current approaches to leveraging transformers to understand language expose one weak spot: Number understanding. In some scenari…
MMTU: A Massive Multi-Task Table Understanding and Reasoning Benchmark
Junjie Xing, Yeye He, Mengyu Zhou +6
Tables and table-based use cases play a crucial role in many important real-world applications, such as spreadsheets, databases, and computational notebooks, which traditionally re…
Text2Analysis: A Benchmark of Table Question Answering with Advanced Data Analysis and Unclear Queries
Xinyi He, Mengyu Zhou, Xinrun Xu +9
Tabular data analysis is crucial in various fields, and large language models show promise in this area. However, current research mostly focuses on rudimentary tasks like Text2SQL…
From Task to Tutorial: An Automated GUI Framework for Excel Tutorial Document and Video Creation
Yuhang Xie, Jian Mu, Xiaojun Ma +9
Excel is one of the most widely used productivity tools across domains, offering rich functionality but also overwhelming users with its complexity. This creates a persistent deman…
On Manipulating Signals of User-Item Graph: A Jacobi Polynomial-based Graph Collaborative Filtering
Jiayan Guo, Lun Du, Xu Chen +5
Collaborative filtering (CF) is an important research direction in recommender systems that aims to make recommendations given the information on user-item interactions. Graph CF h…
Table Meets LLM: Can Large Language Models Understand Structured Table Data? A Benchmark and Empirical Study
Yuan Sui, Mengyu Zhou, Mingjie Zhou +2
Large language models (LLMs) are becoming attractive as few-shot reasoners to solve Natural Language (NL)-related tasks. However, the understanding of their capability to process s…
Demonstration of InsightPilot: An LLM-Empowered Automated Data Exploration System
Pingchuan Ma, Rui Ding, Shuai Wang +2
Exploring data is crucial in data analysis, as it helps users understand and interpret the data more effectively. However, performing effective data exploration requires in-depth k…
Scalable Differentiable Causal Discovery in the Presence of Latent Confounders with Skeleton Posterior (Extended Version)
Pingchuan Ma, Rui Ding, Qiang Fu +4
Differentiable causal discovery has made significant advancements in the learning of directed acyclic graphs. However, its application to real-world datasets remains restricted due…
On the Evaluation of Neural Code Summarization
Ensheng Shi, Yanlin Wang, Lun Du +5
Source code summaries are important for program comprehension and maintenance. However, there are plenty of programs with missing, outdated, or mismatched summaries. Recently, deep…
Towards Robust Numerical Question Answering: Diagnosing Numerical Capabilities of NLP Systems
Jialiang Xu, Mengyu Zhou, Xinyi He +2
Numerical Question Answering is the task of answering questions that require numerical capabilities. Previous works introduce general adversarial attacks to Numerical Question Answ…
SoTaNa: The Open-Source Software Development Assistant
Ensheng Shi, Fengji Zhang, Yanlin Wang +6
Software development plays a crucial role in driving innovation and efficiency across modern societies. To meet the demands of this dynamic field, there is a growing need for an ef…
TUTA: Tree-based Transformers for Generally Structured Table Pre-training
Zhiruo Wang, Haoyu Dong, Ran Jia +4
Tables are widely used with various structures to organize and present data. Recent attempts on table understanding mainly focus on relational tables, yet overlook to other common…
XInsight: eXplainable Data Analysis Through The Lens of Causality
Pingchuan Ma, Rui Ding, Shuai Wang +2
In light of the growing popularity of Exploratory Data Analysis (EDA), understanding the underlying causes of the knowledge acquired by EDA is crucial. However, it remains under-re…
HiTab: A Hierarchical Table Dataset for Question Answering and Natural Language Generation
Zhoujun Cheng, Haoyu Dong, Zhiruo Wang +6
Tables are often created with hierarchies, but existing works on table reasoning mainly focus on flat tables and neglect hierarchical tables. Hierarchical tables challenge existing…
CoCoSum: Contextual Code Summarization with Multi-Relational Graph Neural Network
Yanlin Wang, Ensheng Shi, Lun Du +5
Source code summaries are short natural language descriptions of code snippets that help developers better understand and maintain source code. There has been a surge of work on au…
Formula-R1: Incentivizing LLM Reasoning over Complex Tables with Numerical Computation via Formula-Driven Reinforcement Learning
Lang Cao, Jingxian Xu, Hanbing Liu +5
Tables are a fundamental medium for organizing and analyzing data, making table reasoning a critical capability for intelligent systems. Although large language models (LLMs) exhib…
CAST: Achieving Stable LLM-based Text Analysis for Data Analytics
Jinxiang Xie, Zihao Li, Wei He +3
Text analysis of tabular data relies on two core operations: \emph{summarization} for corpus-level theme extraction and \emph{tagging} for row-level labeling. A critical limitation…
Understanding and Improvement of Adversarial Training for Network Embedding from an Optimization Perspective
Lun Du, Xu Chen, Fei Gao +4
Network Embedding aims to learn a function mapping the nodes to Euclidean space contribute to multiple learning analysis tasks on networks. However, the noisy information behind th…
Precise Condition Synthesis for Program Repair
Yingfei Xiong, Jie Wang, Runfa Yan +4
Due to the difficulty of repairing defect, many research efforts have been devoted into automatic defect repair. Given a buggy program that fails some test cases, a typical automat…
Neuron with Steady Response Leads to Better Generalization
Qiang Fu, Lun Du, Haitao Mao +4
Regularization can mitigate the generalization gap between training and inference by introducing inductive bias. Existing works have already proposed various inductive biases from…
TableLoRA: Low-rank Adaptation on Table Structure Understanding for Large Language Models
Xinyi He, Yihao Liu, Mengyu Zhou +5
Tabular data are crucial in many fields and their understanding by large language models (LLMs) under high parameter efficiency paradigm is important. However, directly applying pa…
Extract Information from Hybrid Long Documents Leveraging LLMs: A Framework and Dataset
Chongjian Yue, Xinrun Xu, Xiaojun Ma +5
Large Language Models (LLMs) demonstrate exceptional performance in textual understanding and tabular reasoning tasks. However, their ability to comprehend and analyze hybrid text,…
Table Pre-training: A Survey on Model Architectures, Pre-training Objectives, and Downstream Tasks
Haoyu Dong, Zhoujun Cheng, Xinyi He +6
Since a vast number of tables can be easily collected from web pages, spreadsheets, PDFs, and various other document types, a flurry of table pre-training frameworks have been prop…
TwT: Thinking without Tokens by Habitual Reasoning Distillation with Multi-Teachers' Guidance
Jingxian Xu, Mengyu Zhou, Weichang Liu +3
Large Language Models (LLMs) have made significant strides in problem-solving by incorporating reasoning processes. However, this enhanced reasoning capability results in an increa…
Make Heterophily Graphs Better Fit GNN: A Graph Rewiring Approach
Wendong Bi, Lun Du, Qiang Fu +3
Graph Neural Networks (GNNs) are popular machine learning methods for modeling graph data. A lot of GNNs perform well on homophily graphs while having unsatisfactory performance on…
Accelerating Code Search with Deep Hashing and Code Classification
Wenchao Gu, Yanlin Wang, Lun Du +4
Code search is to search reusable code snippets from source code corpus based on natural languages queries. Deep learning-based methods of code search have shown promising results.…
FormLM: Recommending Creation Ideas for Online Forms by Modelling Semantic and Structural Information
Yijia Shao, Mengyu Zhou, Yifan Zhong +5
Online forms are widely used to collect data from human and have a multi-billion market. Many software products provide online services for creating semi-structured forms where que…
Homophily-oriented Heterogeneous Graph Rewiring
Jiayan Guo, Lun Du, Wendong Bi +6
With the rapid development of the World Wide Web (WWW), heterogeneous graphs (HG) have explosive growth. Recently, heterogeneous graph neural network (HGNN) has shown great potenti…
On the Evaluation of Commit Message Generation Models: An Experimental Study
Wei Tao, Yanlin Wang, Ensheng Shi +5
Commit messages are natural language descriptions of code changes, which are important for program understanding and maintenance. However, writing commit messages manually is time-…
Test Time Training for Supervised Causal Learning
Zizhen Deng, Jiaru Zhang, Rui Ding +5
Supervised Causal Learning (SCL) has shown promise in causal discovery by framing it as a supervised learning problem. However, it suffers from significant out-of-distribution gene…
Table-LLM-Specialist: Language Model Specialists for Tables using Iterative Generator-Validator Fine-tuning
Junjie Xing, Yeye He, Mengyu Zhou +4
Language models such as GPT and Llama have shown remarkable ability on diverse natural language tasks, yet their performance on complex table tasks (e.g., NL-to-Code and data clean…
Revisiting Code Search in a Two-Stage Paradigm
Fan Hu, Yanlin Wang, Lun Du +4
With a good code search engine, developers can reuse existing code snippets and accelerate software development process. Current code search methods can be divided into two categor…
TabularNet: A Neural Network Architecture for Understanding Semantic Structures of Tabular Data
Lun Du, Fei Gao, Xu Chen +5
Tabular data are ubiquitous for the widespread applications of tables and hence have attracted the attention of researchers to extract underlying information. One of the critical p…
ASTA: Learning Analytical Semantics over Tables for Intelligent Data Analysis and Visualization
Lingbo Li, Tianle Li, Xinyi He +3
Intelligent analysis and visualization of tables use techniques to automatically recommend useful knowledge from data, thus freeing users from tedious multi-dimension data mining.…
Auto-Fill: Learning to Predict Missing Values Accurately with Specialist Language Models
Yurong Liu, Yeye He, Haoyu Dong +4
Predicting missing cell values in tabular data is a fundamental problem in data cleaning. While state-of-the-art reasoning models show great promise in predicting missing values in…
Professional Network Matters: Connections Empower Person-Job Fit
Hao Chen, Lun Du, Yuxuan Lu +6
Online recruitment platforms typically employ Person-Job Fit models in the core service that automatically match suitable job seekers with appropriate job positions. While existing…
Is a Single Model Enough? MuCoS: A Multi-Model Ensemble Learning for Semantic Code Search
Lun Du, Xiaozhou Shi, Yanlin Wang +3
Recently, deep learning methods have become mainstream in code search since they do better at capturing semantic correlations between code snippets and search queries and have prom…
ML4C: Seeing Causality Through Latent Vicinity
Haoyue Dai, Rui Ding, Yuanyuan Jiang +2
Supervised Causal Learning (SCL) aims to learn causal relations from observational data by accessing previously seen datasets associated with ground truth causal relations. This pa…
Robust Mid-Pass Filtering Graph Convolutional Networks
Jincheng Huang, Lun Du, Xu Chen +3
Graph convolutional networks (GCNs) are currently the most promising paradigm for dealing with graph-structure data, while recent studies have also shown that GCNs are vulnerable t…
Causal-Based Supervision of Attention in Graph Neural Network: A Better and Simpler Choice towards Powerful Attention
Hongjun Wang, Jiyuan Chen, Lun Du +3
Recent years have witnessed the great potential of attention mechanism in graph representation learning. However, while variants of attention-based GNNs are setting new benchmarks…
SpreadsheetLLM: Encoding Spreadsheets for Large Language Models
Haoyu Dong, Jianbo Zhao, Yuzhang Tian +8
Spreadsheets are characterized by their extensive two-dimensional grids, flexible layouts, and varied formatting options, which pose significant challenges for large language model…
Automated Refactoring of Nested-IF Formulae in Spreadsheets
Jie Zhang, Shi Han, Dan Hao +2
Spreadsheets are the most popular end-user programming software, where formulae act like programs and also have smells. One well recognized common smell of spreadsheet formulae is…
Learning Identifiable Structures Helps Avoid Bias in DNN-based Supervised Causal Learning
Jiaru Zhang, Rui Ding, Qiang Fu +6
Causal discovery is a structured prediction task that aims to predict causal relations among variables based on their data samples. Supervised Causal Learning (SCL) is an emerging…
TAROT: A Hierarchical Framework with Multitask Co-Pretraining on Semi-Structured Data towards Effective Person-Job Fit
Yihan Cao, Xu Chen, Lun Du +7
Person-job fit is an essential part of online recruitment platforms in serving various downstream applications like Job Search and Candidate Recommendation. Recently, pretrained la…