210 citations · 671 across the 21 of their papers we have counts for
23 papers
Overcoming Long-Context Limitations of State-Space Models via Context-Dependent Sparse Attention
Zhihao Zhan, Jianan Zhao, Zhaocheng Zhu +1
Efficient long-context modeling remains a critical challenge for natural language processing (NLP), as the time complexity of the predominant Transformer architecture scales quadra…
From Passive to Active Reasoning: Can Large Language Models Ask the Right Questions under Incomplete Information?
Zhanke Zhou, Xiao Feng, Zhaocheng Zhu +3
While existing benchmarks probe the reasoning abilities of large language models (LLMs) across diverse domains, they predominantly assess passive reasoning, providing models with a…
Landscape of Thoughts: Visualizing the Reasoning Process of Large Language Models
Zhanke Zhou, Zhaocheng Zhu, Xuan Li +5
Numerous applications of large language models (LLMs) rely on their ability to perform step-by-step reasoning. However, the reasoning behavior of LLMs remains poorly understood, po…
Learning Representations for Reasoning: Generalizing Across Diverse Structures
Zhaocheng Zhu
Reasoning, the ability to logically draw conclusions from existing knowledge, is a hallmark of human. Together with perception, they constitute the two major themes of artificial i…
Fully-inductive Node Classification on Arbitrary Graphs
Jianan Zhao, Zhaocheng Zhu, Mikhail Galkin +3
One fundamental challenge in graph machine learning is generalizing to new graphs. Many existing methods following the inductive setup can generalize to test graphs with new struct…
A Foundation Model for Zero-shot Logical Query Reasoning
Mikhail Galkin, Jincheng Zhou, Bruno Ribeiro +2
Complex logical query answering (CLQA) in knowledge graphs (KGs) goes beyond simple KG completion and aims at answering compositional queries comprised of multiple projections and…
Towards Foundational Models for Molecular Learning on Large-Scale Multi-Task Datasets
Dominique Beaini, Shenyang Huang, Joao Alex Cunha +32
Recently, pre-trained foundation models have enabled significant advancements in multiple fields. In molecular machine learning, however, where datasets are often hand-curated, and…
GraphText: Graph Reasoning in Text Space
Jianan Zhao, Le Zhuo, Yikang Shen +5
Large Language Models (LLMs) have gained the ability to assimilate human knowledge and facilitate natural language interactions with both humans and other LLMs. However, despite th…
Large Language Models can Learn Rules
Zhaocheng Zhu, Yuan Xue, Xinyun Chen +4
When prompted with a few examples and intermediate steps, large language models (LLMs) have demonstrated impressive performance in various reasoning tasks. However, prompting metho…
Towards Foundation Models for Knowledge Graph Reasoning
Mikhail Galkin, Xinyu Yuan, Hesham Mostafa +2
Foundation models in language and vision have the ability to run inference on any textual and visual inputs thanks to the transferable representations such as a vocabulary of token…
Neural Graph Reasoning: Complex Logical Query Answering Meets Graph Databases
Hongyu Ren, Mikhail Galkin, Michael Cochez +2
Complex logical query answering (CLQA) is a recently emerged task of graph machine learning that goes beyond simple one-hop link prediction and solves a far more complex task of mu…
Inductive Logical Query Answering in Knowledge Graphs
Mikhail Galkin, Zhaocheng Zhu, Hongyu Ren +1
Formulating and answering logical queries is a standard communication interface for knowledge graphs (KGs). Alleviating the notorious incompleteness of real-world KGs, neural metho…
A*Net: A Scalable Path-based Reasoning Approach for Knowledge Graphs
Zhaocheng Zhu, Xinyu Yuan, Mikhail Galkin +4
Reasoning on large-scale knowledge graphs has been long dominated by embedding methods. While path-based methods possess the inductive capacity that embeddings lack, their scalabil…
PEER: A Comprehensive and Multi-Task Benchmark for Protein Sequence Understanding
Minghao Xu, Zuobai Zhang, Jiarui Lu +5
We are now witnessing significant progress of deep learning methods in a variety of tasks (or datasets) of proteins. However, there is a lack of a standard benchmark to evaluate th…
Neural-Symbolic Models for Logical Queries on Knowledge Graphs
Zhaocheng Zhu, Mikhail Galkin, Zuobai Zhang +1
Answering complex first-order logic (FOL) queries on knowledge graphs is a fundamental task for multi-hop reasoning. Traditional symbolic methods traverse a complete knowledge grap…
TorchDrug: A Powerful and Flexible Machine Learning Platform for Drug Discovery
Zhaocheng Zhu, Chence Shi, Zuobai Zhang +12
Machine learning has huge potential to revolutionize the field of drug discovery and is attracting increasing attention in recent years. However, lacking domain knowledge (e.g., wh…
Neural Bellman-Ford Networks: A General Graph Neural Network Framework for Link Prediction
Zhaocheng Zhu, Zuobai Zhang, Louis-Pascal Xhonneux +1
Link prediction is a very fundamental task on graphs. Inspired by traditional path-based methods, in this paper we propose a general and flexible representation learning framework…
GraphAF: a Flow-based Autoregressive Model for Molecular Graph Generation
Chence Shi, Minkai Xu, Zhaocheng Zhu +3
Molecular graph generation is a fundamental problem for drug discovery and has been attracting growing attention. The problem is challenging since it requires not only generating c…
KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation
Xiaozhi Wang, Tianyu Gao, Zhaocheng Zhu +4
Pre-trained language representation models (PLMs) cannot well capture factual knowledge from text. In contrast, knowledge embedding (KE) methods can effectively represent the relat…
Self-Adaptive Network Pruning
Jinting Chen, Zhaocheng Zhu, Cheng Li +1
Deep convolutional neural networks have been proved successful on a wide range of tasks, yet they are still hindered by their large computation cost in many industrial scenarios. I…
GraphVite: A High-Performance CPU-GPU Hybrid System for Node Embedding
Zhaocheng Zhu, Shizhen Xu, Meng Qu +1
Learning continuous representations of nodes is attracting growing interest in both academia and industry recently, due to their simplicity and effectiveness in a variety of applic…
Saliency Supervision: An Intuitive and Effective Approach for Pain Intensity Regression
Conghui Li, Zhaocheng Zhu, Yuming Zhao
Getting pain intensity from face images is an important problem in autonomous nursing systems. However, due to the limitation in data sources and the subjectiveness in pain intensi…
Context Aware Document Embedding
Zhaocheng Zhu, Junfeng Hu
Recently, doc2vec has achieved excellent results in different tasks. In this paper, we present a context aware variant of doc2vec. We introduce a novel weight estimating mechanism…