6 citations · 8 across the 17 of their papers we have counts for
17 papers
PUe: Biased Positive-Unlabeled Learning Enhancement by Causal Inference
Xutao Wang, Hanting Chen, Tianyu Guo +1
Positive-Unlabeled (PU) learning aims to achieve high-accuracy binary classification with limited labeled positive examples and numerous unlabeled ones. Existing cost-sensitive-bas…
VersatileFFN: Achieving Parameter Efficiency in LLMs via Adaptive Wide-and-Deep Reuse
Ying Nie, Kai Han, Hongguang Li +5
The rapid scaling of Large Language Models (LLMs) has achieved remarkable performance, but it also leads to prohibitive memory costs. Existing parameter-efficient approaches such a…
From Next-Token to Next-Block: A Principled Adaptation Path for Diffusion LLMs
Yuchuan Tian, Yuchen Liang, Shuo Zhang +10
Diffusion Language Models (DLMs) enable fast generation, yet training large DLMs from scratch is costly. As a practical shortcut, adapting off-the-shelf Auto-Regressive (AR) model…
MoRAgent: Parameter Efficient Agent Tuning with Mixture-of-Roles
Jing Han, Binwei Yan, Tianyu Guo +4
Despite recent advancements of fine-tuning large language models (LLMs) to facilitate agent tasks, parameter-efficient fine-tuning (PEFT) methodologies for agent remain largely une…
Nexus: Higher-Order Attention Mechanisms in Transformers
Hanting Chen, Chong Zhu, Kai Han +6
Transformers have achieved significant success across various domains, relying on self-attention to capture dependencies. However, the standard first-order attention mechanism is o…
Revealing the Power of Post-Training for Small Language Models via Knowledge Distillation
Miao Rang, Zhenni Bi, Hang Zhou +6
The rapid advancement of large language models (LLMs) has significantly advanced the capabilities of artificial intelligence across various domains. However, their massive scale an…
Pangu Ultra: Pushing the Limits of Dense Large Language Models on Ascend NPUs
Yichun Yin, Wenyong Huang, Kaikai Song +49
We present Pangu Ultra, a Large Language Model (LLM) with 135 billion parameters and dense Transformer modules trained on Ascend Neural Processing Units (NPUs). Although the field…
Unshackling Context Length: An Efficient Selective Attention Approach through Query-Key Compression
Haoyu Wang, Tong Teng, Tianyu Guo +4
Handling long-context sequences efficiently remains a significant challenge in large language models (LLMs). Existing methods for token selection in sequence extrapolation either e…
Cross-video Identity Correlating for Person Re-identification Pre-training
Jialong Zuo, Ying Nie, Hanyu Zhou +5
Recent researches have proven that pre-training on large-scale person images extracted from internet videos is an effective way in learning better representations for person re-ide…
CFinBench: A Comprehensive Chinese Financial Benchmark for Large Language Models
Ying Nie, Binwei Yan, Tianyu Guo +9
Large language models (LLMs) have achieved remarkable performance on various NLP tasks, yet their potential in more challenging and domain-specific task, such as finance, has not b…
DenseMamba: State Space Models with Dense Hidden Connection for Efficient Large Language Models
Wei He, Kai Han, Yehui Tang +4
Large language models (LLMs) face a daunting challenge due to the excessive computational and memory requirements of the commonly used Transformer architecture. While state space m…
A robust audio deepfake detection system via multi-view feature
Yujie Yang, Haochen Qin, Hang Zhou +4
With the advancement of generative modeling techniques, synthetic human speech becomes increasingly indistinguishable from real, and tricky challenges are elicited for the audio de…
PanGu-: Enhancing Language Model Architectures via Nonlinearity Compensation
Yunhe Wang, Hanting Chen, Yehui Tang +17
The recent trend of large language models (LLMs) is to increase the scale of both model size (\aka the number of parameters) and dataset to achieve better generative ability, which…
UFineBench: Towards Text-based Person Retrieval with Ultra-fine Granularity
Jialong Zuo, Hanyu Zhou, Ying Nie +5
Existing text-based person retrieval datasets often have relatively coarse-grained text annotations. This hinders the model to comprehend the fine-grained semantics of query texts…
LightCLIP: Learning Multi-Level Interaction for Lightweight Vision-Language Models
Ying Nie, Wei He, Kai Han +4
Vision-language pre-training like CLIP has shown promising performance on various downstream tasks such as zero-shot image classification and image-text retrieval. Most of the exis…
Towards Higher Ranks via Adversarial Weight Pruning
Yuchuan Tian, Hanting Chen, Tianyu Guo +2
Convolutional Neural Networks (CNNs) are hard to deploy on edge devices due to its high computation and storage complexities. As a common practice for model compression, network pr…
Data-Free Distillation of Language Model by Text-to-Text Transfer
Zheyuan Bai, Xinduo Liu, Hailin Hu +3
Data-Free Knowledge Distillation (DFKD) plays a vital role in compressing the model when original training data is unavailable. Previous works for DFKD in NLP mainly focus on disti…