46 citations · 74 across the 23 of their papers we have counts for
25 papers
VideoLatent: Video-Language Learning via Latent Self-Forcing
Zi-Yuan Hu, Zicong Tang, Shijia Huang +3
Recent advancements in chain-of-thought (CoT) reasoning have shown promise in enhancing video understanding and reasoning capabilities of multimodal large language models (MLLMs).…
Membership Inference on LLMs in the Wild
Jiatong Yi, Yanyang Li
Membership Inference Attacks (MIAs) act as a crucial auditing tool for the opaque training data of Large Language Models (LLMs). However, existing techniques predominantly rely on…
NeMo: Needle in a Montage for Video-Language Understanding
Zi-Yuan Hu, Shuo Liang, Duo Zheng +10
Recent advances in video large language models (VideoLLMs) call for new evaluation protocols and benchmarks for video-language understanding. Inspired by the needle in a haystack t…
Learning from Videos for 3D World: Enhancing MLLMs with 3D Vision Geometry Priors
Duo Zheng, Shijia Huang, Yanyang Li +1
Previous research has investigated the application of Multimodal Large Language Models (MLLMs) in understanding 3D scenes by interpreting them as videos. These approaches generally…
Learning to Reason from Feedback at Test-Time
Yanyang Li, Michael Lyu, Liwei Wang
Solving complex tasks in a single attempt is challenging for large language models (LLMs). Iterative interaction with the environment and feedback is often required to achieve succ…
CLEVA: Toward Comprehensive and Contamination-Free Language Model Evaluation
Yanyang Li, Tin Long Wong, Cheung To Hung +5
Recent advances in large language models (LLMs) have shown significant promise, yet their evaluation raises concerns, particularly regarding data contamination due to the lack of a…
Making Long-Context Language Models Better Multi-Hop Reasoners
Yanyang Li, Shuo Liang, Michael R. Lyu +1
Recent advancements in long-context modeling have enhanced language models (LMs) for complex tasks across multiple NLP applications. Despite this progress, we find that these model…
VL-PET: Vision-and-Language Parameter-Efficient Tuning via Granularity Control
Zi-Yuan Hu, Yanyang Li, Michael R. Lyu +1
As the model size of pre-trained language models (PLMs) grows rapidly, full fine-tuning becomes prohibitively expensive for model training and storage. In vision-and-language (VL),…
CLEVA: Chinese Language Models EVAluation Platform
Yanyang Li, Jianqiao Zhao, Duo Zheng +8
With the continuous emergence of Chinese Large Language Models (LLMs), how to evaluate a model's capabilities has become an increasingly significant issue. The absence of a compreh…
Multi-Path Transformer is Better: A Case Study on Neural Machine Translation
Ye Lin, Shuhan Zhou, Yanyang Li +3
For years the model performance in machine learning obeyed a power-law relationship with the model size. For the consideration of parameter efficiency, recent studies focus on incr…
SPRING: Situated Conversation Agent Pretrained with Multimodal Questions from Incremental Layout Graph
Yuxing Long, Binyuan Hui, Fulong Ye +5
Existing multimodal conversation agents have shown impressive abilities to locate absolute positions or retrieve attributes in simple scenarios, but they fail to perform well when…
Eliciting Knowledge from Large Pre-Trained Models for Unsupervised Knowledge-Grounded Conversation
Yanyang Li, Jianqiao Zhao, Michael R. Lyu +1
Recent advances in large-scale pre-training provide large models with the potential to learn knowledge from the raw text. It is thus natural to ask whether it is possible to levera…
Probing Structured Pruning on Multilingual Pre-trained Models: Settings, Algorithms, and Efficiency
Yanyang Li, Fuli Luo, Runxin Xu +3
Structured pruning has been extensively studied on monolingual pre-trained language models and is yet to be fully evaluated on their multilingual counterparts. This work investigat…
Knowledgeable Salient Span Mask for Enhancing Language Models as Knowledge Base
Cunxiang Wang, Fuli Luo, Yanyang Li +3
Pre-trained language models (PLMs) like BERT have made significant progress in various downstream NLP tasks. However, by asking models to do cloze-style tests, recent work finds th…
SSQL: Injecting Syntax to Question-Schema Interaction Graph Encoder for Text-to-SQL Parsers
Binyuan Hui, Ruiying Geng, Lihan Wang +4
The task of converting a natural language question into an executable SQL query, known as text-to-SQL, is an important branch of semantic parsing. The state-of-the-art graph-based…
FlowEval: A Consensus-Based Dialogue Evaluation Framework Using Segment Act Flows
Jianqiao Zhao, Yanyang Li, Wanyu Du +4
Despite recent progress in open-domain dialogue evaluation, how to develop automatic metrics remains an open problem. We explore the potential of dialogue evaluation featuring dial…
The NiuTrans System for WNGT 2020 Efficiency Task
Chi Hu, Bei Li, Ye Lin +5
This paper describes the submissions of the NiuTrans Team to the WNGT 2020 Efficiency Shared Task. We focus on the efficient implementation of deep Transformer models \cite{wang-et…
Bag of Tricks for Optimizing Transformer Efficiency
Ye Lin, Yanyang Li, Tong Xiao +1
Improving Transformer efficiency has become increasingly attractive recently. A wide range of methods has been proposed, e.g., pruning, quantization, new architectures and etc. But…
Stacked Acoustic-and-Textual Encoding: Integrating the Pre-trained Models into Speech Translation Encoders
Chen Xu, Bojie Hu, Yanyang Li +5
Encoder pre-training is promising in end-to-end Speech Translation (ST), given the fact that speech-to-translation data is scarce. But ST encoders are not simple instances of Autom…
An Efficient Transformer Decoder with Compressed Sub-layers
Yanyang Li, Ye Lin, Tong Xiao +1
The large attention-based encoder-decoder network (Transformer) has become prevailing recently due to its effectiveness. But the high computation complexity of its decoder raises t…
A Simple and Effective Approach to Robust Unsupervised Bilingual Dictionary Induction
Yanyang Li, Yingfeng Luo, Ye Lin +5
Unsupervised Bilingual Dictionary Induction methods based on the initialization and the self-learning have achieved great success in similar language pairs, e.g., English-Spanish.…
Towards Fully 8-bit Integer Inference for the Transformer Model
Ye Lin, Yanyang Li, Tengbo Liu +3
8-bit integer inference, as a promising direction in reducing both the latency and storage of deep neural networks, has made great progress recently. On the other hand, previous sy…
Weight Distillation: Transferring the Knowledge in Neural Network Parameters
Ye Lin, Yanyang Li, Ziyang Wang +4
Knowledge distillation has been proven to be effective in model acceleration and compression. It allows a small network to learn to generalize in the same way as a large network. R…
Neural Machine Translation with Joint Representation
Yanyang Li, Qiang Wang, Tong Xiao +2
Though early successes of Statistical Machine Translation (SMT) systems are attributed in part to the explicit modelling of the interaction between any two source and target units,…