924 citations · 1.5k across the 16 of their papers we have counts for
16 papers
StreamTalk: Streaming Co-Speech Gesture Generation with Key-Pose Anchoring
Xiangyue Zhang, Jianfang Li, Jiaxu Zhang +2
Real-time co-speech gesture generation must produce 3D motion clip by clip as speech arrives. Existing streaming methods are open-loop: each clip depends on past context, but the m…
Personalised Distillation: Empowering Open-Sourced LLMs with Adaptive Learning for Code Generation
Hailin Chen, Amrita Saha, Steven Hoi +1
With the rise of powerful closed-sourced LLMs (ChatGPT, GPT-4), there are increasing interests in distilling the capabilies of close-sourced LLMs to smaller open-sourced LLMs. Prev…
RAP-Gen: Retrieval-Augmented Patch Generation with CodeT5 for Automatic Program Repair
Weishi Wang, Yue Wang, Shafiq Joty +1
Automatic program repair (APR) is crucial to reduce manual debugging efforts for developers and improve software reliability. While conventional search-based techniques typically r…
BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editing
Dongxu Li, Junnan Li, Steven C. H. Hoi
Subject-driven text-to-image generation models create novel renditions of an input subject based on text prompts. Existing models suffer from lengthy fine-tuning and difficulties p…
InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning
Wenliang Dai, Junnan Li, Dongxu Li +6
Large-scale pre-training and instruction tuning have been successful at creating general-purpose language models with broad competence. However, building general-purpose vision-lan…
CodeTF: One-stop Transformer Library for State-of-the-art Code LLMs
Nghi D. Q. Bui, Hung Le, Yue Wang +3
Code intelligence plays a key role in transforming modern software engineering. Recently, deep learning-based models, especially Transformer-based large language models (LLMs), hav…
CodeT5+: Open Code Large Language Models for Code Understanding and Generation
Yue Wang, Hung Le, Akhilesh Deepak Gotmare +3
Large language models (LLMs) pretrained on vast source code have achieved prominent progress in code intelligence. However, existing code LLMs have two main limitations in terms of…
LogAI: A Library for Log Analytics and Intelligence
Qian Cheng, Amrita Saha, Wenzhuo Yang +3
Software and System logs record runtime information about processes executing within a system. These logs have become the most critical and ubiquitous forms of observability data t…
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
Junnan Li, Dongxu Li, Silvio Savarese +1
The cost of vision-and-language pre-training has become increasingly prohibitive due to end-to-end training of large-scale models. This paper proposes BLIP-2, a generic and efficie…
Detect-Localize-Repair: A Unified Framework for Learning to Debug with CodeT5
Nghi D. Q. Bui, Yue Wang, Steven Hoi
Automated software debugging is a crucial task for improving the productivity of software developers. Many neural-based techniques have been proven effective for debugging-related…
Learning Label Modular Prompts for Text Classification in the Wild
Hailin Chen, Amrita Saha, Shafiq Joty +1
Machine learning models usually assume i.i.d data during training and testing, but data and tasks in real world often change over time. To emulate the transient nature of real worl…
From Images to Textual Prompts: Zero-shot VQA with Frozen Large Language Models
Jiaxian Guo, Junnan Li, Dongxu Li +4
Large language models (LLMs) have demonstrated excellent zero-shot generalization to new language tasks. However, effective utilization of LLMs for zero-shot visual question-answer…
BotSIM: An End-to-End Bot Simulation Toolkit for Commercial Task-Oriented Dialog Systems
Guangsen Wang, Shafiq Joty, Junnan Li +1
We introduce BotSIM, a modular, open-source Bot SIMulation environment with dialog generation, user simulation and conversation analytics capabilities. BotSIM aims to serve as a on…
BotSIM: An End-to-End Bot Simulation Framework for Commercial Task-Oriented Dialog Systems
Guangsen Wang, Samson Tan, Shafiq Joty +3
We present BotSIM, a data-efficient end-to-end Bot SIMulation toolkit for commercial text-based task-oriented dialog (TOD) systems. BotSIM consists of three major components: 1) a…
Masked Unsupervised Self-training for Label-free Image Classification
Junnan Li, Silvio Savarese, Steven C. H. Hoi
State-of-the-art computer vision models are mostly trained with supervised learning using human-labeled images, which limits their scalability due to the expensive annotation cost.…
Dynamic Fusion with Intra- and Inter- Modality Attention Flow for Visual Question Answering
Gao Peng, Zhengkai Jiang, Haoxuan You +4
Learning effective fusion of multi-modality features is at the heart of visual question answering. We propose a novel method of dynamically fusing multi-modal features with intra-…