Publications (101)
Beyond Instructional Videos: Probing for More Diverse Visual-Textual Grounding on YouTube
Jack Hessel, Zhenhai Zhu, Bo Pang +1
Pretraining from unlabelled web videos has quickly become the de-facto means of achieving high performance on many video understanding tasks. Features are learned via prediction of…
The effect of wording on message propagation: Topic- and author-controlled natural experiments on Twitter
Chenhao Tan, Lillian Lee, Bo Pang
Consider a person trying to spread an important message on a social network. He/she can spend hours trying to craft the message. Does it actually matter? While there has been exten…
A Minimalist Approach to LLM Reasoning: from Rejection Sampling to Reinforce
Wei Xiong, Jiarui Yao, Yuhui Xu +8
Reinforcement learning (RL) has become a prevailing approach for fine-tuning large language models (LLMs) on complex reasoning tasks. Among recent methods, GRPO stands out for its…
Self-Abstraction from Grounded Experience for Plan-Guided Policy Refinement
Hiroaki Hayashi, Bo Pang, Wenting Zhao +6
Large language model (LLM) based agents are increasingly used to tackle software engineering tasks that require multi-step reasoning and code modification, demonstrating promising…
Procedural Memory Distillation: Online Reflection for Self-Improving Language Models
Ye Liu, Srijan Bansal, Bo Pang +6
Reinforcement learning with verifiable rewards (RLVR), along with recent selfdistillation variants such as SDPO, evaluates each rollout against a verifier and updates the policy fr…
Learning the Geodesic Embedding with Graph Neural Networks
Bo Pang, Zhongtian Zheng, Guoping Wang +1
We present GeGnn, a learning-based method for computing the approximate geodesic distance between two arbitrary points on discrete polyhedra surfaces with constant time complexity…
TDAF: Top-Down Attention Framework for Vision Tasks
Bo Pang, Yizhuo Li, Jiefeng Li +3
Human attention mechanisms often work in a top-down manner, yet it is not well explored in vision research. Here, we propose the Top-Down Attention Framework (TDAF) to capture top-…
MCMC Should Mix: Learning Energy-Based Model with Neural Transport Latent Space MCMC
Erik Nijkamp, Ruiqi Gao, Pavel Sountsov +4
Learning energy-based model (EBM) requires MCMC sampling of the learned model as an inner loop of the learning algorithm. However, MCMC sampling of EBMs in high-dimensional data sp…
Learning Latent Space Energy-Based Prior Model for Molecule Generation
Bo Pang, Tian Han, Ying Nian Wu
Deep generative models have recently been applied to molecule design. If the molecules are encoded in linear SMILES strings, modeling becomes convenient. However, models relying on…
ASAP-Net: Attention and Structure Aware Point Cloud Sequence Segmentation
Hanwen Cao, Yongyi Lu, Cewu Lu +3
Recent works of point clouds show that mulit-frame spatio-temporal modeling outperforms single-frame versions by utilizing cross-frame information. In this paper, we further improv…
Inference-Time Rethinking with Latent Thought Vectors for Math Reasoning
Deqian Kong, Minglu Zhao, Aoyang Qin +10
Standard chain-of-thought reasoning generates a solution in a single forward pass, committing irrevocably to each token and lacking a mechanism to recover from early errors. We int…
Few-shot Unified Question Answering: Tuning Models or Prompts?
Srijan Bansal, Semih Yavuz, Bo Pang +2
Question-answering (QA) tasks often investigate specific question types, knowledge domains, or reasoning skills, leading to specialized models catering to specific categories of QA…
PaLI-X: On Scaling up a Multilingual Vision and Language Model
Xi Chen, Josip Djolonga, Piotr Padlewski +40
We present the training recipe and results of scaling up PaLI-X, a multilingual vision and language model, both in terms of size of the components and the breadth of its training t…
For the sake of simplicity: Unsupervised extraction of lexical simplifications from Wikipedia
Mark Yatskar, Bo Pang, Cristian Danescu-Niculescu-Mizil +1
We report on work in progress on extracting lexical simplifications (e.g., "collaborate" -> "work together"), focusing on utilizing edit histories in Simple English Wikipedia for t…
Robust Transfer Learning with Pretrained Language Models through Adapters
Wenjuan Han, Bo Pang, Yingnian Wu
Transfer learning with large pretrained transformer-based language models like BERT has become a dominating approach for most NLP tasks. Simply fine-tuning those large language mod…
XGen-7B Technical Report
Erik Nijkamp, Tian Xie, Hiroaki Hayashi +22
Large Language Models (LLMs) have become ubiquitous across various domains, transforming the way we interact with information and conduct research. However, most high-performing LL…
Molecule Design by Latent Space Energy-Based Modeling and Gradual Distribution Shifting
Deqian Kong, Bo Pang, Tian Han +1
Generation of molecules with desired chemical and biological properties such as high drug-likeness, high binding affinity to target proteins, is critical for drug discovery. In thi…
Kimi Linear: An Expressive, Efficient Attention Architecture
Kimi Team, Yu Zhang, Zongyu Lin +57
We introduce Kimi Linear, a hybrid linear attention architecture that, for the first time, outperforms full attention under fair comparisons across various scenarios -- including s…
Adaptive Optimal Control of Linear Periodic Systems: An Off-Policy Value Iteration Approach
Bo Pang, Zhong-Ping Jiang
This paper studies the infinite-horizon adaptive optimal control of continuous-time linear periodic (CTLP) systems. A novel value iteration (VI) based off-policy ADP algorithm is p…
Latent Thought Models with Variational Bayes Inference-Time Computation
Deqian Kong, Minglu Zhao, Dehong Xu +8
We propose a novel class of language models, Latent Thought Models (LTMs), which incorporate explicit latent thought vectors that follow an explicit prior model in latent space. Th…
CGNN: Traffic Classification with Graph Neural Network
Bo Pang, Yongquan Fu, Siyuan Ren +3
Traffic classification associates packet streams with known application labels, which is vital for network security and network management. With the rise of NAT, port dynamics, and…
Deep RNN Framework for Visual Sequential Applications
Bo Pang, Kaiwen Zha, Hanwen Cao +2
Extracting temporal and representation features efficiently plays a pivotal role in understanding visual sequence information. To deal with this, we propose a new recurrent neural…
Reasoning Curriculum: Bootstrapping Broad LLM Reasoning from Math
Bo Pang, Deqian Kong, Silvio Savarese +2
Reinforcement learning (RL) can elicit strong reasoning in large language models (LLMs), yet most open efforts focus on math and code. We propose Reasoning Curriculum, a simple two…
PGT: A Progressive Method for Training Models on Long Videos
Bo Pang, Gao Peng, Yizhuo Li +1
Convolutional video models have an order of magnitude larger computational complexity than their counterpart image-level models. Constrained by computational resources, there is no…
RLHF Workflow: From Reward Modeling to Online RLHF
Hanze Dong, Wei Xiong, Bo Pang +7
We present the workflow of Online Iterative Reinforcement Learning from Human Feedback (RLHF) in this technical report, which is widely reported to outperform its offline counterpa…
Semi-supervised Learning by Latent Space Energy-Based Model of Symbol-Vector Coupling
Bo Pang, Erik Nijkamp, Jiali Cui +2
This paper proposes a latent space energy-based prior model for semi-supervised learning. The model stands on a generator network that maps a latent vector to the observed example.…
Decoupled Box Proposal and Featurization with Ultrafine-Grained Semantic Labels Improve Image Captioning and Visual Question Answering
Soravit Changpinyo, Bo Pang, Piyush Sharma +1
Object detection plays an important role in current solutions to vision and language tasks like image captioning and visual question answering. However, popular models like Faster…
Latent Diffusion Energy-Based Model for Interpretable Text Modeling
Peiyu Yu, Sirui Xie, Xiaojian Ma +6
Latent space Energy-Based Models (EBMs), also known as energy-based priors, have drawn growing interests in generative modeling. Fueled by its flexibility in the formulation and st…
Neural Cone Radiosity for Interactive Global Illumination with Glossy Materials
Jierui Ren, Haojie Jin, Bo Pang +3
Modeling of high-frequency outgoing radiance distributions has long been a key challenge in rendering, particularly for glossy material. Such distributions concentrate radiative en…
Diverse and Faithful Knowledge-Grounded Dialogue Generation via Sequential Posterior Inference
Yan Xu, Deqian Kong, Dehong Xu +4
The capability to generate responses with diversity and faithfulness using factual knowledge is paramount for creating a human-like, trustworthy dialogue system. Common strategies…
Semantic Segmentation by Early Region Proxy
Yifan Zhang, Bo Pang, Cewu Lu
Typical vision backbones manipulate structured features. As a compromise, semantic segmentation has long been modeled as per-point prediction on dense regular grids. In this work,…
CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis
Erik Nijkamp, Bo Pang, Hiroaki Hayashi +5
Program synthesis strives to generate a computer program as a solution to a given problem specification, expressed with input-output examples or natural language descriptions. The…
Joint Point Cloud Upsampling and Cleaning with Octree-based CNNs
Jihe Li, Bo Pang, Peng-Shuai Wang
Recovering dense and uniformly distributed point clouds from sparse or noisy data remains a significant challenge. Recently, great progress has been made on these tasks, but usuall…
Get out the vote: Determining support or opposition from Congressional floor-debate transcripts
Matt Thomas, Bo Pang, Lillian Lee
We investigate whether one can determine from the transcripts of U.S. Congressional floor debates whether the speeches represent support of or opposition to proposed legislation. T…
Reinforcement Learning for Adaptive Optimal Stationary Control of Linear Stochastic Systems
Bo Pang, Zhong-Ping Jiang
This paper studies the adaptive optimal stationary control of continuous-time linear stochastic systems with both additive and multiplicative noises, using reinforcement learning t…
NTIRE 2020 Challenge on Video Quality Mapping: Methods and Results
Dario Fuoli, Zhiwu Huang, Martin Danelljan +18
This paper reviews the NTIRE 2020 challenge on video quality mapping (VQM), which addresses the issues of quality mapping from source video domain to target video domain. The chall…
Asynchronous Interaction Aggregation for Action Detection
Jiajun Tang, Jin Xia, Xinzhi Mu +2
Understanding interaction is an essential part of video action detection. We propose the Asynchronous Interaction Aggregation network (AIA) that leverages different interactions to…
Robust Reinforcement Learning: A Case Study in Linear Quadratic Regulation
Bo Pang, Zhong-Ping Jiang
This paper studies the robustness of reinforcement learning algorithms to errors in the learning process. Specifically, we revisit the benchmark problem of discrete-time linear qua…
Emergence of Abstract State Representations in Embodied Sequence Modeling
Tian Yun, Zilai Zeng, Kunal Handa +4
Decision making via sequence modeling aims to mimic the success of language models, where actions taken by an embodied agent are modeled as tokens to predict. Despite their promisi…
LIBRA: Measuring Bias of Large Language Model from a Local Context
Bo Pang, Tingrui Qiao, Caroline Walker +2
Large Language Models (LLMs) have significantly advanced natural language processing applications, yet their widespread use raises concerns regarding inherent biases that may reduc…
CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databases
Tao Yu, Rui Zhang, He Yang Er +21
We present CoSQL, a corpus for building cross-domain, general-purpose database (DB) querying dialogue systems. It consists of 30k+ turns plus 10k+ annotated SQL queries, obtained f…
TubeTK: Adopting Tubes to Track Multi-Object in a One-Step Training Model
Bo Pang, Yizhuo Li, Yifan Zhang +2
Multi-object tracking is a fundamental vision problem that has been studied for a long time. As deep learning brings excellent performances to object detection algorithms, Tracking…
Breaking the Cloak! Unveiling Chinese Cloaked Toxicity with Homophone Graph and Toxic Lexicon
Xuchen Ma, Jianxiang Yu, Wenming Shao +2
Social media platforms have experienced a significant rise in toxic content, including abusive language and discriminatory remarks, presenting growing challenges for content modera…
CLOVER: A Test Case Generation Benchmark with Coverage, Long-Context, and Verification
Jiacheng Xu, Bo Pang, Jin Qu +3
Software testing is a critical aspect of software development, yet generating test cases remains a routine task for engineers. This paper presents a benchmark, CLOVER, to evaluate…
Omni-SMoLA: Boosting Generalist Multimodal Models with Soft Mixture of Low-rank Experts
Jialin Wu, Xia Hu, Yaqing Wang +2
Large multi-modal models (LMMs) exhibit remarkable performance across numerous tasks. However, generalist LMMs often suffer from performance degradation when tuned over a large col…
Robust Policy Iteration for Continuous-time Linear Quadratic Regulation
Bo Pang, Tao Bian, Zhong-Ping Jiang
This paper studies the robustness of policy iteration in the context of continuous-time infinite-horizon linear quadratic regulation (LQR) problem. It is shown that Kleinman's poli…
SParC: Cross-Domain Semantic Parsing in Context
Tao Yu, Rui Zhang, Michihiro Yasunaga +16
We present SParC, a dataset for cross-domainSemanticParsing inContext that consists of 4,298 coherent question sequences (12k+ individual questions annotated with SQL queries). It…
Echoes of power: Language effects and power differences in social interaction
Cristian Danescu-Niculescu-Mizil, Lillian Lee, Bo Pang +1
Understanding social interaction within groups is key to analyzing online communities. Most current work focuses on structural properties: who talks to whom, and how such interacti…
Fully Unsupervised Person Re-identification viaSelective Contrastive Learning
Bo Pang, Deming Zhai, Junjun Jiang +1
Person re-identification (ReID) aims at searching the same identity person among images captured by various cameras. Unsupervised person ReID attracts a lot of attention recently,…
Trajectory Prediction with Latent Belief Energy-Based Model
Bo Pang, Tianyang Zhao, Xu Xie +1
Human trajectory prediction is critical for autonomous platforms like self-driving cars or social robots. We present a latent belief energy-based model (LB-EBM) for diverse human t…
Learning Multi-layer Latent Variable Model via Variational Optimization of Short Run MCMC for Approximate Inference
Erik Nijkamp, Bo Pang, Tian Han +3
This paper studies the fundamental problem of learning deep generative models that consist of multiple layers of latent variables organized in top-down architectures. Such models h…
Diversity Empowers Intelligence: Integrating Expertise of Software Engineering Agents
Kexun Zhang, Weiran Yao, Zuxin Liu +13
Large language model (LLM) agents have shown great potential in solving real-world software engineering (SWE) problems. The most advanced open-source SWE agent can resolve over 27%…
Multimodal Pretraining for Dense Video Captioning
Gabriel Huang, Bo Pang, Zhenhai Zhu +2
Learning specific hands-on skills such as cooking, car maintenance, and home repairs increasingly happens via instructional videos. The user experience with such videos is known to…
Learning Latent Space Energy-Based Prior Model
Bo Pang, Tian Han, Erik Nijkamp +2
We propose to learn energy-based model (EBM) in the latent space of a generator model, so that the EBM serves as a prior model that stands on the top-down network of the generator…
Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales
Bo Pang, Lillian Lee
We address the rating-inference problem, wherein rather than simply decide whether a review is "thumbs up" or "thumbs down", as in previous sentiment analysis work, one must determ…
Two-dimensional Asymptotic Generalized Brillouin Zone Theory
Zeqi Xu, Bo Pang, Kai Zhang +1
In this work, we propose a theory on the two-dimensional non-Hermitian skin effect by resolving two representative minimal models. Specifically, we show that for any given non-Herm…
A Case Study on Combining ASR and Visual Features for Generating Instructional Video Captions
Jack Hessel, Bo Pang, Zhenhai Zhu +1
Instructional videos get high-traffic on video sharing platforms, and prior work suggests that providing time-stamped, subtask annotations (e.g., "heat the oil in the pan") improve…
TimeGazer: Temporal Modeling of Predictive Gaze Stabilization for AR Interaction
Yaozheng Xia, Zaiping Zhu, Bo Pang +2
Gaze stabilization is critical for enabling fluid, accurate, and efficient interaction in immersive augmented reality (AR) environments, particularly during task-oriented visual be…
A point cloud reconstruction method based on uncertainty feature enhancement for aerodynamic shape optimization
Junlin Li, Yang Zhang, Bo Pang +2
The precision of shape representation and the dimensionality of the design space significantly influence the cost and outcomes of aerodynamic optimization. The design space can be…
Efficient Long-context Language Model Training by Core Attention Disaggregation
Yonghao Zhuang, Junda Chen, Bo Pang +6
We present core attention disaggregation (CAD), a technique that improves long-context large language model training by decoupling the core attention computation, softmax(QK^T)V, f…
Let AI Read First: Enhancing Reading Abilities for Individuals with Dyslexia through Artificial Intelligence
Sihang Zhao, Shoucong Carol Xiong, Bo Pang +2
Dyslexia, a neurological condition affecting approximately 12% of the global population, presents significant challenges to reading ability and quality of life. Existing assistive…
Neural Laplacian Operator for 3D Point Clouds
Bo Pang, Zhongtian Zheng, Yilong Li +2
The discrete Laplacian operator holds a crucial role in 3D geometry processing, yet it is still challenging to define it on point clouds. Previous works mainly focused on construct…
CVPO: Enhancing LLM Reinforcement Learning Reasoning via Value-Variance Adaptation and Dynamic Curriculum Learning
Ziqi Jia, Yalu Ouyang, Bo Pang +5
Reinforcement learning (RL) has emerged as an effective method for enhancing the reasoning capabilities of large language models (LLMs). However, existing methods suffer from insuf…
xGen-small Technical Report
Erik Nijkamp, Bo Pang, Egor Pakhomov +5
We introduce xGen-small, a family of 4B and 9B Transformer decoder models optimized for long-context applications. Our vertically integrated pipeline unites domain-balanced, freque…
Thumbs up? Sentiment Classification using Machine Learning Techniques
Bo Pang, Lillian Lee, Shivakumar Vaithyanathan
We consider the problem of classifying documents not by topic, but by overall sentiment, e.g., determining whether a review is positive or negative. Using movie reviews as data, we…
Human Pose Regression with Residual Log-likelihood Estimation
Jiefeng Li, Siyuan Bian, Ailing Zeng +4
Heatmap-based methods dominate in the field of human pose estimation by modelling the output distribution through likelihood heatmaps. In contrast, regression-based methods are mor…
Human Action Adverb Recognition: ADHA Dataset and A Three-Stream Hybrid Model
Bo Pang, Kaiwen Zha, Cewu Lu
We introduce the first benchmark for a new problem --- recognizing human action adverbs (HAA): "Adverbs Describing Human Actions" (ADHA). This is the first step for computer vision…
CoDA: Coding LM via Diffusion Adaptation
Haolin Chen, Shiyu Wang, Can Qin +12
Diffusion language models promise bidirectional context and infilling capabilities that autoregressive coders lack, yet practical systems remain heavyweight. We introduce CoDA, a 1…
End-to-end Dense Video Captioning as Sequence Generation
Wanrong Zhu, Bo Pang, Ashish V. Thapliyal +2
Dense video captioning aims to identify the events of interest in an input video, and generate descriptive captions for each event. Previous approaches usually follow a two-stage g…
BOLT: Bootstrap Long Chain-of-Thought in Language Models without Distillation
Bo Pang, Hanze Dong, Jiacheng Xu +3
Large language models (LLMs), such as o1 from OpenAI, have demonstrated remarkable reasoning capabilities. o1 generates a long chain-of-thought (LongCoT) before answering a questio…
Generative Text Modeling through Short Run Inference
Bo Pang, Erik Nijkamp, Tian Han +1
Latent variable models for text, when trained successfully, accurately model the data distribution and capture global semantic and syntactic features of sentences. The prominent ap…
NAT: Neural Acoustic Transfer for Interactive Scenes in Real Time
Xutong Jin, Bo Pang, Chenxi Xu +3
Previous acoustic transfer methods rely on extensive precomputation and storage of data to enable real-time interaction and auditory feedback. However, these methods struggle with…
Hybrid optical pumping of K and Rb atoms in a paraffin coated vapor cell
Wenhao Li, Xiang Peng, Dmitry Budker +4
Dynamic hybrid optical pumping effects with a radio-frequency-field-driven nonlinear magneto-optical rotation (RF NMOR) scheme are studied in a dual-species paraffin coated vapor c…
Seer: Online Context Learning for Fast Synchronous LLM Reinforcement Learning
Ruoyu Qin, Weiran He, Weixiao Huang +7
Reinforcement Learning (RL) has emerged as a critical technique for advancing modern Large Language Models (LLMs), yet existing synchronous RL systems face severe performance bottl…
Long Document Summarization with Top-down and Bottom-up Inference
Bo Pang, Erik Nijkamp, Wojciech KryÅciÅski +3
Text summarization aims to condense long documents and retain key information. Critical to the success of a summarization model is the faithful inference of latent representations…
FEVO: Financial Knowledge Expansion and Reasoning Evolution for Large Language Models
Bo Pang, Yalu Ouyang, Hangfei Xu +6
Advancements in reasoning for large language models (LLMs) have lead to significant performance improvements for LLMs in various fields such as mathematics and programming. However…
Unsupervised Visual Representation Learning by Synchronous Momentum Grouping
Bo Pang, Yifan Zhang, Yaoyi Li +2
In this paper, we propose a genuine group-level contrastive visual representation learning method whose linear evaluation performance on ImageNet surpasses the vanilla supervised l…
GReaTer: Gradients over Reasoning Makes Smaller Language Models Strong Prompt Optimizers
Sarkar Snigdha Sarathi Das, Ryo Kamoi, Bo Pang +3
The effectiveness of large language models (LLMs) is closely tied to the design of prompts, making prompt optimization essential for enhancing their performance across a wide range…
Fire-Image-DenseNet (FIDN) for predicting wildfire burnt area using remote sensing data
Bo Pang, Sibo Cheng, Yuhan Huang +5
Predicting the extent of massive wildfires once ignited is essential to reduce the subsequent socioeconomic losses and environmental damage, but challenging because of the complexi…
Latent Space Energy-Based Model of Symbol-Vector Coupling for Text Generation and Classification
Bo Pang, Ying Nian Wu
We propose a latent space energy-based prior model for text generation and classification. The model stands on a generator network that generates the text sequence based on a conti…
Kimi K2.5: Visual Agentic Intelligence
Kimi Team, Tongtong Bai, Yifan Bai +339
We introduce Kimi K2.5, an open-source multimodal agentic model designed to advance general agentic intelligence. K2.5 emphasizes the joint optimization of text and vision so that…
SSR: Socratic Self-Refine for Large Language Model Reasoning
Haizhou Shi, Ye Liu, Bo Pang +6
Large Language Models (LLMs) have demonstrated remarkable reasoning abilities, yet existing test-time frameworks often rely on coarse self-verification and self-correction, limitin…
Single Image Deraining via Scale-space Invariant Attention Neural Network
Bo Pang, Deming Zhai, Junjun Jiang +1
Image enhancement from degradation of rainy artifacts plays a critical role in outdoor visual computing systems. In this paper, we tackle the notion of scale that deals with visual…
Global River Forecasting with a Topology-Informed AI Foundation Model
Hancheng Ren, Gang Zhao, Shuo Wang +11
River systems operate as inherently interconnected continuous networks, meaning river hydrodynamic simulation ought to be a systemic process. However, widespread hydrology data sca…
Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
Gheorghe Comanici, Eric Bieber, Mike Schaekermann +3431
In this report, we introduce the Gemini 2.X model family: Gemini 2.5 Pro and Gemini 2.5 Flash, as well as our earlier Gemini 2.0 Flash and Flash-Lite models. Gemini 2.5 Pro is our…
Electromagnetic Quantitative Inversion for Translationally Moving Targets via Phase Correlation Registration of Back-Projection Images
Yitao Lin, Dahai Dai, Shilong Sun +2
A novel electromagnetic quantitative inversion scheme for translationally moving targets via phase correlation registration of back-projection (BP) images is proposed. Based on a t…
An Analysis of Structured Data on the Web
Nilesh Dalvi, Ashwin Machanavajjhala, Bo Pang
In this paper, we analyze the nature and distribution of structured data on the Web. Web-scale information extraction, or the problem of creating structured tables using extraction…
Temporal Consistency Constrained Transferable Adversarial Attacks with Background Mixup for Action Recognition
Ping Li, Jianan Ni, Bo Pang
Action recognition models using deep learning are vulnerable to adversarial examples, which are transferable across other models trained on the same data modality. Existing transfe…
Latent Plan Transformer for Trajectory Abstraction: Planning as Latent Space Inference
Deqian Kong, Dehong Xu, Minglu Zhao +6
In tasks aiming for long-term returns, planning becomes essential. We study generative modeling for planning with datasets repurposed from offline reinforcement learning. Specifica…
MetaWild: A Multimodal Dataset for Animal Re-Identification with Environmental Metadata
Yuzhuo Li, Di Zhao, Tingrui Qiao +3
Identifying individual animals within large wildlife populations is essential for effective wildlife monitoring and conservation efforts. Recent advancements in computer vision hav…
Text Detoxification: Data Efficiency, Semantic Preservation and Model Generalization
Jing Yu, Yibo Zhao, Jiapeng Zhu +4
The widespread dissemination of toxic content on social media poses a serious threat to both online environments and public discourse, highlighting the urgent need for detoxificati…
Learning-Based Adaptive Optimal Control of Linear Time-Delay Systems: A Policy Iteration Approach
Leilei Cui, Bo Pang, Zhong-Ping Jiang
This paper studies the adaptive optimal control problem for a class of linear time-delay systems described by delay differential equations (DDEs). A crucial strategy is to take adv…
Non-Intrusive Adaptation: Input-Centric Parameter-efficient Fine-Tuning for Versatile Multimodal Modeling
Yaqing Wang, Jialin Wu, Tanmaya Dabral +8
Large language models (LLMs) and vision language models (VLMs) demonstrate excellent performance on a wide range of tasks by scaling up parameter counts from O(10^9) to O(10^{12})…
Complex Sequential Understanding through the Awareness of Spatial and Temporal Concepts
Bo Pang, Kaiwen Zha, Hanwen Cao +3
Understanding sequential information is a fundamental task for artificial intelligence. Current neural networks attempt to learn spatial and temporal information as a whole, limite…
VibraVerse: A Large-Scale Geometry-Acoustics Alignment Dataset for Physically-Consistent Multimodal Learning
Bo Pang, Chenxi Xu, Jierui Ren +2
Understanding the physical world requires perceptual models grounded in physical laws rather than mere statistical correlations. However, existing multimodal learning frameworks, f…
OctaOctree Neural Radiosity for Real-time Glossy Material Rendering
Jierui Ren, Haojie Jin, Bo Pang +4
Modeling high-frequency outgoing radiance distributions remains a fundamental challenge in global illumination, especially for glossy and specular materials. Existing neural-based…
Understanding Guided Image Captioning Performance across Domains
Edwin G. Ng, Bo Pang, Piyush Sharma +1
Image captioning models generally lack the capability to take into account user interest, and usually default to global descriptions that try to balance readability, informativenes…
Joint Training of Variational Auto-Encoder and Latent Energy-Based Model
Tian Han, Erik Nijkamp, Linqi Zhou +3
This paper proposes a joint training method to learn both the variational auto-encoder (VAE) and the latent energy-based model (EBM). The joint training of VAE and latent EBM are b…
Learning Probabilistic Models from Generator Latent Spaces with Hat EBM
Mitch Hill, Erik Nijkamp, Jonathan Mitchell +2
This work proposes a method for using any generator network as the foundation of an Energy-Based Model (EBM). Our formulation posits that observed images are the sum of unobserved…
BigIssue: A Realistic Bug Localization Benchmark
Paul Kassianik, Erik Nijkamp, Bo Pang +2
As machine learning tools progress, the inevitable question arises: How can machine learning help us write better code? With significant progress being achieved in natural language…