Publications (93)
Theoretical Analysis on how Learning Rate Warmup Accelerates Convergence
Yuxing Liu, Yuze Ge, Rui Pan +2
Learning rate warmup is a popular and practical technique in training large-scale deep neural networks. Despite the huge success in practice, the theoretical advantages of this str…
AstroMLab 1: Who Wins Astronomy Jeopardy!?
Yuan-Sen Ting, Tuan Dung Nguyen, Tirthankar Ghosal +8
We present a comprehensive evaluation of proprietary and open-weights large language models using the first astronomy-specific benchmarking dataset. This dataset comprises 4,425 mu…
LLM-powered Real-time Patent Citation Recommendation for Financial Technologies
Tianang Deng, Yu Deng, Tianchen Gao +2
Rapid financial innovation has been accompanied by a sharp increase in patenting activity, making timely and comprehensive prior-art discovery more difficult. This problem is espec…
AstroMLab 3: Achieving GPT-4o Level Performance in Astronomy with a Specialized 8B-Parameter Large Language Model
Tijmen de Haan, Yuan-Sen Ting, Tirthankar Ghosal +6
AstroSage-Llama-3.1-8B is a domain-specialized natural-language AI assistant tailored for research in astronomy, astrophysics, cosmology, and astronomical instrumentation. Trained…
Universal properties of the evolution of the Universe in modified loop quantum cosmology
Jamal Saeed, Rui Pan, Christian Brown +2
In this paper, we systematically study the evolution of the Universe in the framework of a modified loop quantum cosmological model (mLQC-I) with various inflationary potentials, i…
AstroMLab 4: Benchmark-Topping Performance in Astronomy Q&A with a 70B-Parameter Domain-Specialized Reasoning Model
Tijmen de Haan, Yuan-Sen Ting, Tirthankar Ghosal +7
General-purpose large language models (LLMs), despite their broad capabilities, often struggle with specialized domain knowledge. This gap hinders their deployment as reliable rese…
Bi-SCORE for Weighted Bipartite Networks with Application in Knowledge Source Discovery
Zicheng Xie, Rui Pan, Yan Zhang
Community detection in citation networks offers a powerful approach to understanding knowledge flow and identifying core research areas within academic disciplines. This study focu…
TheoremLlama: Transforming General-Purpose LLMs into Lean4 Experts
Ruida Wang, Jipeng Zhang, Yizhen Jia +4
Proving mathematical theorems using computer-verifiable formal languages like Lean significantly impacts mathematical reasoning. One approach to formal theorem proving involves gen…
Fox-1: Open Small Language Model for Cloud and Edge
Zijian Hu, Jipeng Zhang, Rui Pan +9
We present Fox-1, a series of small language models (SLMs) consisting of Fox-1-1.6B and Fox-1-1.6B-Instruct-v0.1. These models are pre-trained on 3 trillion tokens of web-scraped d…
Safe RLHF-V: Safe Reinforcement Learning from Multi-modal Human Feedback
Jiaming Ji, Xinyu Chen, Rui Pan +13
Multimodal large language models (MLLMs) are essential for building general-purpose AI assistants; however, they pose increasing safety risks. How can we ensure safety alignment of…
Recursive Multi-Agent Systems
Jiaru Zou, Rui Pan, Ruizhong Qiu +8
The paper proposes RecursiveMAS, a framework that treats a multi-agent system as a recursive latent‑space computation, enabling agents to iteratively refine each other's thoughts a…
LISA: Layerwise Importance Sampling for Memory-Efficient Large Language Model Fine-Tuning
Rui Pan, Xiang Liu, Shizhe Diao +4
The machine learning community has witnessed impressive advancements since large language models (LLMs) first appeared. Yet, their massive memory consumption has become a significa…
Residual Channel Boosts Contrastive Learning for Radio Frequency Fingerprint Identification
Rui Pan, Hui Chen, Guanxiong Shen +1
In order to address the issue of limited data samples for the deployment of pre-trained models in unseen environments, this paper proposes a residual channel-based data augmentatio…
Quantum corrections to the entropy and its application in the study of quantum Carnot engines
Tian Qiu, Zhaoyu Fei, Rui Pan +1
Entropy is one of the most basic concepts in thermodynamics and statistical mechanics. The most widely used definition of statistical mechanical entropy for a quantum system is int…
StoSignSGD: Unbiased Structural Stochasticity Fixes SignSGD for Training Large Language Models
Dingzhi Yu, Rui Pan, Yuxing Liu +1
Sign-based optimization algorithms, such as SignSGD, have garnered significant attention for their remarkable performance in distributed learning and training large foundation mode…
Physically interpretable diffractive optical networks for high-dimensional vortex mode sorting
Ruitao Wu, Juncheng Fang, Rui Pan +5
Despite the significant progress achieved by diffractive optical networks in diverse computing tasks, such as mode multiplexing and demultiplexing, investigations into the physical…
Quantum-classical correspondence of work distributions for initial states with quantum coherence
Rui Pan, Zhaoyu Fei, Tian Qiu +2
The standard definition of quantum fluctuating work is based on the two-projective energy measurement, which however does not apply to systems with initial quantum coherence becaus…
ScaleBiO: Scalable Bilevel Optimization for LLM Data Reweighting
Rui Pan, Dylan Zhang, Hanning Zhang +6
Bilevel optimization has shown its utility across various machine learning settings, yet most algorithms in practice require second-order information, making it challenging to scal…
Grounding Visual Illusions in Language: Do Vision-Language Models Perceive Illusions Like Humans?
Yichi Zhang, Jiayi Pan, Yuchen Zhou +2
Vision-Language Models (VLMs) are trained on vast amounts of data captured by humans emulating our understanding of the world. However, known as visual illusions, human's perceptio…
ASGO: Adaptive Structured Gradient Optimization
Kang An, Yuxing Liu, Rui Pan +4
Training deep neural networks is a structured optimization problem, because the parameters are naturally represented by matrices and tensors rather than by vectors. Under this stru…
PhysProver: Advancing Automatic Theorem Proving for Physics
Hanning Zhang, Ruida Wang, Rui Pan +3
The combination of verifiable languages and LLMs has significantly influenced both the mathematical and computer science communities because it provides a rigorous foundation for t…
Effective Bilevel Optimization via Minimax Reformulation
Xiaoyu Wang, Rui Pan, Renjie Pi +1
Bilevel optimization has found successful applications in various machine learning problems, including hyper-parameter optimization, data cleaning, and meta-learning. However, its…
Humanity's Last Exam
Long Phan, Alice Gatti, Ziwen Han +1144
Benchmarks are important tools for tracking the rapid advancements in large language model (LLM) capabilities. However, benchmarks are not keeping pace in difficulty: LLMs now achi…
AgentSPEX: An Agent SPecification and EXecution Language
Pengcheng Wang, Jerry Huang, Jiarui Yao +7
Language-model agent systems commonly rely on reactive prompting, in which a single instruction guides the model through an open-ended sequence of reasoning and tool-use steps, lea…
BASIC: Bipartite Assisted Spectral-clustering for Identifying Communities in Large-scale Networks
Tianchen Gao, Jingyuan Liu, Rui Pan +1
Community detection, which focuses on recovering the group structure within networks, is a crucial and fundamental task in network analysis. However, the detection process can be q…
AdaGrad under Anisotropic Smoothness
Yuxing Liu, Rui Pan, Tong Zhang
Adaptive gradient methods have been widely adopted in training large-scale deep neural networks, especially large foundation models. Despite the huge success in practice, their the…
The Instinctive Bias: Spurious Images lead to Illusion in MLLMs
Tianyang Han, Qing Lian, Rui Pan +5
Large language models (LLMs) have recently experienced remarkable progress, where the advent of multi-modal large language models (MLLMs) has endowed LLMs with visual capabilities,…
Slipstream: Trajectory-Grounded Compaction Validation for Long-Horizon Agents
Zhuofu Chen, Rui Pan, Yinwei Dai +1
To cope with the large contexts that long-horizon LLM agents produce, modern frameworks increasingly rely on compaction -- invoking an LLM to rewrite the accumulated trajectory int…
A Sequential Addressing Subsampling Method for Massive Data Analysis under Memory Constraint
Rui Pan, Yingqiu Zhu, Baishan Guo +2
The emergence of massive data in recent years brings challenges to automatic statistical inference. This is particularly true if the data are too numerous to be read into memory as…
Investigating Tool-Memory Conflicts in Tool-Augmented LLMs
Jiali Cheng, Rui Pan, Hadi Amiri
Tool-augmented large language models (LLMs) have powered many applications. However, they are likely to suffer from knowledge conflict. In this paper, we propose a new type of know…
AstroMLab 2: AstroLLaMA-2-70B Model and Benchmarking Specialised LLMs for Astronomy
Rui Pan, Tuan Dung Nguyen, Hardik Arora +3
Continual pretraining of large language models on domain-specific data has been proposed to enhance performance on downstream tasks. In astronomy, the previous absence of astronomy…
METIS: Fast Quality-Aware RAG Systems with Configuration Adaptation
Siddhant Ray, Rui Pan, Zhuohan Gu +5
RAG (Retrieval Augmented Generation) allows LLMs (large language models) to generate better responses with external knowledge, but using more external knowledge often improves gene…
GAR: Generative Adversarial Reinforcement Learning for Formal Theorem Proving
Ruida Wang, Jiarui Yao, Rui Pan +2
Solving math problems through verifiable languages such as Lean has significantly impacted both the mathematics and computer science communities. Current state-of-the-art models ar…
CollaMamba: Efficient Collaborative Perception with Cross-Agent Spatial-Temporal State Space Model
Yang Li, Quan Yuan, Guiyang Luo +5
By sharing complementary perceptual information, multi-agent collaborative perception fosters a deeper understanding of the environment. Recent studies on collaborative perception…
MLLM-Protector: Ensuring MLLM's Safety without Hurting Performance
Renjie Pi, Tianyang Han, Jianshu Zhang +6
The deployment of multimodal large language models (MLLMs) has brought forth a unique vulnerability: susceptibility to malicious attacks through visual inputs. This paper investiga…
Image Textualization: An Automatic Framework for Creating Accurate and Detailed Image Descriptions
Renjie Pi, Jianshu Zhang, Jipeng Zhang +3
Image description datasets play a crucial role in the advancement of various applications such as image understanding, text-to-image generation, and text-image retrieval. Currently…
A Latent Factor Model for High-Dimensional Binary Data
Jiaxin Shi, Yuan Gao, Rui Pan +1
In this study, we develop a latent factor model for analysing high-dimensional binary data. Specifically, a standard probit model is used to describe the regression relationship be…
Entropy-Regularized Process Reward Model
Hanning Zhang, Pengcheng Wang, Shizhe Diao +6
Large language models (LLMs) have shown promise in performing complex multi-step reasoning, yet they continue to struggle with mathematical reasoning, often making systematic error…
Stable initial conditions and analytical investigations of cosmological perturbations in a modified loop quantum cosmology
Rui Pan, Jamal Saeed, Anzhong Wang
In this paper, we study cosmological perturbations in a modified theory of loop quantum cosmologies, the so-called mLQC-I model. Our purposes are two-fold: First, using a method de…
Fail Fast, Win Big: Rethinking the Drafting Strategy in Speculative Decoding via Diffusion LLMs
Rui Pan, Zhuofu Chen, Hongyi Liu +2
Diffusion Large Language Models (dLLMs) offer fast, parallel token generation, but their standalone use is plagued by an inherent efficiency-quality tradeoff. We show that, if care…
Marconi: Prefix Caching for the Era of Hybrid LLMs
Rui Pan, Zhuang Wang, Zhen Jia +5
Hybrid models that combine the language modeling capabilities of Attention layers with the efficiency of Recurrent layers (e.g., State Space Models) have gained traction in practic…
Eigencurve: Optimal Learning Rate Schedule for SGD on Quadratic Objectives with Skewed Hessian Spectrums
Rui Pan, Haishan Ye, Tong Zhang
Learning rate schedulers have been widely adopted in training deep neural networks. Despite their practical importance, there is a discrepancy between its practice and its theoreti…
Active Prompting with Chain-of-Thought for Large Language Models
Shizhe Diao, Pengcheng Wang, Yong Lin +3
The increasing scale of large language models (LLMs) brings emergent abilities to various complex tasks requiring reasoning, such as arithmetic and commonsense reasoning. It is kno…
Experimental realization of Feynman's ratchet
Jaehoon Bang, Rui Pan, Thai M. Hoang +4
Feynman's ratchet is a microscopic machine in contact with two heat reservoirs, at temperatures and , that was proposed by Richard Feynman to illustrate the second law o…
DetGPT: Detect What You Need via Reasoning
Renjie Pi, Jiahui Gao, Shizhe Diao +8
In recent years, the field of computer vision has seen significant advancements thanks to the development of large language models (LLMs). These models have enabled more effective…
Code as Agent Harness
Xuying Ning, Katherine Tieu, Dongqi Fu +39
Recent large language models (LLMs) have demonstrated strong capabilities in understanding and generating code, from competitive programming to repository-level software engineerin…
TAGCOS: Task-agnostic Gradient Clustered Coreset Selection for Instruction Tuning Data
Jipeng Zhang, Yaxuan Qin, Renjie Pi +3
Instruction tuning has achieved unprecedented success in NLP, turning large language models into versatile chatbots. However, the increasing variety and volume of instruction datas…
RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignment
Hanze Dong, Wei Xiong, Deepanshu Goyal +7
Generative foundation models are susceptible to implicit biases that can arise from extensive unsupervised training data. Such biases can produce suboptimal samples, skewed outcome…
MiCRo: Mixture Modeling and Context-aware Routing for Personalized Preference Learning
Jingyan Shen, Jiarui Yao, Rui Yang +5
Reward modeling is a key step in building safe foundation models when applying reinforcement learning from human feedback (RLHF) to align Large Language Models (LLMs). However, rew…
Automated Gateways: A Smart Contract-Powered Solution for Interoperability Across Blockchains
Koosha Esmaeilzadeh Khorasani, Sara Rouhani, Rui Pan +1
Interoperability is a significant challenge in blockchain technology, hindering seamless data and service sharing across diverse blockchain networks. This study introduces Automate…
Strengthening Multimodal Large Language Model with Bootstrapped Preference Optimization
Renjie Pi, Tianyang Han, Wei Xiong +4
Multimodal Large Language Models (MLLMs) excel in generating responses based on visual inputs. However, they often suffer from a bias towards generating responses similar to their…
Apparate: Rethinking Early Exits to Tame Latency-Throughput Tensions in ML Serving
Yinwei Dai, Rui Pan, Anand Iyer +2
Machine learning (ML) inference platforms are tasked with balancing two competing goals: ensuring high throughput given many requests, and delivering low-latency responses to suppo…
Understanding Overadaptation in Supervised Fine-Tuning: The Role of Ensemble Methods
Yifan Hao, Xingyuan Pan, Hanning Zhang +3
Supervised fine-tuning (SFT) on domain-specific data is the dominant approach for adapting foundation models to specialized tasks. However, it has been observed that SFT models ten…
Unbiased Gradient Low-Rank Projection
Rui Pan, Yang Luo, Yuxing Liu +2
Memory-efficient optimization is critical for training increasingly large language models (LLMs). A popular strategy involves gradient low-rank projection, storing only the project…
Grid Point Approximation for Distributed Nonparametric Smoothing and Prediction
Yuan Gao, Rui Pan, Feng Li +2
Kernel smoothing is a widely used nonparametric method in modern statistical analysis. The problem of efficiently conducting kernel smoothing for a massive dataset on a distributed…
LMFlow: An Extensible Toolkit for Finetuning and Inference of Large Foundation Models
Shizhe Diao, Rui Pan, Hanze Dong +4
Foundation models have demonstrated a great ability to achieve general human-level intelligence far beyond traditional approaches. As the technique keeps attracting attention from…
VL-GenRM: Enhancing Vision-Language Verification via Vision Experts and Iterative Training
Jipeng Zhang, Kehao Miao, Renjie Pi +4
Reinforcement Fine-Tuning (RFT) with verifiable rewards has advanced large language models but remains underexplored for Vision-Language (VL) models. The Vision-Language Reward Mod…
Path integral approach to the calculation of the characteristic function of work
Tian Qiu, Zhaoyu Fei, Rui Pan +1
Work statistics characterizes important features of a non-equilibrium thermodynamic process. But the calculation of the work statistics in an arbitrary non-equilibrium process is u…
Uniform Asymptotic Approximation Method with Pöschl-Teller Potential
Rui Pan, John Joseph Marchetta, Jamal Saeed +4
In this paper, we study analytical approximate solutions of the second-order homogeneous differential equations with the existence of only two turning points (but without poles), b…
Academic Literature Recommendation in Large-scale Citation Networks Enhanced by Large Language Models
Kun Liu, Yan Zhang, Rui Pan +2
Literature recommendation is essential for researchers to find relevant articles in an ever-growing academic field. However, traditional methods often struggle due to data limitati…
NegoCollab: A Common Representation Negotiation Approach for Heterogeneous Collaborative Perception
Congzhang Shao, Quan Yuan, Guiyang Luo +6
Collaborative perception improves task performance by expanding the perception range through information sharing among agents. . Immutable heterogeneity poses a significant challen…
An LLM-Powered Semantic Alignment Framework for Journal Recommendation
Yanglin Yan, Zicheng Xie, Tianchen Gao +2
Journal recommendation is an important task in scholarly information systems. Existing approaches typically rely on supervised learning models, manually engineered features, or his…
Adapt-Pruner: Adaptive Structural Pruning for Efficient Small Language Model Training
Rui Pan, Shivanshu Shekhar, Boyao Wang +5
Small language models (SLMs) have attracted considerable attention from both academia and industry due to their broad range of applications in edge devices. To obtain SLMs with str…
Bridge-Coder: Unlocking LLMs' Potential to Overcome Language Gaps in Low-Resource Code
Jipeng Zhang, Jianshu Zhang, Yuanzhe Li +5
Large Language Models (LLMs) demonstrate strong proficiency in generating code for high-resource programming languages (HRPLs) like Python but struggle significantly with low-resou…
Optimizing Mixture-of-Experts Inference Time Combining Model Deployment and Communication Scheduling
Jialong Li, Shreyansh Tripathi, Lakshay Rastogi +3
As machine learning models scale in size and complexity, their computational requirements become a significant barrier. Mixture-of-Experts (MoE) models alleviate this issue by sele…
Quantum work distributions associated with the dynamical Casimir effect
Zhaoyu Fei, Jing-Ning Zhang, Rui Pan +2
We study the joint probability distribution function of the work and the change of photon number of the nonequilibrium process of driving the electromagnetic (EM) field in a three-…
Beyond BEV: Optimizing Point-Level Tokens for Collaborative Perception
Yang Li, Quan Yuan, Guiyang Luo +6
Collaborative perception allows agents to enhance their perceptual capabilities by exchanging intermediate features. Existing methods typically organize these intermediate features…
Align Anything: Training All-Modality Models to Follow Instructions with Language Feedback
Jiaming Ji, Jiayi Zhou, Hantao Lou +16
Reinforcement learning from human feedback (RLHF) has proven effective in enhancing the instruction-following capabilities of large language models; however, it remains underexplor…
Are Large Language Models able to Predict Highly Cited Papers? Evidence from Statistical Publications
Zhanshuo Ye, Yiming Hou, Rui Pan +2
Predicting highly-cited papers is a long-standing challenge due to the complex interactions of research content, scholarly communities, and temporal dynamics. Recent advances in la…
Mitigating the Alignment Tax of RLHF
Yong Lin, Hangyu Lin, Wei Xiong +14
LLMs acquire a wide range of abilities during pre-training, but aligning LLMs under Reinforcement Learning with Human Feedback (RLHF) can lead to forgetting pretrained abilities, w…
Plum: Prompt Learning using Metaheuristic
Rui Pan, Shuo Xing, Shizhe Diao +6
Since the emergence of large language models, prompt learning has become a popular method for optimizing and customizing these models. Special prompts, such as Chain-of-Thought, ha…
Shockwave: Fair and Efficient Cluster Scheduling for Dynamic Adaptation in Machine Learning
Pengfei Zheng, Rui Pan, Tarannum Khan +2
Dynamic adaptation has become an essential technique in accelerating distributed machine learning (ML) training. Recent studies have shown that dynamically adjusting model structur…
Accelerated Convergence of Stochastic Heavy Ball Method under Anisotropic Gradient Noise
Rui Pan, Yuxing Liu, Xiaoyu Wang +1
Heavy-ball momentum with decaying learning rates is widely used with SGD for optimizing deep learning models. In contrast to its empirical popularity, the understanding of its theo…
Diffusion-Proof: Recipe for Formal Theorem Proving Beyond Auto-Regressive Generation
Ruida Wang, Rui Pan, Pengcheng Wang +2
Enhancing the formal math reasoning capabilities of Large Language Models (LLMs) has become a key focus in both mathematical and computer science communities in recent years. While…
MA-LoT: Model-Collaboration Lean-based Long Chain-of-Thought Reasoning enhances Formal Theorem Proving
Ruida Wang, Rui Pan, Yuxin Li +6
Solving mathematical problems using computer-verifiable languages like Lean has significantly impacted the mathematical and computer science communities. State-of-the-art methods u…
Lean4Physics: Comprehensive Reasoning Framework for College-level Physics in Lean4
Yuxin Li, Minghao Liu, Ruida Wang +6
We present **Lean4PHYS**, a comprehensive reasoning framework for college-level physics problems in Lean4. **Lean4PHYS** includes *LeanPhysBench*, a college-level benchmark for for…
Experimental test of the differential fluctuation theorem and a generalized Jarzynski equality for arbitrary initial states
Thai M. Hoang, Rui Pan, Jonghoon Ahn +3
Nonequilibrium processes of small systems such as molecular machines are ubiquitous in biology, chemistry and physics, but are often challenging to comprehend. In the past two deca…
SP-Mamba: Spatial-Perception State Space Model for Unsupervised Medical Anomaly Detection
Rui Pan, Ruiying Lu
Radiography imaging protocols target on specific anatomical regions, resulting in highly consistent images with recurrent structural patterns across patients. Recent advances in me…
ExtremeBERT: A Toolkit for Accelerating Pretraining of Customized BERT
Rui Pan, Shizhe Diao, Jianlin Chen +1
In this paper, we present ExtremeBERT, a toolkit for accelerating and customizing BERT pretraining. Our goal is to provide an easy-to-use BERT pretraining toolkit for the research…
Large-scale Multi-layer Academic Networks Derived from Statistical Publications
Tianchen Gao, Yan Zhang, Rui Pan +1
The utilization of multi-layer network structures now enables the explanation of complex systems in nature from multiple perspectives. Multi-layer academic networks capture diverse…
ERA: Transforming VLMs into Embodied Agents via Embodied Prior Learning and Online Reinforcement Learning
Hanyang Chen, Mark Zhao, Rui Yang +15
Recent advances in embodied AI highlight the potential of vision language models (VLMs) as agents capable of perception, reasoning, and interaction in complex environments. However…
A BERT-based Hierarchical Classification Model with Applications in Chinese Commodity Classification
Kun Liu, Tuozhen Liu, Feifei Wang +1
Existing e-commerce platforms heavily rely on manual annotation for product categorization, which is inefficient and inconsistent. These platforms often employ a hierarchical struc…
GUIDE: Towards Scalable Advising for Research Ideas
Yaowenqi Liu, Bingxu Meng, Rui Pan +4
The field of AI research is advancing at an unprecedented pace, enabling automated hypothesis generation and experimental design across diverse domains such as biology, mathematics…
A Selective Review on Statistical Methods for Massive Data Computation: Distributed Computing, Subsampling, and Minibatch Techniques
Xuetong Li, Yuan Gao, Hong Chang +11
This paper presents a selective review of statistical computation methods for massive data analysis. A huge amount of statistical methods for massive data computation have been rap…
SpecReason: Fast and Accurate Inference-Time Compute via Speculative Reasoning
Rui Pan, Yinwei Dai, Zhihao Zhang +3
Recent advances in inference-time compute have significantly improved performance on complex tasks by generating long chains of thought (CoTs) using Large Reasoning Models (LRMs).…
Generalizable Geometric Image Caption Synthesis
Yue Xin, Wenyuan Wang, Rui Pan +5
Multimodal large language models have various practical applications that demand strong reasoning abilities. Despite recent advancements, these models still struggle to solve compl…
Subjective Perspectives within Learned Representations Predict High-Impact Innovation
Likun Cao, Rui Pan, James Evans
Existing studies of innovation emphasize the power of social structures to shape innovation capacity. Emerging machine learning approaches, however, enable us to model innovators'…
Personalized Visual Instruction Tuning
Renjie Pi, Jianshu Zhang, Tianyang Han +3
Recent advancements in multimodal large language models (MLLMs) have demonstrated significant progress; however, these models exhibit a notable limitation, which we refer to as "fa…
Mowgli: Passively Learned Rate Control for Real-Time Video
Neil Agarwal, Rui Pan, Francis Y. Yan +1
Rate control algorithms are at the heart of video conferencing platforms, determining target bitrates that match dynamic network characteristics for high quality. Recent data-drive…
Quantifying the validity and breakdown of the overdamped approximation in stochastic thermodynamics: Theory and experiment
Rui Pan, Thai M. Hoang, Zhaoyu Fei +4
Stochastic thermodynamics provides an important framework to explore small physical systems where thermal fluctuations are inevitable. In the studies of stochastic thermodynamics,…
Chain-of-Experts: Unlocking the Communication Power of Mixture-of-Experts Models
Zihan Wang, Rui Pan, Jiarui Yao +7
We propose Chain-of-Experts (CoE), a new Mixture-of-Experts (MoE) architecture that introduces sequential expert communication within each layer. Unlike traditional MoE models, whe…
AstroMLab 5: Structured Summaries and Concept Extraction for 400,000 Astrophysics Papers
Yuan-Sen Ting, Alberto Accomazzi, Tirthankar Ghosal +4
We present a dataset of 408,590 astrophysics papers from arXiv (astro-ph), spanning 1992 through July 2025. Each paper has been processed through a multi-stage pipeline to produce:…
AstroLLaMA-Chat: Scaling AstroLLaMA with Conversational and Diverse Datasets
Ernest Perkowski, Rui Pan, Tuan Dung Nguyen +11
We explore the potential of enhancing LLM performance in astronomy-focused question-answering through targeted, continual pre-training. By employing a compact 7B-parameter LLaMA-2…