Publications (144)
Regularizing Black-box Models for Improved Interpretability
Gregory Plumb, Maruan Al-Shedivat, Angel Alexander Cabrera +3
Most of the work on interpretable machine learning has focused on designing either inherently interpretable models, which typically trade-off accuracy for interpretability, or post…
Post-Inference Prior Swapping
Willie Neiswanger, Eric Xing
While Bayesian methods are praised for their ability to incorporate useful prior knowledge, in practice, convenient priors that allow for computationally cheap or tractable inferen…
Pathological Visual Question Answering
Xuehai He, Zhuo Cai, Wenlan Wei +4
Is it possible to develop an "AI Pathologist" to pass the board-certified examination of the American Board of Pathology (ABP)? To build such a system, three challenges need to be…
On the Generalization Error Bounds of Neural Networks under Diversity-Inducing Mutual Angular Regularization
Pengtao Xie, Yuntian Deng, Eric Xing
Recently diversity-inducing regularization methods for latent variable models (LVMs), which encourage the components in LVMs to be diverse, have been studied to address several iss…
GQA-μP: The maximal parameterization update for grouped query attention
Kyle R. Chickering, Huijuan Wang, Mengxi Wu +7
Hyperparameter transfer across model architectures dramatically reduces the amount of compute necessary for tuning large language models (LLMs). The maximal update parameterization…
Towards Open-World Generation of Stereo Images and Unsupervised Matching
Feng Qiao, Zhexiao Xiong, Eric Xing +1
Stereo images are fundamental to numerous applications, including extended reality (XR) devices, autonomous driving, and robotics. Unfortunately, acquiring high-quality stereo imag…
Methods for comparing uncertainty quantifications for material property predictions
Kevin Tran, Willie Neiswanger, Junwoong Yoon +3
Data science and informatics tools have been proliferating recently within the computational materials science and catalysis fields. This proliferation has spurned the creation of…
AutoLoss: Learning Discrete Schedules for Alternate Optimization
Haowen Xu, Hao Zhang, Zhiting Hu +3
Many machine learning problems involve iteratively and alternately optimizing different task objectives with respect to different sets of parameters. Appropriately scheduling the o…
Learning to Prompt Segment Anything Models
Jiaxing Huang, Kai Jiang, Jingyi Zhang +4
Segment Anything Models (SAMs) like SEEM and SAM have demonstrated great potential in learning to segment anything. The core design of SAMs lies with Promptable Segmentation, which…
Stochastic Neural Networks with Infinite Width are Deterministic
Liu Ziyin, Hanlin Zhang, Xiangming Meng +3
This work theoretically studies stochastic neural networks, a main type of neural network in use. We prove that as the width of an optimized stochastic neural network tends to infi…
Global and Local Entailment Learning for Natural World Imagery
Srikumar Sastry, Aayush Dhakal, Eric Xing +2
Learning the hierarchical structure of data in vision-language models is a significant challenge. Previous works have attempted to address this challenge by employing entailment le…
Unsupervised Pseudo-Labeling for Extractive Summarization on Electronic Health Records
Xiangan Liu, Keyang Xu, Pengtao Xie +1
Extractive summarization is very useful for physicians to better manage and digest Electronic Health Records (EHRs). However, the training of a supervised model requires disease-sp…
Deep Generative Models with Learnable Knowledge Constraints
Zhiting Hu, Zichao Yang, Ruslan Salakhutdinov +4
The broad set of deep generative models (DGMs) has achieved remarkable advances. However, it is often difficult to incorporate rich structured domain knowledge with the end-to-end…
Fast Function to Function Regression
Junier Oliva, Willie Neiswanger, Barnabas Poczos +2
We analyze the problem of regression when both input covariates and output responses are functions from a nonparametric function class. Function to function regression (FFR) covers…
Distributed Machine Learning via Sufficient Factor Broadcasting
Pengtao Xie, Jin Kyu Kim, Yi Zhou +4
Matrix-parametrized models, including multiclass logistic regression and sparse coding, are used in machine learning (ML) applications ranging from computer vision to computational…
Temporally Disentangled Representation Learning under Unknown Nonstationarity
Xiangchen Song, Weiran Yao, Yewen Fan +5
In unsupervised causal representation learning for sequential data with time-delayed latent causal influences, strong identifiability results for the disentanglement of causally-re…
TrustLLM: Trustworthiness in Large Language Models
Yue Huang, Lichao Sun, Haoran Wang +67
Large language models (LLMs), exemplified by ChatGPT, have gained considerable attention for their excellent natural language processing capabilities. Nonetheless, these LLMs prese…
Tessellating The Earth
Daniel Cher, Hamza Iqbal, Eric Xing +2
Geolocation encoders, which map geographic coordinates to learned representations, are emerging as an effective means of capturing visual and non-visual characteristics from a lati…
Evaluating Step-by-Step Reasoning through Symbolic Verification
Yi-Fan Zhang, Hanlin Zhang, Li Erran Li +1
Pre-trained language models (LMs) have shown remarkable reasoning performance using explanations or chain-of-thoughts (CoT)) for in-context learning. On the other hand, these reaso…
IsoCompute Playbook: Optimally Scaling Sampling Compute for LLM RL
Zhoujun Cheng, Yutao Xie, Yuxiao Qu +12
While scaling laws guide compute allocation for LLM pre-training, analogous prescriptions for reinforcement learning (RL) post-training of large language models (LLMs) remain poorl…
Connecting the Dots Between MLE and RL for Sequence Prediction
Bowen Tan, Zhiting Hu, Zichao Yang +2
Sequence prediction models can be learned from example sequences with a variety of training algorithms. Maximum likelihood learning is simple and efficient, yet can suffer from com…
Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimation
Zechun Liu, Kwang-Ting Cheng, Dong Huang +2
The nonuniform quantization strategy for compressing neural networks usually achieves better performance than its counterpart, i.e., uniform strategy, due to its superior represent…
Deep learning based supervised semantic segmentation of Electron Cryo-Subtomograms
Chang Liu, Xiangrui Zeng, Ruogu Lin +4
Cellular Electron Cryo-Tomography (CECT) is a powerful imaging technique for the 3D visualization of cellular structure and organization at submolecular resolution. It enables anal…
KD-DLGAN: Data Limited Image Generation via Knowledge Distillation
Kaiwen Cui, Yingchen Yu, Fangneng Zhan +3
Generative Adversarial Networks (GANs) rely heavily on large-scale training data for training high-quality image generation models. With limited training data, the GAN discriminato…
Text Infilling
Wanrong Zhu, Zhiting Hu, Eric Xing
Recent years have seen remarkable progress of text generation in different contexts, such as the most common setting of generating text from scratch, and the emerging paradigm of r…
Toward Learning Human-aligned Cross-domain Robust Models by Countering Misaligned Features
Haohan Wang, Zeyi Huang, Hanlin Zhang +2
Machine learning has demonstrated remarkable prediction accuracy over i.i.d data, but the accuracy often drops when tested with data from another distribution. In this paper, we ai…
Critique of Agent Model
Eric Xing, Mingkai Deng, Jinyu Hou
What is an agent? What constitutes agency? With the rise of Large Language Model (LLM) systems marketed as ``coding agents'', ``AI co-scientists'', and other ``agentic" tools that…
Counterfactual Generation with Identifiability Guarantees
Hanqi Yan, Lingjing Kong, Lin Gui +4
Counterfactual generation lies at the core of various machine learning tasks, including image translation and controllable text generation. This generation process usually requires…
Discrete Temporal Models of Social Networks
Steve Hanneke, Wenjie Fu, Eric Xing
We propose a family of statistical models for social network evolution over time, which represents an extension of Exponential Random Graph Models (ERGMs). Many of the methods for…
EMO: Frustratingly Easy Progressive Training of Extendable MoE
Linghao Jin, Chufan Shi, Huijuan Wang +4
Sparse Mixture-of-Experts (MoE) models offer a powerful way to scale model size without increasing compute, as per-token FLOPs depend only on k active experts rather than the total…
Large Scale Distributed Distance Metric Learning
Pengtao Xie, Eric Xing
In large scale machine learning and data mining problems with high feature dimensionality, the Euclidean distance between data points can be uninformative, and Distance Metric Lear…
PSM: Learning Probabilistic Embeddings for Multi-scale Zero-Shot Soundscape Mapping
Subash Khanal, Eric Xing, Srikumar Sastry +4
A soundscape is defined by the acoustic environment a person perceives at a location. In this work, we propose a framework for mapping soundscapes across the Earth. Since soundscap…
MLSys: The New Frontier of Machine Learning Systems
Alexander Ratner, Dan Alistarh, Gustavo Alonso +66
Machine learning (ML) techniques are enjoying rapidly increasing adoption. However, designing and implementing the systems that support ML models in real-world deployments remains…
CIRL: Controllable Imitative Reinforcement Learning for Vision-based Self-driving
Xiaodan Liang, Tairui Wang, Luona Yang +1
Autonomous urban driving navigation with complex multi-agent dynamics is under-explored due to the difficulty of learning an optimal driving policy. The traditional modular pipelin…
AMP: Automatically Finding Model Parallel Strategies with Heterogeneity Awareness
Dacheng Li, Hongyi Wang, Eric Xing +1
Scaling up model sizes can lead to fundamentally new capabilities in many machine learning (ML) tasks. However, training big models requires strong distributed system expertise to…
Embarrassingly Parallel Variational Inference in Nonconjugate Models
Willie Neiswanger, Chong Wang, Eric Xing
We develop a parallel variational inference (VI) procedure for use in data-distributed settings, where each machine only has access to a subset of data and runs VI independently, w…
ConText-CIR: Learning from Concepts in Text for Composed Image Retrieval
Eric Xing, Pranavi Kolouju, Robert Pless +2
Composed image retrieval (CIR) is the task of retrieving a target image specified by a query image and a relative text that describes a semantic modification to the query image. Ex…
How Does Controllability Emerge In Language Models During Pretraining?
Jianshu She, Xinyue Li, Eric Xing +2
Language models can be steered by modifying their internal representations to control concepts such as emotion, style, or truthfulness in generation. However, the conditions for an…
Identification of Nonlinear Latent Hierarchical Models
Lingjing Kong, Biwei Huang, Feng Xie +3
Identifying latent variables and causal structures from observational data is essential to many real-world applications involving biological data, medical data, and unstructured da…
Efficient Long-context Language Model Training by Core Attention Disaggregation
Yonghao Zhuang, Junda Chen, Bo Pang +6
We present core attention disaggregation (CAD), a technique that improves long-context large language model training by decoupling the core attention computation, softmax(QK^T)V, f…
good4cir: Generating Detailed Synthetic Captions for Composed Image Retrieval
Pranavi Kolouju, Eric Xing, Robert Pless +2
Composed image retrieval (CIR) enables users to search images using a reference image combined with textual modifications. Recent advances in vision-language models have improved C…
TerraDiT-: Unified Spatial Control for Satellite Image Synthesis with Any Geospatial Primitive
Brian Wei, Srikumar Sastry, Daniel Cher +2
Generative models have achieved remarkable progress, yet applying them to satellite imagery remains challenging. Unlike natural imagery, satellite scenes are structured by spatiall…
Cappy: Outperforming and Boosting Large Multi-Task LMs with a Small Scorer
Bowen Tan, Yun Zhu, Lijuan Liu +3
Large language models (LLMs) such as T0, FLAN, and OPT-IML, excel in multi-tasking under a unified instruction-following paradigm, where they also exhibit remarkable generalization…
Multi-task Learning of Order-Consistent Causal Graphs
Xinshi Chen, Haoran Sun, Caleb Ellington +2
We consider the problem of discovering related Gaussian directed acyclic graphs (DAGs), where the involved graph structures share a consistent causal order and sparse unions of…
Nonparametric Variational Auto-encoders for Hierarchical Representation Learning
Prasoon Goyal, Zhiting Hu, Xiaodan Liang +2
The recently developed variational autoencoders (VAEs) have proved to be an effective confluence of the rich representational power of neural networks with Bayesian methods. Howeve…
Pruning Spurious Subgraphs for Graph Out-of-Distribution Generalization
Tianjun Yao, Haoxuan Li, Yongqiang Chen +4
Graph Neural Networks (GNNs) often encounter significant performance degradation under distribution shifts between training and test data, hindering their applicability in real-wor…
Un-Mix: Rethinking Image Mixtures for Unsupervised Visual Representation Learning
Zhiqiang Shen, Zechun Liu, Zhuang Liu +3
The recently advanced unsupervised learning approaches use the siamese-like framework to compare two "views" from the same image for learning representations. Making the two views…
3D Semantic Segmentation in the Wild: Learning Generalized Models for Adverse-Condition Point Clouds
Aoran Xiao, Jiaxing Huang, Weihao Xuan +6
Robust point cloud parsing under all-weather conditions is crucial to level-5 autonomy in autonomous driving. However, how to learn a universal 3D semantic segmentation (3DSS) mode…
Nile-Chat: Egyptian Language Models for Arabic and Latin Scripts
Guokan Shang, Hadi Abdine, Ahmad Chamma +7
We introduce Nile-Chat-4B, 3x4B-A6B, and 12B, a collection of LLMs for Egyptian dialect, uniquely designed to understand and generate texts written in both Arabic and Latin scripts…
Asymptotically Exact, Embarrassingly Parallel MCMC
Willie Neiswanger, Chong Wang, Eric Xing
Communication costs, resulting from synchronization requirements during learning, can greatly slow down many parallel machine learning algorithms. In this paper, we present a paral…
StyleRF: Zero-shot 3D Style Transfer of Neural Radiance Fields
Kunhao Liu, Fangneng Zhan, Yiwen Chen +5
3D style transfer aims to render stylized novel views of a 3D scene with multi-view consistency. However, most existing work suffers from a three-way dilemma over accurate geometry…
PathVQA: 30000+ Questions for Medical Visual Question Answering
Xuehai He, Yichen Zhang, Luntian Mou +2
Is it possible to develop an "AI Pathologist" to pass the board-certified examination of the American Board of Pathology? To achieve this goal, the first step is to create a visual…
Learning from Imperfect Annotations
Emmanouil Antonios Platanios, Maruan Al-Shedivat, Eric Xing +1
Many machine learning systems today are trained on large amounts of human-annotated data. Data annotation tasks that require a high level of competency make data acquisition expens…
Poseidon: A System Architecture for Efficient GPU-based Deep Learning on Multiple Machines
Hao Zhang, Zhiting Hu, Jinliang Wei +4
Deep learning (DL) has achieved notable successes in many machine learning tasks. A number of frameworks have been developed to expedite the process of designing and training deep…
Show, Describe and Conclude: On Exploiting the Structure Information of Chest X-Ray Reports
Baoyu Jing, Zeya Wang, Eric Xing
Chest X-Ray (CXR) images are commonly used for clinical screening and diagnosis. Automatically writing reports for these images can considerably lighten the workload of radiologist…
LAPS: A Length-Aware-Prefill LLM Serving System
Jianshu She, Zonghang Li, Hongchao Du +7
LAPS identifies and disaggregates requests with different prompt lengths in LLM serving to reduce TTFT latency. While recent systems have decoupled the prefill and decode stages to…
QuARI: Query Adaptive Retrieval Improvement
Eric Xing, Abby Stylianou, Robert Pless +1
Massive-scale pretraining has made vision-language models increasingly popular for image-to-image and text-to-image retrieval across a broad collection of domains. However, these m…
General Agentic Planning Through Simulative Reasoning with World Models
Mingkai Deng, Jinyu Hou, Zhiting Hu +1
What does it mean to plan? Current agentic systems, whether scaffolded workflows or end-to-end policies, rely on reactive decision-making: selecting the next action via a fixed pro…
Efficient Test-Time Adaptation of Vision-Language Models
Adilbek Karmanov, Dayan Guan, Shijian Lu +2
Test-time adaptation with pre-trained vision-language models has attracted increasing attention for tackling distribution shifts during the test time. Though prior studies have ach…
SDQ: Stochastic Differentiable Quantization with Mixed Precision
Xijie Huang, Zhiqiang Shen, Shichao Li +5
In order to deploy deep models in a computationally efficient manner, model quantization approaches have been frequently used. In addition, as new hardware that supports mixed bitw…
Fusing Models with Complementary Expertise
Hongyi Wang, Felipe Maia Polo, Yuekai Sun +3
Training AI models that generalize across tasks and domains has long been among the open problems driving AI research. The emergence of Foundation Models made it easier to obtain e…
COPUS: Co-adaptive Parallelism and Batch Size Selection in Large Language Model Training
Akhmed Sakip, Erland Hilman Fuadi, Omar Sayedelahl +6
Training large language models requires jointly configuring two interdependent aspects of the system: the global batch size, which governs statistical efficiency, and the 3D parall…
Prototypical Graph Contrastive Learning
Shuai Lin, Pan Zhou, Zi-Yuan Hu +6
Graph-level representations are critical in various real-world applications, such as predicting the properties of molecules. But in practice, precise graph annotations are generall…
Dynamic-structured Semantic Propagation Network
Xiaodan Liang, Hongfei Zhou, Eric Xing
Semantic concept hierarchy is still under-explored for semantic segmentation due to the inefficiency and complicated optimization of incorporating structural inference into dense p…
How to Build the Virtual Cell with Artificial Intelligence: Priorities and Opportunities
Charlotte Bunne, Yusuf Roohani, Yanay Rosen +39
The cell is arguably the most fundamental unit of life and is central to understanding biology. Accurate modeling of cells is important for this understanding as well as for determ…
One-for-All: Generalized LoRA for Parameter-Efficient Fine-tuning
Arnav Chavan, Zhuang Liu, Deepak Gupta +2
We present Generalized LoRA (GLoRA), an advanced approach for universal parameter-efficient fine-tuning tasks. Enhancing Low-Rank Adaptation (LoRA), GLoRA employs a generalized pro…
SegMix: A Simple Structure-Aware Data Augmentation Method
Yuxin Pei, Pushkar Bhuse, Zhengzhong Liu +1
Interpolation-based Data Augmentation (DA) methods (Mixup) linearly interpolate the inputs and labels of two or more training examples. Mixup has more recently been adapted to the…
Latent Variable Modeling with Diversity-Inducing Mutual Angular Regularization
Pengtao Xie, Yuntian Deng, Eric Xing
Latent Variable Models (LVMs) are a large family of machine learning models providing a principled and effective way to extract underlying patterns, structure and knowledge from ob…
Data-to-Text Generation with Style Imitation
Shuai Lin, Wentao Wang, Zichao Yang +4
Recent neural approaches to data-to-text generation have mostly focused on improving content fidelity while lacking explicit control over writing styles (e.g., word choices, senten…
Seeing the Forest from the Trees in Two Looks: Matrix Sketching by Cascaded Bilateral Sampling
Kai Zhang, Chuanren Liu, Jie Zhang +3
Matrix sketching is aimed at finding close approximations of a matrix by factors of much smaller dimensions, which has important applications in optimization and machine learning.…
A Fast Knowledge Distillation Framework for Visual Recognition
Zhiqiang Shen, Eric Xing
While Knowledge Distillation (KD) has been recognized as a useful tool in many visual tasks, such as supervised classification and self-supervised representation learning, the main…
Reducing Hyperparameter Tuning Costs in ML, Vision and Language Model Training Pipelines via Memoization-Awareness
Abdelmajid Essofi, Ridwan Salahuddeen, Munachiso Nwadike +5
The training or fine-tuning of machine learning, vision, and language models is often implemented as a pipeline: a sequence of stages encompassing data preparation, model training…
MixMask: Revisiting Masking Strategy for Siamese ConvNets
Kirill Vishniakov, Eric Xing, Zhiqiang Shen
The recent progress in self-supervised learning has successfully combined Masked Image Modeling (MIM) with Siamese Networks, harnessing the strengths of both methodologies. Nonethe…
On the Complexity of Exploration in Goal-Driven Navigation
Maruan Al-Shedivat, Lisa Lee, Ruslan Salakhutdinov +1
Building agents that can explore their environments intelligently is a challenging open problem. In this paper, we make a step towards understanding how a hierarchical design of th…
Predicting Discharge Medications at Admission Time Based on Deep Learning
Yuan Yang, Pengtao Xie, Xin Gao +4
Predicting discharge medications right after a patient being admitted is an important clinical decision, which provides physicians with guidance on what type of medication regimen…
RANGE: Retrieval Augmented Neural Fields for Multi-Resolution Geo-Embeddings
Aayush Dhakal, Srikumar Sastry, Subash Khanal +3
The choice of representation for geographic location significantly impacts the accuracy of models for a broad range of geospatial tasks, including fine-grained species classificati…
A Study on the Calibration of In-context Learning
Hanlin Zhang, Yi-Fan Zhang, Yaodong Yu +5
Accurate uncertainty quantification is crucial for the safe deployment of machine learning models, and prior research has demonstrated improvements in the calibration of modern lan…
Squeeze, Recover and Relabel: Dataset Condensation at ImageNet Scale From A New Perspective
Zeyuan Yin, Eric Xing, Zhiqiang Shen
We present a new dataset condensation framework termed Squeeze, Recover and Relabel (SReL) that decouples the bilevel optimization of model and synthetic data during training,…
Generalized Zero-shot ICD Coding
Congzheng Song, Shanghang Zhang, Najmeh Sadoughi +2
The International Classification of Diseases (ICD) is a list of classification codes for the diagnoses. Automatic ICD coding is in high demand as the manual coding can be labor-int…
Understanding the Skill Gap in Recurrent Language Models: The Role of the Gather-and-Aggregate Mechanism
Aviv Bick, Eric Xing, Albert Gu
State-space models (SSMs) offer efficient alternatives to Transformers for long sequences, but their fixed-size recurrent state limits capability on algorithmic tasks, such as retr…
Critique of World Model
Eric Xing, Mingkai Deng, Jinyu Hou
World Model, the algorithmic simulator of the real-world environment which biological agents experience and act upon, has been an emerging topic in recent years due to the rising n…
Betty: An Automatic Differentiation Library for Multilevel Optimization
Sang Keun Choe, Willie Neiswanger, Pengtao Xie +1
Gradient-based multilevel optimization (MLO) has gained attention as a framework for studying numerous problems, ranging from hyperparameter optimization and meta-learning to neura…
Multimodal Image Synthesis and Editing: The Generative AI Era
Fangneng Zhan, Yingchen Yu, Rongliang Wu +6
As information exists in various modalities in real world, effective interaction and fusion among multimodal information plays a key role for the creation and perception of multimo…
Cauchy Principal Component Analysis
Pengtao Xie, Eric Xing
Principal Component Analysis (PCA) has wide applications in machine learning, text mining and computer vision. Classical PCA based on a Gaussian noise model is fragile to noise of…
CryptGraph: Privacy Preserving Graph Analytics on Encrypted Graph
Pengtao Xie, Eric Xing
Many graph mining and analysis services have been deployed on the cloud, which can alleviate users from the burden of implementing and maintaining graph algorithms. However, puttin…
Weakly Supervised 3D Open-vocabulary Segmentation
Kunhao Liu, Fangneng Zhan, Jiahui Zhang +6
Open-vocabulary segmentation of 3D scenes is a fundamental function of human perception and thus a crucial objective in computer vision research. However, this task is heavily impe…
Linker-Tuning: Optimizing Continuous Prompts for Heterodimeric Protein Prediction
Shuxian Zou, Hui Li, Shentong Mo +3
Predicting the structure of interacting chains is crucial for understanding biological systems and developing new drugs. Large-scale pre-trained Protein Language Models (PLMs), suc…
Federated Learning via Posterior Averaging: A New Perspective and Practical Algorithms
Maruan Al-Shedivat, Jennifer Gillenwater, Eric Xing +1
Federated learning is typically approached as an optimization problem, where the goal is to minimize a global loss function by distributing computation across client devices that p…
ProBO: Versatile Bayesian Optimization Using Any Probabilistic Programming Language
Willie Neiswanger, Kirthevasan Kandasamy, Barnabas Poczos +2
Optimizing an expensive-to-query function is a common task in science and engineering, where it is beneficial to keep the number of queries to a minimum. A popular strategy is Baye…
VideoGLaMM: A Large Multimodal Model for Pixel-Level Visual Grounding in Videos
Shehan Munasinghe, Hanan Gani, Wenqi Zhu +4
Fine-grained alignment between videos and text is challenging due to complex spatial and temporal dynamics in videos. Existing video-based Large Multimodal Models (LMMs) handle bas…
Improved Logical Reasoning of Language Models via Differentiable Symbolic Programming
Hanlin Zhang, Jiani Huang, Ziyang Li +2
Pre-trained large language models (LMs) struggle to perform logical reasoning reliably despite advances in scale and compositionality. In this work, we tackle this challenge throug…
LLM360 K2: Building a 65B 360-Open-Source Large Language Model from Scratch
Zhengzhong Liu, Bowen Tan, Hongyi Wang +22
We detail the training of the LLM360 K2-65B model, scaling up our 360-degree OPEN SOURCE approach to the largest and most powerful models under project LLM360. While open-source LL…
Making Scalable Meta Learning Practical
Sang Keun Choe, Sanket Vaibhav Mehta, Hwijeen Ahn +4
Despite its flexibility to learn diverse inductive biases in machine learning programs, meta learning (i.e., learning to learn) has long been recognized to suffer from poor scalabi…
Nonparametric Latent Tree Graphical Models: Inference, Estimation, and Structure Learning
Le Song, Han Liu, Ankur Parikh +1
Tree structured graphical models are powerful at expressing long range or hierarchical dependency among many variables, and have been widely applied in different areas of computer…
Neural Architecture Search with Bayesian Optimisation and Optimal Transport
Kirthevasan Kandasamy, Willie Neiswanger, Jeff Schneider +2
Bayesian Optimisation (BO) refers to a class of methods for global optimisation of a function which is only accessible via point evaluations. It is typically used in settings w…
Community Specific Temporal Topic Discovery from Social Media
Zhiting Hu, Chong Wang, Junjie Yao +3
Studying temporal dynamics of topics in social media is very useful to understand online user behaviors. Most of the existing work on this subject usually monitors the global trend…
SlimPajama-DC: Understanding Data Combinations for LLM Training
Zhiqiang Shen, Tianhua Tao, Liqun Ma +8
This paper aims to understand the impacts of various data combinations (e.g., web text, Wikipedia, GitHub, books) on the pretraining of large language models using SlimPajama. Slim…
ALISON: Fast and Effective Stylometric Authorship Obfuscation
Eric Xing, Saranya Venkatraman, Thai Le +1
Authorship Attribution (AA) and Authorship Obfuscation (AO) are two competing tasks of increasing importance in privacy research. Modern AA leverages an author's consistent writing…
Explaining a black-box using Deep Variational Information Bottleneck Approach
Seojin Bang, Pengtao Xie, Heewook Lee +2
Interpretable machine learning has gained much attention recently. Briefness and comprehensiveness are necessary in order to provide a large amount of information concisely when ex…
Generative AI for Biosciences: Emerging Threats and Roadmap to Biosecurity
Zaixi Zhang, Souradip Chakraborty, Amrit Singh Bedi +16
The rapid adoption of generative artificial intelligence (GenAI) in the biosciences is transforming biotechnology, medicine, and synthetic biology. Yet this advancement is intrinsi…