6 citations · 14 across the 17 of their papers we have counts for
18 papers
Hybrid Latent Attention for Looped Language Models
Yuhan Chen, Siyuan Zhang, Nan Wang +2
Looped language models apply the same stack of layers T times to each token, which deepens the model without adding parameters but multiplies its key-value (KV) cache by T. The lar…
Belief-Calibrated Optimization: An Explicit World Model for Agentic Optimization
Yuhan Chen, Zhihua Tian, Mahavir Dabas +7
The performance of an LLM agent depends on the scaffold around a frozen model. A common way to improve that scaffold is to use a coding agent as an optimizer: it reads current scor…
Can Generalist Agents Automate Data Curation?
Feiyang Kang, Hanze Li, Adam Nguyen +5
Curating training data is among the most consequential yet labor-intensive parts of modern AI development: practitioners iteratively propose, implement, evaluate, and revise data p…
Characterizing Model-Native Skills
Feiyang Kang, Mahavir Dabas, Myeongseob Ko +1
Skills are a natural unit for describing what a language model can do and how its behavior can be changed. However, existing characterizations rely on human-written taxonomies, tex…
MixAtlas: Uncertainty-aware Data Mixture Optimization for Multimodal LLM Midtraining
Bingbing Wen, Sirajul Salekin, Feiyang Kang +4
Domain reweighting can improve sample efficiency and downstream generalization, but data-mixture optimization for multimodal midtraining remains largely unexplored. Current multimo…
A Sustainable AI Economy Needs Data Deals That Work for Generators
Ruoxi Jia, Luis Oala, Wenjie Xiong +4
We argue that the machine learning value chain is structurally unsustainable due to an economic data processing inequality: each state in the data cycle from inputs to model weight…
Demystifying Synthetic Data in LLM Pre-training: A Systematic Study of Scaling Laws, Benefits, and Pitfalls
Feiyang Kang, Newsha Ardalani, Michael Kuchnik +7
Training data plays a crucial role in Large Language Models (LLM) scaling, yet high quality data is of limited supply. Synthetic data techniques offer a potential path toward sides…
Quagmires in SFT-RL Post-Training: When High SFT Scores Mislead and What to Use Instead
Feiyang Kang, Michael Kuchnik, Karthik Padthe +4
In post-training for reasoning Large Language Models (LLMs), the current state of practice trains LLMs in two independent stages: Supervised Fine-Tuning (SFT) and Reinforcement Lea…
AdaDeDup: Adaptive Hybrid Data Pruning for Efficient Large-Scale Object Detection Training
Feiyang Kang, Nadine Chang, Maying Shen +4
The computational burden and inherent redundancy of large-scale datasets challenge the training of contemporary machine learning models. Data pruning offers a solution by selecting…
AutoScale: Scale-Aware Data Mixing for Pre-Training LLMs
Feiyang Kang, Yifan Sun, Bingbing Wen +4
Domain reweighting is an emerging research area aimed at adjusting the relative weights of different data sources to improve the effectiveness and efficiency of LLM pre-training. W…
Get more for less: Principled Data Selection for Warming Up Fine-Tuning in LLMs
Feiyang Kang, Hoang Anh Just, Yifan Sun +5
This work focuses on leveraging and selecting from vast, unlabeled, open data to pre-fine-tune a pre-trained language model. The goal is to minimize the need for costly domain-spec…
FASTTRACK: Fast and Accurate Fact Tracing for LLMs
Si Chen, Feiyang Kang, Ning Yu +1
Fact tracing seeks to identify specific training examples that serve as the knowledge source for a given query. Existing approaches to fact tracing rely on assessing the similarity…
The Mirrored Influence Hypothesis: Efficient Data Influence Estimation by Harnessing Forward Passes
Myeongseob Ko, Feiyang Kang, Weiyan Shi +3
Large-scale black-box models have become ubiquitous across numerous applications. Understanding the influence of individual training data sources on predictions made by these model…
Data Acquisition: A New Frontier in Data-centric AI
Lingjiao Chen, Bilge Acun, Newsha Ardalani +8
As Machine Learning (ML) systems continue to grow, the demand for relevant and comprehensive datasets becomes imperative. There is limited study on the challenges of data acquisiti…
Performance Scaling via Optimal Transport: Enabling Data Selection from Partially Revealed Sources
Feiyang Kang, Hoang Anh Just, Anit Kumar Sahu +1
Traditionally, data selection has been studied in settings where all samples from prospective sources are fully revealed to a machine learning developer. However, in practical data…
LAVA: Data Valuation without Pre-Specified Learning Algorithms
Hoang Anh Just, Feiyang Kang, Jiachen T. Wang +4
Traditionally, data valuation (DV) is posed as a problem of equitably splitting the validation performance of a learning algorithm among the training data. As a result, the calcula…
Visual Navigation with a 2-pixel Camera---Possibilities and Limitations
John Baillieul, Feiyang Kang
Borrowing terminology from fluid mechanics, the concepts of {\em Eulerian} and {\em Lagrangian optical flow sensing} are introduced. Eulerian optical flow sensing assumes that each…
Prediction-Based Fast Thermoelectric Generator Reconfiguration for Energy Harvesting from Vehicle Radiators
Hanchen Yang, Feiyang Kang, Caiwen Ding +7
Thermoelectric generation (TEG) has increasingly drawn attention for being environmentally friendly. A few researches have focused on improving TEG efficiency at the system level o…