43 citations · 288 across the 47 of their papers we have counts for
54 papers
DuoDiscover: Scaling Autonomous Discovery Beyond the Evaluation Bottleneck
Shubham Agarwal, Akshat Gupta, Mike Heddes +21
Evaluation has been central to advances in AI, and it is equally critical to autonomous discovery. Yet scaling autonomous discovery to real-world engineering challenges faces a fun…
Meta-Reflective Evolution Effectively Improves Production AI Systems
Seung Won Wilson Yoo, Chang Zeng, Aditi Somayajula +7
Large Language Model (LLM)-guided evolutionary optimization has successfully improved prompts, programs, and even entire AI systems. Most optimization methods select a candidate fr…
Revisiting Prompt Engineering via Declarative Crowdsourcing
Aditya G. Parameswaran, Shreya Shankar, Parth Asawa +2
Large language models (LLMs) are incredibly powerful at comprehending and generating data in the form of text, but are brittle and error-prone. There has been an advent of toolkits…
Moving Fast With Broken Data
Shreya Shankar, Labib Fawaz, Karl Gyllstrom +1
Machine learning (ML) models in production pipelines are frequently retrained on the latest partitions of large, continually-growing datasets. Due to engineering bugs, partitions i…
Efficient and Compact Spreadsheet Formula Graphs
Dixin Tang, Fanchao Chen, Christopher De Leon +4
Spreadsheets are one of the most popular data analysis tools, wherein users can express computation as formulae alongside data. The ensuing dependencies are tracked as formula grap…
Transactional Panorama: A Conceptual Framework for User Perception in Analytical Visual Interfaces
Dixin Tang, Alan Fekete, Indranil Gupta +1
Many tools empower analysts and data scientists to consume analysis results in a visual interface, such as a dashboard. When the underlying data changes, these results need to be u…
Operationalizing Machine Learning: An Interview Study
Shreya Shankar, Rolando Garcia, Joseph M. Hellerstein +1
Organizations rely on machine learning engineers (MLEs) to operationalize ML, i.e., deploy and maintain ML pipelines in production. The process of operationalizing ML, or MLOps, co…
Rethinking Streaming Machine Learning Evaluation
Shreya Shankar, Bernease Herman, Aditya G. Parameswaran
While most work on evaluating machine learning (ML) models focuses on computing accuracy on batches of data, tracking accuracy alone in a streaming setting (i.e., unbounded, timest…
The Sky Above The Clouds
Sarah Chasins, Alvin Cheung, Natacha Crooks +14
Technology ecosystems often undergo significant transformations as they mature. For example, telephony, the Internet, and PCs all started with a single provider, but in the United…
Towards Observability for Production Machine Learning Pipelines
Shreya Shankar, Aditya Parameswaran
Software organizations are increasingly incorporating machine learning (ML) into their product offerings, driving a need for new data management tools. Many of these tools facilita…
Lux: Always-on Visualization Recommendations for Exploratory Dataframe Workflows
Doris Jung-Lin Lee, Dixin Tang, Kunal Agarwal +8
Exploratory data science largely happens in computational notebooks with dataframe APIs, such as pandas, that support flexible means to transform, clean, and analyze data. Yet, vis…
Production Machine Learning Pipelines: Empirical Analysis and Optimization Opportunities
Doris Xin, Hui Miao, Aditya Parameswaran +1
Machine learning (ML) is now commonplace, powering data-driven applications in various organizations. Unlike the traditional perception of ML in research, ML production pipelines a…
Enhancing the Interactivity of Dataframe Queries by Leveraging Think Time
Doris Xin, Devin Petersohn, Dixin Tang +5
We propose opportunistic evaluation, a framework for accelerating interactions with dataframes. Interactive latency is critical for iterative, human-in-the-loop dataframe workloads…
Deconstructing Categorization in Visualization Recommendation: A Taxonomy and Comparative Study
Doris Jung-Lin Lee, Vidya Setlur, Melanie Tory +2
Visualization recommendation (VisRec) systems provide users with suggestions for potentially interesting and useful next steps during exploratory data analysis. These recommendatio…
Whither AutoML? Understanding the Role of Automation in Machine Learning Workflows
Doris Xin, Eva Yiwei Wu, Doris Jung-Lin Lee +2
Efforts to make machine learning more widely accessible have led to a rapid increase in Auto-ML tools that aim to automate the process of training and deploying machine learning. T…
Fine-Grained Lineage for Safer Notebook Interactions
Stephen Macke, Hongpu Gong, Doris Jung-Lin Lee +3
Computational notebooks have emerged as the platform of choice for data science and analytical workflows, enabling rapid iteration and exploration. By keeping intermediate program…
Rapid Approximate Aggregation with Distribution-Sensitive Interval Guarantees
Stephen Macke, Maryam Aliakbarpour, Ilias Diakonikolas +2
Aggregating data is fundamental to data analytics, data exploration, and OLAP. Approximate query processing (AQP) techniques are often used to accelerate computation of aggregates…
Demystifying a Dark Art: Understanding Real-World Machine Learning Model Development
Angela Lee, Doris Xin, Doris Lee +1
It is well-known that the process of developing machine learning (ML) workflows is a dark-art; even experts struggle to find an optimal workflow leading to a high accuracy model. U…
Towards Scalable Dataframe Systems
Devin Petersohn, Stephen Macke, Doris Xin +7
Dataframes are a popular abstraction to represent, prepare, and analyze data. Despite the remarkable success of dataframe libraries in Rand Python, dataframes face performance issu…
SCATTERSEARCH: Visual Querying of Scatterplot Visualizations
Doris Jung-Lin Lee, Jaewoo Kim, Renxuan Wang +1
Scatterplots are one of the simplest and most commonly-used visualizations for understanding quantitative, multidimensional data. However, since scatterplots only depict two attrib…
Helix: Holistic Optimization for Accelerating Iterative Machine Learning
Doris Xin, Stephen Macke, Litian Ma +3
Machine learning workflow development is a process of trial-and-error: developers iterate on workflows by testing out small modifications until the desired accuracy is achieved. Un…
ShapeSearch: A Flexible and Efficient System for Shape-based Exploration of Trendlines
Tarique Siddiqui, Zesheng Wang, Paul Luh +2
Identifying trendline visualizations with desired patterns is a common and fundamental data exploration task. Existing visual analytics tools offer limited flexibility and expressi…
Helix: Accelerating Human-in-the-loop Machine Learning
Doris Xin, Litian Ma, Jialin Liu +3
Data application developers and data scientists spend an inordinate amount of time iterating on machine learning (ML) workflows -- by modifying the data pre-processing, model train…
Accelerating Human-in-the-loop Machine Learning: Challenges and Opportunities
Doris Xin, Litian Ma, Jialin Liu +3
Development of machine learning (ML) workflows is a tedious process of iterative experimentation: developers repeatedly make changes to workflows until the desired accuracy is atta…