5 papers · 1 filter
Metadata Management for AI-Augmented Data Workflows
Jinjin Zhao, Sanjay Krishnan
AI-augmented data workflows introduce complex governance challenges, as both human and model-driven processes generate, transform, and consume data artifacts. These workflows blend…
TableVault: Managing Dynamic Data Collections for LLM-Augmented Workflows
Jinjin Zhao, Sanjay Krishnan
Large Language Models (LLMs) have emerged as powerful tools for automating and executing complex data tasks. However, their integration into more complex data workflows introduces…
Fast Capture of Cell-Level Provenance in Numpy
Jinjin Zhao, Sanjay Krishnan
Effective provenance tracking enhances reproducibility, governance, and data quality in array workflows. However, significant challenges arise in capturing this provenance, includi…
Compression and In-Situ Query Processing for Fine-Grained Array Lineage
Jinjin Zhao, Sanjay Krishnan
Tracking data lineage is important for data integrity, reproducibility, and debugging data science workflows. However, fine-grained lineage (i.e., at a cell level) is challenging t…
Data Station: Delegated, Trustworthy, and Auditable Computation to Enable Data-Sharing Consortia with a Data Escrow
Siyuan Xia, Zhiru Zhu, Chris Zhu +7
Pooling and sharing data increases and distributes its value. But since data cannot be revoked once shared, scenarios that require controlled release of data for regulatory, privac…