TARexp: A Python Framework for Technology-Assisted Review Experiments
arXiv:2202.11827 · doi:10.1145/3477495.3531663
Abstract
Technology-assisted review (TAR) is an important industrial application of information retrieval (IR) and machine learning (ML). While a small TAR research community exists, the complexity of TAR software and workflows is a major barrier to entry. Drawing on past open source TAR efforts, as well as design patterns from the IR and ML open source software, we present an open source Python framework for conducting experiments on TAR algorithms. Key characteristics of this framework are declarative representations of workflows and experiment plans, the ability for components to play variable numbers of workflow roles, and state maintenance and restart capabilities. Users can draw on reference implementations of standard TAR algorithms while incorporating novel components to explore their research interests. The framework is available at https://github.com/eugene-yang/tarexp.
arXiv comment6 pages, 4 figures, accepted as a SIGIR 2022 demo paper
References in corpus (9)
- Open Source Software for Efficient and Transparent Reviews
- Simplified Data Wrangling with ir_datasets
- Autonomy and Reliability of Continuous Active Learning for Technology-Assisted Review
- libact: Pool-based Active Learning in Python
- A Survey on Deep Learning Toolkits and Libraries for Intelligent User Interfaces
- Heuristic Stopping Rules For Technology-Assisted Review
- On Minimizing Cost in Legal Document Review Workflows
- Certifying One-Phase Technology-Assisted Reviews
- Goldilocks: Just-Right Tuning of BERT for Technology-Assisted Review