Towards Feature Engineering with Human and AI's Knowledge: Understanding Data Science Practitioners' Perceptions in Human&AI-Assisted Feature Engineering Design
arXiv:2405.14107 · doi:10.1145/3643834.3661517
Abstract
As AI technology continues to advance, the importance of human-AI collaboration becomes increasingly evident, with numerous studies exploring its potential in various fields. One vital field is data science, including feature engineering (FE), where both human ingenuity and AI capabilities play pivotal roles. Despite the existence of AI-generated recommendations for FE, there remains a limited understanding of how to effectively integrate and utilize humans' and AI's knowledge. To address this gap, we design a readily-usable prototype, human\&AI-assisted FE in Jupyter notebooks. It harnesses the strengths of humans and AI to provide feature suggestions to users, seamlessly integrating these recommendations into practical workflows. Using the prototype as a research probe, we conducted an exploratory study to gain valuable insights into data science practitioners' perceptions, usage patterns, and their potential needs when presented with feature suggestions from both humans and AI. Through qualitative analysis, we discovered that the Creator of the feature (i.e., AI or human) significantly influences users' feature selection, and the semantic clarity of the suggested feature greatly impacts its adoption rate. Furthermore, our findings indicate that users perceive both differences and complementarity between features generated by humans and those generated by AI. Lastly, based on our study results, we derived a set of design recommendations for future human&AI FE design. Our findings show the collaborative potential between humans and AI in the field of FE.
Computational Notebooks, Human-AI Collaboration, Feature Recommendation
References in corpus (13)
- A Survey on Knowledge Graphs: Representation, Acquisition and Applications
- Showing Academic Performance Predictions during Term Planning: Effects on Students' Decisions, Behaviors, and Preferences
- To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making
- Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making
- Expanding Explainability: Towards Social Transparency in AI systems
- Human-AI Collaboration in Data Science: Exploring Data Scientists' Perceptions of Automated AI
- How AI Developers Overcome Communication Challenges in a Multidisciplinary Team: A Case Study
- Trust in AutoML: Exploring Information Needs for Establishing Trust in Automated Machine Learning Systems
- AutoDS: Towards Human-Centered Automation of Data Science
- FeatureExplorer: Interactive Feature Selection and Exploration of Regression Models for Hyperspectral Images
- Bias-Aware Design for Informed Decisions: Raising Awareness of Self-Selection Bias in User Ratings and Reviews
- Enabling Collaborative Data Science Development with the Ballet Framework
- FactSheets: Increasing Trust in AI Services through Supplier's Declarations of Conformity