End2You -- The Imperial Toolkit for Multimodal Profiling by End-to-End Learning
arXiv:1802.01115
Abstract
We introduce End2You -- the Imperial College London toolkit for multimodal profiling by end-to-end deep learning. End2You is an open-source toolkit implemented in Python and is based on Tensorflow. It provides capabilities to train and evaluate models in an end-to-end manner, i.e., using raw input. It supports input from raw audio, visual, physiological or other types of information or combination of those, and the output can be of an arbitrary representation, for either classification or regression tasks. To our knowledge, this is the first toolkit that provides generic end-to-end learning for profiling capabilities in either unimodal or multimodal cases. To test our toolkit, we utilise the RECOLA database as was used in the AVEC 2016 challenge. Experimental results indicate that End2You can provide comparable results to state-of-the-art methods despite no need of expert-alike feature representations, but self-learning these from the data "end to end".
References in corpus (2)
Cited by in corpus (8)
- MuSe 2020 -- The First International Multimodal Sentiment Analysis in Real-life Media Challenge and Workshop
- deepSELF: An Open Source Deep Self End-to-End Learning Framework
- Mask Detection and Breath Monitoring from Speech: on Data Augmentation, Feature Representation and Modeling
- Variants of BERT, Random Forests and SVM approach for Multimodal Emotion-Target Sub-challenge
- DeepSpectrumLite: A Power-Efficient Transfer Learning Framework for Embedded Speech and Audio Processing from Decentralised Data
- Using Self-Supervised Feature Extractors with Attention for Automatic COVID-19 Detection from Speech
- Facial Emotion Recognition using Deep Residual Networks in Real-World Environments
- Aalto's End-to-End DNN systems for the INTERSPEECH 2020 Computational Paralinguistics Challenge