187 citations · 423 across the 18 of their papers we have counts for
21 papers
PRAGMA: Revolut Foundation Model
Maxim Ostroukhov, Ruslan Mikhailov, Vladimir Iashin +11
Modern financial systems generate vast quantities of transactional and event-level data that encode rich economic signals. This paper presents PRAGMA, a family of foundation models…
Training and Evaluation of Guideline-Based Medical Reasoning in LLMs
Michael Staniek, Artem Sokolov, Stefan Riezler
Machine learning for early prediction in medicine has recently shown breakthrough performance, however, the focus on improving prediction accuracy has led to a neglect of faithful…
Cooperative Face Liveness Detection from Optical Flow
Artem Sokolov, Mikhail Nikitin, Anton Konushin
In this work, we proposed a novel cooperative video-based face liveness detection method based on a new user interaction scenario where participants are instructed to slowly move t…
3D Audio-Visual Segmentation
Artem Sokolov, Swapnil Bhosale, Xiatian Zhu
Recognizing the sounding objects in scenes is a longstanding objective in embodied AI, with diverse applications in robotics and AR/VR/MR. To that end, Audio-Visual Segmentation (A…
Imitating Language via Scalable Inverse Reinforcement Learning
Markus Wulfmeier, Michael Bloesch, Nino Vieillard +13
The majority of language model training builds on imitation learning. It covers pretraining, supervised fine-tuning, and affects the starting conditions for reinforcement learning…
Improving End-to-End Speech Translation by Imitation-Based Knowledge Distillation with Synthetic Transcripts
Rebekka Hubert, Artem Sokolov, Stefan Riezler
End-to-end automatic speech translation (AST) relies on data that combines audio inputs with text translation outputs. Previous work used existing large parallel corpora of transcr…
Make Every Example Count: On the Stability and Utility of Self-Influence for Learning from Noisy NLP Datasets
Irina Bejan, Artem Sokolov, Katja Filippova
Increasingly larger datasets have become a standard ingredient to advancing the state-of-the-art in NLP. However, data quality might have already become the bottleneck to unlock fu…
Scaling Up Influence Functions
Andrea Schioppa, Polina Zablotskaia, David Vilar +1
We address efficient calculation of influence functions for tracking predictions back to the training data. We propose and analyze a new approach to speeding up the inverse Hessian…
Bandits Don't Follow Rules: Balancing Multi-Facet Machine Translation with Multi-Armed Bandits
Julia Kreutzer, David Vilar, Artem Sokolov
Training data for machine translation (MT) is often sourced from a multitude of large corpora that are multi-faceted in nature, e.g. containing contents from multiple domains or di…
Don't Search for a Search Method -- Simple Heuristics Suffice for Adversarial Text Attacks
Nathaniel Berger, Stefan Riezler, Artem Sokolov +1
Recently more attention has been given to adversarial attacks on neural networks for natural language processing (NLP). A central research topic has been the investigation of searc…
Fixing exposure bias with imitation learning needs powerful oracles
Luca Hormann, Artem Sokolov
We apply imitation learning (IL) to tackle the NMT exposure bias problem with error-correcting oracles, and evaluate an SMT lattice-based oracle which, despite its excellent perfor…
Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets
Julia Kreutzer, Isaac Caswell, Lisa Wang +49
With the success of large-scale pre-training and multilingual modeling in Natural Language Processing (NLP), recent years have seen a proliferation of large, web-mined text dataset…
Sparse Perturbations for Improved Convergence in Stochastic Zeroth-Order Optimization
Mayumi Ohta, Nathaniel Berger, Artem Sokolov +1
Interest in stochastic zeroth-order (SZO) methods has recently been revived in black-box optimization scenarios such as adversarial black-box attacks to deep neural networks. SZO m…
Learning to Segment Inputs for NMT Favors Character-Level Processing
Julia Kreutzer, Artem Sokolov
Most modern neural machine translation (NMT) systems rely on presegmented inputs. Segmentation granularity importantly determines the input and output sequence lengths, hence the m…
Sparse Stochastic Zeroth-Order Optimization with an Application to Bandit Structured Prediction
Artem Sokolov, Julian Hitschler, Mayumi Ohta +1
Stochastic zeroth-order (SZO), or gradient-free, optimization allows to optimize arbitrary functions by relying only on function evaluations under parameter perturbations, however,…
Sockeye: A Toolkit for Neural Machine Translation
Felix Hieber, Tobias Domhan, Michael Denkowski +4
We describe Sockeye (version 1.12), an open-source sequence-to-sequence toolkit for Neural Machine Translation (NMT). Sockeye is a production-ready framework for training and apply…
A Shared Task on Bandit Learning for Machine Translation
Artem Sokolov, Julia Kreutzer, Kellen Sunderland +4
We introduce and describe the results of a novel shared task on bandit learning for machine translation. The task was organized jointly by Amazon and Heidelberg University for the…
Counterfactual Learning from Bandit Feedback under Deterministic Logging: A Case Study in Statistical Machine Translation
Carolin Lawrence, Artem Sokolov, Stefan Riezler
The goal of counterfactual learning for statistical machine translation (SMT) is to optimize a target SMT system from logged data that consist of user feedback to translations that…
Bandit Structured Prediction for Neural Sequence-to-Sequence Learning
Julia Kreutzer, Artem Sokolov, Stefan Riezler
Bandit structured prediction describes a stochastic optimization framework where learning is performed from partial feedback. This feedback is received in the form of a task loss e…
Stochastic Structured Prediction under Bandit Feedback
Artem Sokolov, Julia Kreutzer, Christopher Lo +1
Stochastic structured prediction under bandit feedback follows a learning protocol where on each of a sequence of iterations, the learner receives an input, predicts an output stru…
Bandit Structured Prediction for Learning from Partial Feedback in Statistical Machine Translation
Artem Sokolov, Stefan Riezler, Tanguy Urvoy
We present an approach to structured prediction from bandit feedback, called Bandit Structured Prediction, where only the value of a task loss function at a single predicted point,…