224 citations · 308 across the 17 of their papers we have counts for
20 papers
Pixels to Keys: Exploring Spatial and Motion Cues in Gameplay Inverse Dynamics
Abhishek Pillai, Ekta Prashnani, Joohwan Kim +1
Video games offer scalable environments for studying perception and control in embodied agents. Abundant online gameplay videos could supply demonstrations, but they rarely include…
Training a Conditioned Video Game Agent on a VLM Annotated Dataset
Katrin Schmid, Iuri Frosio
Reinforcement Learning (RL) is a powerful but far from easy-to-use technique for policy learning. In the specific case of video games, access to the game engine is required to get…
VLMs for Videogame Data Annotation
Katrin Schmid, Iuri Frosio
Vision Language Models (VLMs) and Artificial Intelligence (AI) agents have revolutionized how engineers approach complex problems in real-world applications. Their adoption in vide…
Learning to Move Like Professional Counter-Strike Players
David Durst, Feng Xie, Vishnu Sarukkai +9
In multiplayer, first-person shooter games like Counter-Strike: Global Offensive (CS:GO), coordinated movement is a critical component of high-level strategic play. However, the co…
Online Overexposed Pixels Hallucination in Videos with Adaptive Reference Frame Selection
Yazhou Xing, Amrita Mazumdar, Anjul Patney +5
Low dynamic range (LDR) cameras cannot deal with wide dynamic range inputs, frequently leading to local overexposure issues. We present a learning-based system to reduce these arti…
The Best Defense is a Good Offense: Adversarial Augmentation against Adversarial Attacks
Iuri Frosio, Jan Kautz
Many defenses against adversarial attacks (\eg robust classifiers, randomization, or image purification) use countermeasures put to work only after the attack has been crafted. We…
Improve Agents without Retraining: Parallel Tree Search with Off-Policy Correction
Assaf Hallak, Gal Dalal, Steven Dalton +3
Tree Search (TS) is crucial to some of the most influential successes in reinforcement learning. Here, we tackle two major challenges with TS that limit its usability: \textit{dist…
Noise-Aware Video Saliency Prediction
Ekta Prashnani, Orazio Gallo, Joohwan Kim +3
We tackle the problem of predicting saliency maps for videos of dynamic scenes. We note that the accuracy of the maps reconstructed from the gaze data of a fixed number of observer…
Robust Vision-Based Cheat Detection in Competitive Gaming
Aditya Jonnalagadda, Iuri Frosio, Seth Schneider +2
Game publishers and anti-cheat companies have been unsuccessful in blocking cheating in online gaming. We propose a novel, vision-based approach that captures the final state of th…
Generating and Characterizing Scenarios for Safety Testing of Autonomous Vehicles
Zahra Ghodsi, Siva Kumar Sastry Hari, Iuri Frosio +5
Extracting interesting scenarios from real-world data as well as generating failure cases is important for the development and testing of autonomous systems. We propose efficient m…
How to Close Sim-Real Gap? Transfer with Segmentation!
Mengyuan Yan, Qingyun Sun, Iuri Frosio +2
One fundamental difficulty in robotic learning is the sim-real gap problem. In this work, we propose to use segmentation as the interface between perception and control, as a domai…
Accelerating Reinforcement Learning through GPU Atari Emulation
Steven Dalton, Iuri Frosio, Michael Garland
We introduce CuLE (CUDA Learning Environment), a CUDA port of the Atari Learning Environment (ALE) which is used for the development of deep reinforcement algorithms. CuLE overcome…
Importance Estimation for Neural Network Pruning
Pavlo Molchanov, Arun Mallya, Stephen Tyree +2
Structural pruning of neural network parameters reduces computation, energy, and memory transfer costs during inference. We propose a novel method that estimates the contribution o…
Metaoptimization on a Distributed System for Deep Reinforcement Learning
Greg Heinrich, Iuri Frosio
Training intelligent agents through reinforcement learning is a notoriously unstable procedure. Massive parallelization on GPUs and distributed systems has been exploited to genera…
Tackling 3D ToF Artifacts Through Learning and the FLAT Dataset
Qi Guo, Iuri Frosio, Orazio Gallo +2
Scene motion, multiple reflections, and sensor noise introduce artifacts in the depth reconstruction performed by time-of-flight cameras. We propose a two-stage, deep-learning appr…
Sim-to-Real Transfer of Accurate Grasping with Eye-In-Hand Observations and Continuous Control
Mengyuan Yan, Iuri Frosio, Stephen Tyree +1
In the context of deep learning for robotics, we show effective method of training a real robot to grasp a tiny sphere (1.37cm of diameter), with an original combination of system…
On Nearest Neighbors in Non Local Means Denoising
Iuri Frosio, Jan Kautz
To denoise a reference patch, the Non-Local-Means denoising filter processes a set of neighbor patches. Few Nearest Neighbors (NN) are used to limit the computational burden of the…
Reinforcement Learning through Asynchronous Advantage Actor-Critic on a GPU
Mohammad Babaeizadeh, Iuri Frosio, Stephen Tyree +2
We introduce a hybrid CPU/GPU version of the Asynchronous Advantage Actor-Critic (A3C) algorithm, currently the state-of-the-art method in reinforcement learning for various gaming…
Learning Adaptive Parameter Tuning for Image Processing
Jingming Dong, Iuri Frosio, Jan Kautz
The non-stationary nature of image characteristics calls for adaptive processing, based on the local image content. We propose a simple and flexible method to learn local tuning of…
Loss Functions for Neural Networks for Image Processing
Hang Zhao, Orazio Gallo, Iuri Frosio +1
Neural networks are becoming central in several areas of computer vision and image processing and different architectures have been proposed to solve specific problems. The impact…