5 papers
How You Move Tells What You'll Do: Trajectory-Conditioned Egocentric Prediction
Sejoon Jun, Hai Nguyen-Truong, Luigi Seminara +1
Predicting how a person's first-person view will evolve (what action will follow, what plan completes a task, whether an in-progress shot will score) is fundamentally under-specifi…
RECIPE: Procedural Planning via Grounding in Instructional Video
Luigi Seminara, Antonino Furnari, Lorenzo Torresani
Visual planning asks a model to generate the remaining steps of a procedure in natural language given a partial video context and a goal. Progress on this task is bottlenecked by a…
ViterbiPlanNet: Injecting Procedural Knowledge via Differentiable Viterbi for Planning in Instructional Videos
Luigi Seminara, Davide Moltisanti, Antonino Furnari
Procedural planning aims to predict a sequence of actions that transforms an initial visual state into a desired goal, a fundamental ability for intelligent agents operating in com…
Task Graph Maximum Likelihood Estimation for Procedural Activity Understanding in Egocentric Videos
Luigi Seminara, Giovanni Maria Farinella, Antonino Furnari
We introduce a gradient-based approach for learning task graphs from procedural activities, improving over hand-crafted methods. Our method directly optimizes edge weights via maxi…
Differentiable Task Graph Learning: Procedural Activity Representation and Online Mistake Detection from Egocentric Videos
Luigi Seminara, Giovanni Maria Farinella, Antonino Furnari
Procedural activities are sequences of key-steps aimed at achieving specific goals. They are crucial to build intelligent agents able to assist users effectively. In this context,…