164 citations · 291 across the 40 of their papers we have counts for
43 papers
Gemini Embedding 2: A Native Multimodal Embedding Model from Gemini
Madhuri Shanbhogue, Zhe Li, Shanfeng Zhang +86
We introduce Gemini Embedding 2, a native multimodal embedding model that allows embedding video, audio, image, and text modalities in a unified representation space. We leverage t…
Learning to Learn from Language Feedback with Social Meta-Learning
Jonathan Cook, Diego Antognini, Martin Klissarov +2
Large language models (LLMs) often struggle to learn from corrective feedback within a conversational context. They are rarely proactive in soliciting this feedback, even when face…
Improving Interactive In-Context Learning from Natural Language Feedback
Martin Klissarov, Jonathan Cook, Diego Antognini +5
Adapting one's thought process based on corrective feedback is an essential ability in human learning, particularly in collaborative settings. In contrast, the current large langua…
Position: Introspective Experience from Conversational Environments as a Path to Better Learning
Claudiu Cristian Musat, Jackson Tolins, Diego Antognini +3
Current approaches to AI training treat reasoning as an emergent property of scale. We argue instead that robust reasoning emerges from linguistic self-reflection, itself internali…
Sketch-to-Layout: Sketch-Guided Multimodal Layout Generation
Riccardo Brioschi, Aleksandr Alekseev, Emanuele Nevali +9
Graphic layout generation is a growing research area focusing on generating aesthetically pleasing layouts ranging from poster designs to documents. While recent research has explo…
InkFM: A Foundational Model for Full-Page Online Handwritten Note Understanding
Anastasiia Fadeeva, Vincent Coriou, Diego Antognini +2
Tablets and styluses are increasingly popular for taking notes. To optimize this experience and ensure a smooth and efficient workflow, it's important to develop methods for accura…
Representing Online Handwriting for Recognition in Large Vision-Language Models
Anastasiia Fadeeva, Philippe Schlattner, Andrii Maksai +4
The adoption of tablets with touchscreens and styluses is increasing, and a key feature is converting handwriting to text, enabling search, indexing, and AI assistance. Meanwhile,…
InkSight: Offline-to-Online Handwriting Conversion by Teaching Vision-Language Models to Read and Write
Blagoj Mitrevski, Arina Rak, Julian Schnitzler +4
Digital note-taking is gaining popularity, offering a durable, editable, and easily indexable way of storing notes in a vectorized form, known as digital ink. However, a substantia…
DSS: Synthesizing long Digital Ink using Data augmentation, Style encoding and Split generation
Aleksandr Timofeev, Anastasiia Fadeeva, Andrei Afonin +2
As text generative models can give increasingly long answers, we tackle the problem of synthesizing long text in digital ink. We show that the commonly used models for this task fa…
Character Queries: A Transformer-based Approach to On-Line Handwritten Character Segmentation
Michael Jungo, Beat Wolf, Andrii Maksai +2
On-line handwritten character segmentation is often associated with handwriting recognition and even though recognition models include mechanisms to locate relevant positions durin…
Sampling and Ranking for Digital Ink Generation on a tight computational budget
Andrei Afonin, Andrii Maksai, Aleksandr Timofeev +1
Digital ink (online handwriting) generation has a number of potential applications for creating user-visible content, such as handwriting autocompletion, spelling correction, and b…
Inkorrect: Online Handwriting Spelling Correction
Andrii Maksai, Henry Rowley, Jesse Berent +1
We introduce Inkorrect, a data- and label-efficient approach for online handwriting (Digital Ink) spelling correction - DISC. Unlike previous work, the proposed method does not req…
Recommending Burgers based on Pizza Preferences: Addressing Data Sparsity with a Product of Experts
Martin Milenkoski, Diego Antognini, Claudiu Musat
In this paper, we describe a method to tackle data sparsity and create recommendations in domains with limited knowledge about user preferences. We expand the variational autoencod…
OpenCSI: An Open-Source Dataset for Indoor Localization Using CSI-Based Fingerprinting
Arthur Gassner, Claudiu Musat, Alexandru Rusu +1
Many applications require accurate indoor localization. Fingerprint-based localization methods propose a solution to this problem, but rely on a radio map that is effort-intensive…
Modeling Online Behavior in Recommender Systems: The Importance of Temporal Context
Milena Filipovic, Blagoj Mitrevski, Diego Antognini +3
Recommender systems research tends to evaluate model performance offline and on randomly sampled targets, yet the same systems are later used to predict user behavior sequentially…
Momentum-based Gradient Methods in Multi-Objective Recommendation
Blagoj Mitrevski, Milena Filipovic, Diego Antognini +3
Multi-objective gradient methods are becoming the standard for solving multi-objective problems. Among others, they show promising results in developing multi-objective recommender…
Addressing Fairness in Classification with a Model-Agnostic Multi-Objective Algorithm
Kirtan Padh, Diego Antognini, Emma Lejal Glaude +2
The goal of fairness in classification is to learn a classifier that does not discriminate against groups of individuals based on sensitive attributes, such as race and gender. One…
Benefiting from Bicubically Down-Sampled Images for Learning Real-World Image Super-Resolution
Mohammad Saeed Rad, Thomas Yu, Claudiu Musat +3
Super-resolution (SR) has traditionally been based on pairs of high-resolution images (HR) and their low-resolution (LR) counterparts obtained artificially with bicubic downsamplin…
Interacting with Explanations through Critiquing
Diego Antognini, Claudiu Musat, Boi Faltings
Using personalized explanations to support recommendations has been shown to increase trust and perceived quality. However, to actually obtain better recommendations, there needs t…
Control, Generate, Augment: A Scalable Framework for Multi-Attribute Text Generation
Giuseppe Russo, Nora Hollenstein, Claudiu Musat +1
We introduce CGA, a conditional VAE architecture, to control, generate, and augment text. CGA is able to generate natural English sentences controlling multiple semantic and syntac…
A Swiss German Dictionary: Variation in Speech and Writing
Larissa Schmidt, Lucy Linder, Sandra Djambazovska +3
We introduce a dictionary containing forms of common words in various Swiss German dialects normalized into High German. As Swiss German is, for now, a predominantly spoken languag…
Fast Cross-domain Data Augmentation through Neural Sentence Editing
Guillaume Raille, Sandra Djambazovska, Claudiu Musat
Data augmentation promises to alleviate data scarcity. This is most important in cases where the initial data is in short supply. This is, for existing methods, also where augmenti…
Multi-Gradient Descent for Multi-Objective Recommender Systems
Nikola Milojkovic, Diego Antognini, Giancarlo Bergamin +2
Recommender systems need to mirror the complexity of the environment they are applied in. The more we know about what might benefit the user, the more objectives the recommender sy…
Automatic Creation of Text Corpora for Low-Resource Languages from the Internet: The Case of Swiss German
Lucy Linder, Michael Jungo, Jean Hennebert +2
This paper presents SwissCrawl, the largest Swiss German text corpus to date. Composed of more than half a million sentences, it was generated using a customized web scraping tool…