6 citations · 13 across the 17 of their papers we have counts for
20 papers
EmailBench: A Benchmark for Evaluating LLM Agents on Enterprise Email and Productivity Tasks
Mukul Singh, Mansi Uniyal, Devin Devlin +5
Enterprise email agents must combine information retrieval, structured state changes, temporal reasoning, and multi-step coordination. Recent agent benchmarks include productivity…
Scaling Competence, Shrinking Reasoning: Cognitive Signatures in Language Model Learning
Mukul Singh, Ananya Singha, Arjun Radhakrishna +1
We analyze reasoning in language models during task-specific fine-tuning and draws parallel between reasoning tokens--intermediate steps generated while solving problem and the hum…
Training Emergent Joint Associations: A Reinforcement Learning Approach to Creative Thinking in Language Models
Mukul Singh, Ananya Singha, Aishni Parab +2
Associative thinking--the ability to connect seemingly unrelated ideas--is a foundational element of human creativity and problem-solving. This paper explores whether reinforcement…
Do Code Models Suffer from the Dunning-Kruger Effect?
Mukul Singh, Somya Chatterjee, Arjun Radhakrishna +1
As artificial intelligence systems increasingly collaborate with humans in creative and technical domains, questions arise about the cognitive boundaries and biases that shape our…
Collaboration and Conflict between Humans and Language Models through the Lens of Game Theory
Mukul Singh, Arjun Radhakrishna, Sumit Gulwani
Language models are increasingly deployed in interactive online environments, from personal chat assistants to domain-specific agents, raising questions about their cooperative and…
Tabularis Formatus: Predictive Formatting for Tables
Mukul Singh, José Cambronero, Sumit Gulwani +2
Spreadsheet manipulation software are widely used for data management and analysis of tabular data, yet the creation of conditional formatting (CF) rules remains a complex task req…
Diffusion is a code repair operator and generator
Mukul Singh, Gust Verbruggen, Vu Le +1
Code diffusion models generate code by iteratively removing noise from the latent representation of a code snippet. During later steps of the diffusion process, when the code snipp…
Ordered Semantically Diverse Sampling for Textual Data
Ashish Tiwari, Mukul Singh, Ananya Singha +1
The goal of diversity sampling is to select a representative subset of data in a way that maximizes information contained in the subset while keeping its cardinality small. We intr…
An Empirical Study of Validating Synthetic Data for Formula Generation
Usneek Singh, José Cambronero, Sumit Gulwani +5
Large language models (LLMs) can be leveraged to help with writing formulas in spreadsheets, but resources on these formulas are scarce, impacting both the base performance of pre-…
Semantically Aligned Question and Code Generation for Automated Insight Generation
Ananya Singha, Bhavya Chopra, Anirudh Khatry +6
Automated insight generation is a common tactic for helping knowledge workers, such as data scientists, to quickly understand the potential value of new and unfamiliar data. Unfort…
Assessing GPT4-V on Structured Reasoning Tasks
Mukul Singh, José Cambronero, Sumit Gulwani +2
Multi-modality promises to unlock further uses for large language models. Recently, the state-of-the-art language model GPT-4 was enhanced with vision capabilities. We carry out a…
CodeFusion: A Pre-trained Diffusion Model for Code Generation
Mukul Singh, José Cambronero, Sumit Gulwani +3
Imagine a developer who can only change their last line of code, how often would they have to start writing a function from scratch before it is correct? Auto-regressive models for…
FormaT5: Abstention and Examples for Conditional Table Formatting with Natural Language
Mukul Singh, José Cambronero, Sumit Gulwani +5
Formatting is an important property in tables for visualization, presentation, and analysis. Spreadsheet software allows users to automatically format their tables by writing data-…
TST: Target Similarity Tuning Meets the Real World
Anirudh Khatry, Sumit Gulwani, Priyanshu Gupta +4
Target similarity tuning (TST) is a method of selecting relevant examples in natural language (NL) to code generation through large language models (LLMs) to improve performance. I…
InstructExcel: A Benchmark for Natural Language Instruction in Excel
Justin Payan, Swaroop Mishra, Mukul Singh +7
With the evolution of Large Language Models (LLMs) we can solve increasingly more complex NLP tasks across various domains, including spreadsheets. This work investigates whether L…
DataVinci: Learning Syntactic and Semantic String Repairs
Mukul Singh, José Cambronero, Sumit Gulwani +3
String data is common in real-world datasets: 67.6% of values in a sample of 1.8 million real Excel spreadsheets from the web were represented as text. Systems that successfully cl…
Demonstration of CORNET: A System For Learning Spreadsheet Formatting Rules By Example
Mukul Singh, Jose Cambronero, Sumit Gulwani +3
Data management and analysis tasks are often carried out using spreadsheet software. A popular feature in most spreadsheet platforms is the ability to define data-dependent formatt…
Personalized action suggestions in low-code automation platforms
Saksham Gupta, Gust Verbruggen, Mukul Singh +2
Automation platforms aim to automate repetitive tasks using workflows, which start with a trigger and then perform a series of actions. However, with many possible actions, the use…
From Words to Code: Harnessing Data for Program Synthesis from Natural Language
Anirudh Khatry, Joyce Cahoon, Jordan Henkel +9
Creating programs to correctly manipulate data is a difficult task, as the underlying programming languages and APIs can be challenging to learn for many users who are not skilled…
CORNET: Learning Table Formatting Rules By Example
Mukul Singh, José Cambronero, Sumit Gulwani +4
Spreadsheets are widely used for table manipulation and presentation. Stylistic formatting of these tables is an important property for both presentation and analysis. As a result,…