44 citations · 49 across the 4 of their papers we have counts for
5 papers
Towards Execution-Grounded Automated AI Research
Chenglei Si, Zitong Yang, Yejin Choi +3
Automated AI research holds great potential to accelerate scientific discovery. However, current LLMs often generate plausible-looking but ineffective ideas. Execution grounding ma…
AutoMetrics: Approximate Human Judgements with Automatically Generated Evaluators
Michael J. Ryan, Yanzhe Zhang, Amol Salunkhe +3
Evaluating user-facing AI applications remains a central challenge, especially in open-ended domains such as travel planning, clinical note generation, or dialogue. The gold standa…
The Ideation-Execution Gap: Execution Outcomes of LLM-Generated versus Human Research Ideas
Chenglei Si, Tatsunori Hashimoto, Diyi Yang
Large Language Models (LLMs) have shown promise in accelerating the scientific research pipeline. A key capability for this process is the ability to generate novel research ideas,…
Attacking Vision-Language Computer Agents via Pop-ups
Yanzhe Zhang, Tao Yu, Diyi Yang
Autonomous agents powered by large vision and language models (VLM) have demonstrated significant potential in completing daily computer tasks, such as browsing the web to book tra…
Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers
Chenglei Si, Diyi Yang, Tatsunori Hashimoto
Recent advancements in large language models (LLMs) have sparked optimism about their potential to accelerate scientific discovery, with a growing number of works proposing researc…