8 papers
Captions Speak Louder than Images: Generalizing Foundation Models for E-commerce from High-quality Multimodal Instruction Data
Xinyi Ling, Hanwen Du, Bo Peng +2
Leveraging multimodal data to drive breakthroughs in e-commerce applications through Multimodal Foundation Models (MFMs) is gaining increasing attention from the research community…
Planning with Diffusion Models for Target-Oriented Dialogue Systems
Hanwen Du, Bo Peng, Xia Ning
Target-Oriented Dialogue (TOD) remains a significant challenge in the LLM era, where strategic dialogue planning is crucial for directing conversations toward specific targets. How…
DiffER: Categorical Diffusion for Chemical Retrosynthesis
Sean Current, Ziqi Chen, Daniel Adu-Ampratwum +2
Methods for automatic chemical retrosynthesis have found recent success through the application of models traditionally built for natural language processing, primarily through tra…
ChemToolAgent: The Impact of Tools on Language Agents for Chemistry Problem Solving
Botao Yu, Frazier N. Baker, Ziru Chen +5
To enhance large language models (LLMs) for chemistry problem solving, several LLM-based agents augmented with tools have been proposed, such as ChemCrow and Coscientist. However,…
Analyzing Fine-Grained Alignment and Enhancing Vision Understanding in Multimodal Language Models
Jiachen Jiang, Jinxin Zhou, Bo Peng +2
Achieving better alignment between vision embeddings and Large Language Models (LLMs) is crucial for enhancing the abilities of Multimodal LLMs (MLLMs), particularly for recent mod…
SAPIENT: Mastering Multi-turn Conversational Recommendation with Strategic Planning and Monte Carlo Tree Search
Hanwen Du, Bo Peng, Xia Ning
Conversational Recommender Systems (CRS) proactively engage users in interactive dialogues to elicit user preferences and provide personalized recommendations. Existing methods tra…