7 citations · 18 across the 28 of their papers we have counts for
29 papers
CLAMP: Constrained Decoding for Vision-Language Embodied Planning
Tianyi Ma, Parisa Kordjamshidi
Embodied planning increasingly relies on vision-language models (VLMs) to translate instructions and visual observations into executable action sequences. However, fluent plans are…
MMGR: Multi-Modal Generative Reasoning Benchmark and Evaluation
Zefan Cai, Haoyi Qiu, Tianyi Ma +13
Modern multimodal generative models can synthesize visually compelling images and videos, but it remains unclear whether this visual fluency reflects genuine reasoning: when prompt…
Bayesian Network Structure Discovery Using Large Language Models
Yinghuan Zhang, Yufei Zhang, Parisa Kordjamshidi +1
Understanding probabilistic dependencies among variables is central to analyzing complex systems. Traditional structure learning methods often require extensive observational data…
Vision-and-Language Navigation with Analogical Textual Descriptions in LLMs
Yue Zhang, Tianyi Ma, Zun Wang +2
Integrating large language models (LLMs) into embodied AI models is becoming increasingly prevalent. However, existing zero-shot LLM-based Vision-and-Language Navigation (VLN) agen…
Breaking Down and Building Up: Mixture of Skill-Based Vision-and-Language Navigation Agents
Tianyi Ma, Yue Zhang, Zehao Wang +1
Vision-and-Language Navigation (VLN) poses significant challenges for agents to interpret natural language instructions and navigate complex 3D environments. While recent progress…
Extracting Probabilistic Knowledge from Large Language Models for Bayesian Network Parameterization
Aliakbar Nafar, Kristen Brent Venable, Zijun Cui +1
In this work, we evaluate the potential of Large Language Models (LLMs) in building Bayesian Networks (BNs) by approximating domain expert priors. LLMs have demonstrated potential…
SPARTUN3D: Situated Spatial Understanding of 3D World in Large Language Models
Yue Zhang, Zhiyang Xu, Ying Shen +2
Integrating the 3D world into large language models (3D-based LLMs) has been a promising research direction for 3D scene understanding. However, current 3D-based LLMs fall short in…
Learning vs Retrieval: The Role of In-Context Examples in Regression with Large Language Models
Aliakbar Nafar, Kristen Brent Venable, Parisa Kordjamshidi
Generative Large Language Models (LLMs) are capable of being in-context learners. However, the underlying mechanism of in-context learning (ICL) is still a major research question,…
Narrowing the Gap between Vision and Action in Navigation
Yue Zhang, Parisa Kordjamshidi
The existing methods for Vision and Language Navigation in the Continuous Environment (VLN-CE) commonly incorporate a waypoint predictor to discretize the environment. This simplif…
Prompt2DeModel: Declarative Neuro-Symbolic Modeling with Natural Language
Hossein Rajaby Faghihi, Aliakbar Nafar, Andrzej Uszok +2
This paper presents a conversational pipeline for crafting domain knowledge for complex neuro-symbolic models through natural language prompts. It leverages large language models t…
Consistent Joint Decision-Making with Heterogeneous Learning Models
Hossein Rajaby Faghihi, Parisa Kordjamshidi
This paper introduces a novel decision-making framework that promotes consistency among decisions made by diverse models while utilizing external knowledge. Leveraging the Integer…
NavHint: Vision and Language Navigation Agent with a Hint Generator
Yue Zhang, Quan Guo, Parisa Kordjamshidi
Existing work on vision and language navigation mainly relies on navigation-related losses to establish the connection between vision and language modalities, neglecting aspects of…
Reasoning over Uncertain Text by Generative Large Language Models
Aliakbar Nafar, Kristen Brent Venable, Parisa Kordjamshidi
This paper considers the challenges Large Language Models (LLMs) face when reasoning over text that includes information involving uncertainty explicitly quantified via probability…
Disentangling Extraction and Reasoning in Multi-hop Spatial Reasoning
Roshanak Mirzaee, Parisa Kordjamshidi
Spatial reasoning over text is challenging as the models not only need to extract the direct spatial information from the text but also reason over those and infer implicit spatial…
Teaching Probabilistic Logical Reasoning to Transformers
Aliakbar Nafar, Kristen Brent Venable, Parisa Kordjamshidi
In this paper, we evaluate the capability of transformer-based language models in making inferences over uncertain text that includes uncertain rules of reasoning. We cover both Pr…
VLN-Trans: Translator for the Vision and Language Navigation Agent
Yue Zhang, Parisa Kordjamshidi
Language understanding is essential for the navigation agent to follow instructions. We observe two kinds of issues in the instructions that can make the navigation task challengin…
GLUECons: A Generic Benchmark for Learning Under Constraints
Hossein Rajaby Faghihi, Aliakbar Nafar, Chen Zheng +7
Recent research has shown that integrating domain knowledge into deep learning architectures is effective -- it helps reduce the amount of required data, improves the accuracy of t…
The Role of Semantic Parsing in Understanding Procedural Text
Hossein Rajaby Faghihi, Parisa Kordjamshidi, Choh Man Teng +1
In this paper, we investigate whether symbolic semantic representations, extracted from deep semantic parsers, can help reasoning over the states of involved entities in a procedur…
Transfer Learning with Synthetic Corpora for Spatial Role Labeling and Reasoning
Roshanak Mirzaee, Parisa Kordjamshidi
Recent research shows synthetic data as a source of supervision helps pretrained language models (PLM) transfer learning to new target tasks/domains. However, this idea is less exp…
LOViS: Learning Orientation and Visual Signals for Vision and Language Navigation
Yue Zhang, Parisa Kordjamshidi
Understanding spatial and visual information is essential for a navigation agent who follows natural language instructions. The current Transformer-based VLN agents entangle the or…
Dynamic Relevance Graph Network for Knowledge-Aware Question Answering
Chen Zheng, Parisa Kordjamshidi
This work investigates the challenge of learning and reasoning for Commonsense Question Answering given an external source of knowledge in the form of a knowledge graph (KG). We pr…
Relevant CommonSense Subgraphs for "What if..." Procedural Reasoning
Chen Zheng, Parisa Kordjamshidi
We study the challenge of learning causal reasoning over procedural text to answer "What if..." questions when external commonsense knowledge is required. We propose a novel multi-…
DomiKnowS: A Library for Integration of Symbolic Domain Knowledge in Deep Learning
Hossein Rajaby Faghihi, Quan Guo, Andrzej Uszok +3
We demonstrate a library for the integration of domain knowledge in deep learning architectures. Using this library, the structure of the data is expressed symbolically via graph d…
Relational Gating for "What If" Reasoning
Chen Zheng, Parisa Kordjamshidi
This paper addresses the challenge of learning to do procedural reasoning over text to answer "What if..." questions. We propose a novel relational gating network that learns to fi…