80 citations · 353 across the 47 of their papers we have counts for
47 papers
When Prompts Become Pixels: Prompt-Region Grounding for Multimodal Reasoning
Yongxin Wang, Ruizhe Zhou, Yueling Tang +4
Multimodal large language models increasingly reason over screenshots and documents where the task itself may be written in pixels. Yet benchmarks usually place questions in text,…
Towards Deviation-Robust Agent Navigation via Perturbation-Aware Contrastive Learning
Bingqian Lin, Yanxin Long, Yi Zhu +4
Vision-and-language navigation (VLN) asks an agent to follow a given language instruction to navigate through a real 3D environment. Despite significant advances, conventional VLN…
Dynamic Graph Enhanced Contrastive Learning for Chest X-ray Report Generation
Mingjie Li, Bingqian Lin, Zicong Chen +3
Automatic radiology reporting has great clinical potential to relieve radiologists from heavy workloads and improve diagnosis interpretation. Recently, researchers have enhanced da…
Actional Atomic-Concept Learning for Demystifying Vision-Language Navigation
Bingqian Lin, Yi Zhu, Xiaodan Liang +2
Vision-Language Navigation (VLN) is a challenging task which requires an agent to align complex visual observations to language instructions to reach the goal position. Most existi…
CoupAlign: Coupling Word-Pixel with Sentence-Mask Alignments for Referring Image Segmentation
Zicheng Zhang, Yi Zhu, Jianzhuang Liu +2
Referring image segmentation aims at localizing all pixels of the visual objects described by a natural language sentence. Previous works learn to straightforwardly align the sente…
Learning Self-Regularized Adversarial Views for Self-Supervised Vision Transformers
Tao Tang, Changlin Li, Guangrun Wang +3
Automatic data augmentation (AutoAugment) strategies are indispensable in supervised data-efficient training protocols of vision transformers, and have led to state-of-the-art resu…
MARLlib: A Scalable and Efficient Multi-agent Reinforcement Learning Library
Siyi Hu, Yifan Zhong, Minquan Gao +6
A significant challenge facing researchers in the area of multi-agent reinforcement learning (MARL) pertains to the identification of a library that can offer fast and compatible d…
Cross-modal Clinical Graph Transformer for Ophthalmic Report Generation
Mingjie Li, Wenjia Cai, Karin Verspoor +3
Automatic generation of ophthalmic reports using data-driven neural networks has great potential in clinical practice. When writing a report, ophthalmologists make inferences with…
Policy Diagnosis via Measuring Role Diversity in Cooperative Multi-agent RL
Siyi Hu, Chuanlong Xie, Xiaodan Liang +1
Cooperative multi-agent reinforcement learning (MARL) is making rapid progress for solving tasks in a grid world and real-world scenarios, in which agents are given different attri…
Unbiased Math Word Problems Benchmark for Mitigating Solving Bias
Zhicheng Yang, Jinghui Qin, Jiaqi Chen +1
In this paper, we revisit the solving bias when evaluating models on current Math Word Problem (MWP) benchmarks. However, current solvers exist solving bias which consists of data…
"My nose is running.""Are you also coughing?": Building A Medical Diagnosis Agent with Interpretable Inquiry Logics
Wenge Liu, Yi Cheng, Hao Wang +6
With the rise of telemedicine, the task of developing Dialogue Systems for Medical Diagnosis (DSMD) has received much attention in recent years. Different from early researches tha…
Knowledge Distillation via the Target-aware Transformer
Sihao Lin, Hongwei Xie, Bing Wang +4
Knowledge distillation becomes a de facto standard to improve the performance of small neural networks. Most of the previous works propose to regress the representational features…
LogicSolver: Towards Interpretable Math Word Problem Solving with Logical Prompt-enhanced Learning
Zhicheng Yang, Jinghui Qin, Jiaqi Chen +2
Recently, deep learning models have made great progress in MWP solving on answer accuracy. However, they are uninterpretable since they mainly rely on shallow heuristics to achieve…
Automated Progressive Learning for Efficient Training of Vision Transformers
Changlin Li, Bohan Zhuang, Guangrun Wang +3
Recent advances in vision Transformers (ViTs) have come with a voracious appetite for computing power, high-lighting the urgent need to develop efficient training methods for ViTs.…
elBERto: Self-supervised Commonsense Learning for Question Answering
Xunlin Zhan, Yuan Li, Xiao Dong +3
Commonsense question answering requires reasoning about everyday situations and causes and effects implicit in context. Typically, existing approaches first retrieve external evide…
Contrastive Instruction-Trajectory Learning for Vision-Language Navigation
Xiwen Liang, Fengda Zhu, Yi Zhu +3
The vision-language navigation (VLN) task requires an agent to reach a target with the guidance of natural language instruction. Previous works learn to navigate step-by-step follo…
Image Comes Dancing with Collaborative Parsing-Flow Video Synthesis
Bowen Wu, Zhenyu Xie, Xiaodan Liang +3
Transferring human motion from a source to a target person poses great potential in computer vision and graphics applications. A crucial step is to manipulate sequential future mot…
IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoning
Pan Lu, Liang Qiu, Jiaqi Chen +6
Current visual question answering (VQA) tasks mainly consider answering human-annotated questions for natural images. However, aside from natural images, abstract diagrams with sem…
DS-Net++: Dynamic Weight Slicing for Efficient Inference in CNNs and Transformers
Changlin Li, Guangrun Wang, Bing Wang +3
Dynamic networks have shown their promising capability in reducing theoretical computation complexity by adapting their architectures to the input during inference. However, their…
Pi-NAS: Improving Neural Architecture Search by Reducing Supernet Training Consistency Shift
Jiefeng Peng, Jiqi Zhang, Changlin Li +3
Recently proposed neural architecture search (NAS) methods co-train billions of architectures in a supernet and estimate their potential accuracy using the network weights detached…
Medical-VLBERT: Medical Visual Language BERT for COVID-19 CT Report Generation With Alternate Learning
Guangyi Liu, Yinghong Liao, Fuyu Wang +8
Medical imaging technologies, including computed tomography (CT) or chest X-Ray (CXR), are largely employed to facilitate the diagnosis of the COVID-19. Since manual report writing…
Adversarial Reinforced Instruction Attacker for Robust Vision-Language Navigation
Bingqian Lin, Yi Zhu, Yanxin Long +3
Language instruction plays an essential role in the natural language grounded navigation tasks. However, navigators trained with limited human-annotated instructions may have diffi…
Towards Quantifiable Dialogue Coherence Evaluation
Zheng Ye, Liucun Lu, Lishan Huang +2
Automatic dialogue coherence evaluation has attracted increasing attention and is crucial for developing promising dialogue systems. However, existing metrics have two major limita…
Neural-Symbolic Solver for Math Word Problems with Auxiliary Tasks
Jinghui Qin, Xiaodan Liang, Yining Hong +2
Previous math word problem solvers following the encoder-decoder paradigm fail to explicitly incorporate essential math symbolic constraints, leading to unexplainable and unreasona…