325 citations · 601 across the 37 of their papers we have counts for
19 papers · 1 filter
How Well Can LLMs Echo Us? Evaluating AI Chatbots' Role-Play Ability with ECHO
Man Tik Ng, Hui Tung Tse, Jen-tse Huang +3
The role-play ability of Large Language Models (LLMs) has emerged as a popular research direction. However, existing studies focus on imitating well-known public figures or fiction…
A Spectrum Evaluation Benchmark for Medical Multi-Modal Large Language Models
Jie Liu, Wenxuan Wang, Yihang Su +8
The significant breakthroughs of Medical Multi-Modal Large Language Models (Med-MLLMs) renovate modern healthcare with robust information synthesis and medical decision support. Ho…
Leveraging Word Guessing Games to Assess the Intelligence of Large Language Models
Tian Liang, Zhiwei He, Jen-tse Huang +7
The automatic evaluation of LLM-based agent intelligence is critical in developing advanced LLM-based agents. Although considerable effort has been devoted to developing human-anno…
Not All Countries Celebrate Thanksgiving: On the Cultural Dominance in Large Language Models
Wenxuan Wang, Wenxiang Jiao, Jingyuan Huang +4
This paper identifies a cultural dominance issue within large language models (LLMs) due to the predominant use of English data in model training (e.g., ChatGPT). LLMs often provid…
Who is ChatGPT? Benchmarking LLMs' Psychological Portrayal Using PsychoBench
Jen-tse Huang, Wenxuan Wang, Eric John Li +6
Large Language Models (LLMs) have recently showcased their remarkable capacities, not only in natural language processing tasks but also across diverse domains such as clinical med…
All Languages Matter: On the Multilingual Safety of Large Language Models
Wenxuan Wang, Zhaopeng Tu, Chang Chen +4
Safety lies at the core of developing and deploying large language models (LLMs). However, previous safety benchmarks only concern the safety in one language, e.g. the majority lan…
GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher
Youliang Yuan, Wenxiang Jiao, Wenxuan Wang +4
Safety lies at the core of the development of Large Language Models (LLMs). There is ample work on aligning LLMs with human ethics and preferences, including data filtering in pret…
Emotionally Numb or Empathetic? Evaluating How LLMs Feel Using EmotionBench
Jen-tse Huang, Man Ho Lam, Eric John Li +5
Evaluating Large Language Models' (LLMs) anthropomorphic capabilities has become increasingly important in contemporary discourse. Utilizing the emotion appraisal theory from psych…
BiasAsker: Measuring the Bias in Conversational AI System
Yuxuan Wan, Wenxuan Wang, Pinjia He +3
Powered by advanced Artificial Intelligence (AI) techniques, conversational AI systems, such as ChatGPT and digital assistants like Siri, have been widely deployed in daily life. H…
Revisiting the Reliability of Psychological Scales on Large Language Models
Jen-tse Huang, Wenxiang Jiao, Man Ho Lam +3
Recent research has focused on examining Large Language Models' (LLMs) characteristics from a psychological standpoint, acknowledging the necessity of understanding their behaviora…
ParroT: Translating during Chat using Large Language Models tuned with Human Translation and Feedback
Wenxiang Jiao, Jen-tse Huang, Wenxuan Wang +5
Large language models (LLMs) like ChatGPT have exhibited remarkable abilities on a wide range of natural language processing~(NLP) tasks, including various machine translation abil…
ChatGPT or Grammarly? Evaluating ChatGPT on Grammatical Error Correction Benchmark
Haoran Wu, Wenxuan Wang, Yuxuan Wan +2
ChatGPT is a cutting-edge artificial intelligence language model developed by OpenAI, which has attracted a lot of attention due to its surprisingly strong ability in answering fol…
MTTM: Metamorphic Testing for Textual Content Moderation Software
Wenxuan Wang, Jen-tse Huang, Weibin Wu +5
The exponential growth of social media platforms such as Twitter and Facebook has revolutionized textual communication and textual content publication in human society. However, th…
Is ChatGPT A Good Translator? Yes With GPT-4 As The Engine
Wenxiang Jiao, Wenxuan Wang, Jen-tse Huang +3
This report provides a preliminary evaluation of ChatGPT for machine translation, including translation prompt, multilingual translation, and translation robustness. We adopt the p…
Tencent's Multilingual Machine Translation System for WMT22 Large-Scale African Languages
Wenxiang Jiao, Zhaopeng Tu, Jiarui Li +3
This paper describes Tencent's multilingual machine translation systems for the WMT22 shared task on Large-Scale Machine Translation Evaluation for African Languages. We participat…
Understanding and Mitigating the Uncertainty in Zero-Shot Translation
Wenxuan Wang, Wenxiang Jiao, Shuo Wang +2
Zero-shot translation is a promising direction for building a comprehensive multilingual neural machine translation~(MNMT) system. However, its quality is still not satisfactory du…
Understanding and Improving Sequence-to-Sequence Pretraining for Neural Machine Translation
Wenxuan Wang, Wenxiang Jiao, Yongchang Hao +4
In this paper, we present a substantial step in better understanding the SOTA sequence-to-sequence (Seq2Seq) pretraining for neural machine translation~(NMT). We focus on studying…
Language Models are Good Translators
Shuo Wang, Zhaopeng Tu, Zhixing Tan +3
Recent years have witnessed the rapid advance in neural machine translation (NMT), the core of which lies in the encoder-decoder architecture. Inspired by the recent progress of la…
Rethinking the Value of Transformer Components
Wenxuan Wang, Zhaopeng Tu
Transformer becomes the state-of-the-art translation model, while it is not well studied how each intermediate component contributes to the model performance, which poses significa…