325 citations · 601 across the 37 of their papers we have counts for
19 papers · 1 filter
Leveraging Word Guessing Games to Assess the Intelligence of Large Language Models
Tian Liang, Zhiwei He, Jen-tse Huang +7
The automatic evaluation of LLM-based agent intelligence is critical in developing advanced LLM-based agents. Although considerable effort has been devoted to developing human-anno…
LAE-ST-MoE: Boosted Language-Aware Encoder Using Speech Translation Auxiliary Task for E2E Code-switching ASR
Guodong Ma, Wenxuan Wang, Yuke Li +3
Recently, to mitigate the confusion between different languages in code-switching (CS) automatic speech recognition (ASR), the conditionally factorized models, such as the language…
Not All Countries Celebrate Thanksgiving: On the Cultural Dominance in Large Language Models
Wenxuan Wang, Wenxiang Jiao, Jingyuan Huang +4
This paper identifies a cultural dominance issue within large language models (LLMs) due to the predominant use of English data in model training (e.g., ChatGPT). LLMs often provid…
Who is ChatGPT? Benchmarking LLMs' Psychological Portrayal Using PsychoBench
Jen-tse Huang, Wenxuan Wang, Eric John Li +6
Large Language Models (LLMs) have recently showcased their remarkable capacities, not only in natural language processing tasks but also across diverse domains such as clinical med…
All Languages Matter: On the Multilingual Safety of Large Language Models
Wenxuan Wang, Zhaopeng Tu, Chang Chen +4
Safety lies at the core of developing and deploying large language models (LLMs). However, previous safety benchmarks only concern the safety in one language, e.g. the majority lan…
An Image is Worth a Thousand Toxic Words: A Metamorphic Testing Framework for Content Moderation Software
Wenxuan Wang, Jingyuan Huang, Jen-tse Huang +4
The exponential growth of social media platforms has brought about a revolution in communication and content dissemination in human society. Nevertheless, these platforms are being…
GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher
Youliang Yuan, Wenxiang Jiao, Wenxuan Wang +4
Safety lies at the core of the development of Large Language Models (LLMs). There is ample work on aligning LLMs with human ethics and preferences, including data filtering in pret…
Emotionally Numb or Empathetic? Evaluating How LLMs Feel Using EmotionBench
Jen-tse Huang, Man Ho Lam, Eric John Li +5
Evaluating Large Language Models' (LLMs) anthropomorphic capabilities has become increasingly important in contemporary discourse. Utilizing the emotion appraisal theory from psych…
Generative Type Inference for Python
Yun Peng, Chaozheng Wang, Wenxuan Wang +2
Python is a popular dynamic programming language, evidenced by its ranking as the second most commonly used language on GitHub. However, its dynamic type system can lead to potenti…
Language-Routing Mixture of Experts for Multilingual and Code-Switching Speech Recognition
Wenxuan Wang, Guodong Ma, Yuke Li +1
Multilingual speech recognition for both monolingual and code-switching speech is a challenging task. Recently, based on the Mixture of Experts (MoE), many works have made good pro…
Validating Multimedia Content Moderation Software via Semantic Fusion
Wenxuan Wang, Jingyuan Huang, Chang Chen +5
The exponential growth of social media platforms, such as Facebook and TikTok, has revolutionized communication and content publication in human society. Users on these platforms c…
BiasAsker: Measuring the Bias in Conversational AI System
Yuxuan Wan, Wenxuan Wang, Pinjia He +3
Powered by advanced Artificial Intelligence (AI) techniques, conversational AI systems, such as ChatGPT and digital assistants like Siri, have been widely deployed in daily life. H…
Revisiting the Reliability of Psychological Scales on Large Language Models
Jen-tse Huang, Wenxiang Jiao, Man Ho Lam +3
Recent research has focused on examining Large Language Models' (LLMs) characteristics from a psychological standpoint, acknowledging the necessity of understanding their behaviora…
What Makes Good In-context Demonstrations for Code Intelligence Tasks with LLMs?
Shuzheng Gao, Xin-Cheng Wen, Cuiyun Gao +3
Pre-trained models of source code have gained widespread popularity in many code intelligence tasks. Recently, with the scaling of the model and corpus size, large language models…
ParroT: Translating during Chat using Large Language Models tuned with Human Translation and Feedback
Wenxiang Jiao, Jen-tse Huang, Wenxuan Wang +5
Large language models (LLMs) like ChatGPT have exhibited remarkable abilities on a wide range of natural language processing~(NLP) tasks, including various machine translation abil…
Improving the Transferability of Adversarial Samples by Path-Augmented Method
Jianping Zhang, Jen-tse Huang, Wenxuan Wang +5
Deep neural networks have achieved unprecedented success on diverse vision tasks. However, they are vulnerable to adversarial noise that is imperceptible to humans. This phenomenon…
ChatGPT or Grammarly? Evaluating ChatGPT on Grammatical Error Correction Benchmark
Haoran Wu, Wenxuan Wang, Yuxuan Wan +2
ChatGPT is a cutting-edge artificial intelligence language model developed by OpenAI, which has attracted a lot of attention due to its surprisingly strong ability in answering fol…
MTTM: Metamorphic Testing for Textual Content Moderation Software
Wenxuan Wang, Jen-tse Huang, Weibin Wu +5
The exponential growth of social media platforms such as Twitter and Facebook has revolutionized textual communication and textual content publication in human society. However, th…
Is ChatGPT A Good Translator? Yes With GPT-4 As The Engine
Wenxiang Jiao, Wenxuan Wang, Jen-tse Huang +3
This report provides a preliminary evaluation of ChatGPT for machine translation, including translation prompt, multilingual translation, and translation robustness. We adopt the p…