4 papers · 1 filter
BenchBench: Benchmarking Automated Benchmark Generation
Yandan Zheng, Haoran Luo, Zhenghong Lin +2
Benchmarks are the de facto standard for tracking progress in large language models (LLMs), yet static test sets can rapidly saturate, become vulnerable to contamination, and are c…
Asking the Right Questions: Ontology-Grounded Interpretable Embeddings for Biomedical Text
Yixuan Tang, Zhenghong Lin, Yandong Sun +3
While dense biomedical embeddings achieve strong performance, their opaque dimensions limit transparency in biomedical NLP. Recent question-based interpretable embeddings represent…
Probing Large Language Models in Reasoning and Translating Complex Linguistic Puzzles
Zheng-Lin Lin, Yu-Fei Shih, Shu-Kai Hsieh
This paper investigates the utilization of Large Language Models (LLMs) for solving complex linguistic puzzles, a domain requiring advanced reasoning and adept translation capabili…
Reasoning Over the Glyphs: Evaluation of LLM's Decipherment of Rare Scripts
Yu-Fei Shih, Zheng-Lin Lin, Shu-Kai Hsieh
We explore the capabilities of LVLMs and LLMs in deciphering rare scripts not encoded in Unicode. We introduce a novel approach to construct a multimodal dataset of linguistic puzz…