Evidence of interrelated cognitive-like capabilities in large language models: Indications of artificial general intelligence or achievement?

Abstract

Large language models (LLMs) are advanced artificial intelligence (AI) systems that can perform a variety of tasks commonly found in human intelligence tests, such as defining words, performing calculations, and engaging in verbal reasoning. There are also substantial individual differences in LLM capacities. Given the consistent observation of a positive manifold and general intelligence factor in human samples, along with group-level factors (e.g., crystallized intelligence), we hypothesized that LLM test scores may also exhibit positive intercorrelations, which could potentially give rise to an artificial general ability (AGA) factor and one or more group-level factors. Based on a sample of 591 LLMs and scores from 12 tests aligned with fluid reasoning (Gf), domain-specific knowledge (Gkn), reading/writing (Grw), and quantitative knowledge (Gq), we found strong empirical evidence for a positive manifold and a general factor of ability. Additionally, we identified a combined Gkn/Grw group-level factor. Finally, the number of LLM parameters correlated positively with both general factor of ability and Gkn/Grw factor scores, although the effects showed diminishing returns. We interpreted our results to suggest that LLMs, like human cognitive abilities, may share a common underlying efficiency in processing information and solving problems, though whether LLMs manifest primarily achievement/expertise rather than intelligence remains to be determined. Finally, while models with greater numbers of parameters exhibit greater general cognitive-like abilities, akin to the connection between greater neuronal density and human general intelligence, other characteristics must also be involved.

9 pages, 2 figures

References in corpus (9)

• A Survey of Large Language Models
• Training Compute-Optimal Large Language Models
• How well do Large Language Models perform in Arithmetic tasks?
• Metrology for AI: From Benchmarks to Instruments
• Mitigating the Alignment Tax of RLHF
• In Search of Needles in a 11M Haystack: Recurrent Memory Finds What LLMs Miss
• LLMs cannot find reasoning errors, but can correct them given the error location
• Do Large Language Models Perform the Way People Expect? Measuring the Human Generalization Function
• How predictable is language model benchmark performance?