Optimization models of natural communication

Abstract

A family of information theoretic models of communication was introduced more than a decade ago to explain the origins of Zipf's law for word frequencies. The family is a based on a combination of two information theoretic principles: maximization of mutual information between forms and meanings and minimization of form entropy. The family also sheds light on the origins of three other patterns: the principle of contrast, a related vocabulary learning bias and the meaning-frequency law. Here two important components of the family, namely the information theoretic principles and the energy function that combines them linearly, are reviewed from the perspective of psycholinguistics, language learning, information theory and synergetic linguistics. The minimization of this linear function is linked to the problem of compression of standard information theory and might be tuned by self-organization.

References in corpus (7)

• Power-law distributions in empirical data
• Collaborative Tagging and Semiotic Dynamics
• Languages cool as they expand: Allometric scaling and the decreasing need for new words
• Compression and the origins of Zipf's law for word frequencies
• Zipf's law for word frequencies: word forms versus lemmas in long texts
• Notes on Kullback-Leibler Divergence and Likelihood
• The infochemical core

Cited by in corpus (9)

• Optimal coding and the origins of Zipfian laws
• Linguistic laws in biology
• Statistical laws in linguistics
• Polysemy and brevity versus frequency in language
• The polysemy of the words that children learn over time
• Parallels of human language in the behavior of bottlenose dolphins
• On the class of coding optimality of human languages and the origins of Zipf's law
• Dependency distance minimization predicts compression
• The advent and fall of a vocabulary learning bias from communicative efficiency