Training and Scaling Preference Functions for Disambiguation

Abstract

We present an automatic method for weighting the contributions of preference functions used in disambiguation. Initial scaling factors are derived as the solution to a least-squares minimization problem, and improvements are then made by hill-climbing. The method is applied to disambiguating sentences in the ATIS (Air Travel Information System) corpus, and the performance of the resulting scaling factors is compared with hand-tuned factors. We then focus on one class of preference function, those based on semantic lexical collocations. Experimental results are presented showing that such functions vary considerably in selecting correct analyses. In particular we define a function that performs significantly better than ones based on mutual information and likelihood ratios of lexical associations.

To appear in Computational Linguistics (probably volume 20, December 94). LaTeX, 21 pages

Cited by in corpus (10)

• Generalizing Case Frames Using a Thesaurus and the MDL Principle
• Designing Statistical Language Learners: Experiments on Noun Compounds
• Adapting the Core Language Engine to French and Spanish
• Heuristics and Parse Ranking
• Hybrid Transfer in an English-French Spoken Language Translator
• A Probabilistic Disambiguation Method Based on Psycholinguistic Principles
• The Speech-Language Interface in the Spoken Language Translator
• Improving Language Models by Clustering Training Sentences
• Using the Web as an Implicit Training Set: Application to Noun Compound Syntax and Semantics
• Estimating Performance of Pipelined Spoken Language Translation Systems