The Cost of Natural Language Processing

A Data Compression Approach

A Ph.D. Dissertation by Matt Mahoney

• What is the cost of NLP?
• Language models
• Model performance vs. size
  • 35 statistical language models
    • Character models
    • Prediction strategies
    • Semantic models
    • Syntactic models
    • Evaluating the human model
• Research issues
  • A neural network character model
  • Learning word boundaries
  • Shannon game simulation
• Conclusions