Given a string y, a language model gives us a probability, P(y).
P(the big dog) > P(dog big the) > P(dgo gib eth)
- Easy for humans
- Difficult for machines
Application to speech recognition
- x = Input (speech)
- y = Output (text)
- max P(y|x) = max P(x|y)P(y) (acoustic model
´ language model)
Language modeling solves the Turing test for AI
- P(qa)/P(q) = Probability of human answering q with a
- P(qb)/P(q) = Probability of machine answering q with b
- If P(qa) = P(qb) then machine successfully imitates a human.