Chapter 25 · Worked solutions
Chapter 25 workings: The Robot That Read Every Press Conference
Chapter twenty-five worked: manual checks on forty parsed press conferences, one success in five attempts, and a model's stated confidence.
Two short probability sums and one sentence. The first two use rates quoted in the chapter as rough orders of magnitude, not football-specific measurements.
The exercises
Forty press conferences at a 45 per cent issue rate
How many of forty parsed press conferences would need a manual check at a 45 per cent rate?
Expected count
40 × 0.45 = 18.
The answer
About 18, as a caution rather than a measured football figure.
At least one success in five tries
With independent success rates of 16.9 and 7.6 per cent, the chance of at least one success in five attempts.
Complement
P(at least one) = 1 − P(none) = 1 − (1 − p)5.
16.9 per cent
1 − 0.8315 = 60.4 per cent.
7.6 per cent
1 − 0.9245 = 32.6 per cent.
The answer
60 per cent and 33 per cent.
“A 70 per cent chance of a draw”
Explain simply why a model's spoken 70 per cent is not yet a stakeable probability.
The sentence
A language model's stated confidence is a way of talking, not a number checked against results, so I would score a season of its calls by confidence band first and only trust the bands that turn out true about as often as claimed.
The answer
It is uncalibrated talk until tested on held-out results, band by band.
Related on this site
These are the workings. The argument for them, and the exercises themselves, are in chapter 25 of the book, The Robot That Read Every Press Conference.
Out now · free to read on Kindle Unlimited