Computational Linguistics 9, Exercise 2: F-beta

Code (run with py -X utf8, after the chapter's script):

def fbeta(p, r, b): return (1 + b*b) * p * r / (b*b * p + r)
p, r, f1, acc = scores(*counts(gold, pred))
print(round(fbeta(p, r, 1), 3), round(fbeta(p, r, 2), 3), round(fbeta(p, r, 0.5), 3))

Output from the run (precision 0.5, recall 0.6):

F1 0.545, F2 0.577, F0.5 0.517

The formula is the F-score article's: F-beta = (1 + b^2) * P * R /
(b^2 * P + R). F2 weights recall more (here 0.577, closer to recall 0.6);
F0.5 weights precision more (0.517, closer to precision 0.5). Because recall
is slightly higher than precision, F2 comes out above F1 and F0.5 below it.

Which to report for a tool that flags sentences for a human to read: that is
my judgement, not a rule from the articles. If missing a lighthouse sentence
matters more than reading a few extra false alarms, report F2 (and recall).
If readers' time is scarce and false alarms are costly, report F0.5 (and
precision). Whichever is chosen, I would give precision and recall as well,
since a single F-number hides the trade-off. The Precision and recall article
makes the same kind of argument with its smoke-detector (recall) and criminal
justice (precision) examples.