Testing truth, fatality rates, obesity risk and trampolines.

Episode Summary

Executive Summary: The episode examines how numbers can mislead during COVID-19: the UK’s 100,000-tests target was met only by changing what counted as a test, the virus’s true fatality risk is better expressed as infection fatality rate than case fatality rate, obesity is not 10x risk on current evidence, and lockdown altered children’s injury patterns rather than simply reducing them.

Main Topics: The 100,000 COVID tests target and metric manipulation (Priority: 5/5): The government claimed it met a 100,000-tests-per-day target, but the show argues this relied on redefining tests to include kits merely posted out, not completed or result-producing tests. Goodhart’s Law and targets (Priority: 5/5): The episode uses the testing target as a case study in Goodhart’s Law: once a measure becomes a target, it is vulnerable to distortion, gaming, and loss of usefulness. How dangerous is COVID-19? (Priority: 5/5): The show distinguishes between case fatality rate and infection fatality rate, explaining why the UK’s high observed death rate among confirmed cases overstates the danger to the general population. Antibody testing and estimating true fatality risk (Priority: 4/5): Experts discuss serology studies in China, California, Germany, and New York, showing how uncertain prevalence estimates affect estimates of infection fatality rate. Obesity and COVID risk (Priority: 4/5): The episode challenges claims that obesity raises COVID mortality risk tenfold, finding only limited evidence for increased hospitalization risk and stronger evidence for severe obesity as a risk factor. Children’s injuries during lockdown (Priority: 4/5): A hospital surgeon’s data show that childhood injuries did not collapse during lockdown; instead, injuries shifted from sports and travel to home-based falls, sibling conflict, and especially trampolines.

Key Arguments: The UK’s 100,000-tests claim was achieved by counting tests when they were posted, not when they were completed or produced a result. Changing a metric’s definition to meet a target is a textbook example of Goodhart’s Law. The case fatality rate is not a good estimate of personal risk because it is biased by limited testing toward the sickest patients. The infection fatality rate is the more meaningful measure, but it is hard to estimate accurately without broad, random testing and antibody surveys. Early serology studies gave very different estimates, showing the current uncertainty about how many people have been infected. Current evidence does not support a tenfold COVID death risk from obesity alone; severe obesity appears more strongly associated with complications than simple overweight. Lockdown reduced some injuries, such as sports and trips, but increased others like falls inside the home and trampoline injuries. Children seem to reallocate risk rather than eliminate it: fewer football injuries, more sibling and home accidents. Systematic, repeated, randomized antibody surveys are needed to understand infection spread and fatality risk properly.

Data Points: Daily tests claimed by UK government: 122,347 - Official figure for 30 April, used to say the 100,000 target was met Target announced by Matt Hancock: 100,000 tests per day - Government goal for end of April Posted tests included in daily total: more than 40,000 - Included in the 30 April figure even though they had only been mailed out Estimated actual tests carried out on 30 April: around 82,000 - Approximate number after subtracting posted kits from the official total Days before recording when target still missed: 3 days - Government failed to hit 100,000 even with posted tests included in each of the last three days UK confirmed-case fatality rate: close to 15% - Shown as misleading because testing was limited mainly to severe cases South Korea case fatality rate: about 2% - Compared with the UK to illustrate the effect of broader testing Estimated infection fatality rate in early China data: 0.66% - Imperial College estimate based on Wuhan and repatriation survey data Adjusted estimated infection fatality rate for UK: 0.99% - Imperial College estimate after age adjustments for UK population Potential infected share without lockdown: 81% - Model estimate of Great Britain epidemic spread without transmission reduction Potential deaths without lockdown: around 510,000 - Model estimate based on 81% infection and the estimated fatality rate Stanford antibody study implication: more than 50 times official cases - Suggested many more Californians had been infected than recorded Heinsberg study infection multiplier: five times more - German hotspot study found many more infections than official counts Heinsberg asymptomatic share: 20% - Proportion of infected people with no symptoms in that study New York City estimated infection share: up to a quarter - Based on random testing in New York State among people out in public NYC implied infection fatality rate: 0.5% to 0.8% - Derived from New York data and death counts Risk increase from obesity alone: no evidence of tenfold increase - Expert assessment on the claim made in media headlines Hospitalization risk with obesity: about 37% more - Some UK and US evidence suggests a modest increase in hospitalization risk Intensive care cases from overweight people: 35% - Compared with 42% of population, indicating no elevated risk for overweight category Population share of overweight category: 42% - Body mass index 25-30 group Intensive care cases from severely obese people: over 7% - Compared with about 3% of the population, suggesting elevated risk Population share of severely obese people: about 3% - Severely obese category in the comparison study Children injured at Essex hospital last year: 165 - Comparison period for pediatric injury study Children injured at Essex hospital during lockdown: 135 - Same time period during lockdown Football injuries during lockdown: down 78% - Reduction due to fewer team sports Sibling-related injuries during lockdown: up 150% - Increase in home-based conflict injuries Trampoline injuries during lockdown: 4 times the number last year - Largest rise in pediatric injury mechanism Risk estimate for younger children on trampolines: 14 times increase in injury - Cited from existing studies on trampoline injuries Relative danger of trampoline vs Joe Wicks: 20 times more dangerous - Humorous end-point comparing injury risk

Pivotal Quotes: "as soon as a measure becomes a target, it ceases to become a good measure" — Tom Chivers: Explaining Goodhart’s Law in relation to the 100,000-tests target "So before a swab had even seen the inside of a nostril, this is the epidemiological equivalent of saying the checks in the post." — Tim Harford: Mocking the inclusion of posted kits in the official test count "I don't think we have any evidence of a tenfold increase just based on obesity." — Professor Julian Hamilton Shield: Challenging the headline claim about obesity and COVID risk

Implications: The episode warns listeners to scrutinize headline metrics, especially during a crisis. It also shows that policy decisions need better testing, representative surveys, and careful interpretation of risk, or governments and media may overstate success and danger alike.