Amid a federal debate over whether children should still get annual flu shots, a new study shows that vaccine effectiveness can be measured reliably using data the health care system already collects, without running a randomized controlled trial.
The study, published this summer, exploits a natural experiment created by children's birth dates. Young children typically have an annual checkup around their birthday, and those with fall birthdays often receive the season's flu vaccine during that visit. Children with summer birthdays usually need a separate trip, which many families never get around to making. Because birth month is essentially random with respect to flu risk, the researchers compared health outcomes between the two groups to estimate how well the vaccine performed in five recent flu seasons.
The authors said the vaccine clearly worked as intended in every season they examined. For every 100 children whose vaccination was linked to birthday timing, there were between 9 and 14 fewer diagnosed influenza cases, depending on the season.
The analysis comes as U.S. officials clash over childhood flu vaccination. Earlier this year, the Department of Health and Human Services stopped recommending that all children receive an annual flu shot, making it instead a matter of «shared clinical decision-making» for parents and doctors to weigh case by case. The department cited, among other reasons, a lack of randomized controlled trials proving the vaccine's efficacy in children, including the very young.
Public health organizations sued, and a federal court blocked the change, leaving the previous recommendation in place. The case is ongoing. President Donald Trump also issued an executive order directing the government to treat a recent HHS assessment as a «guiding resource» and to revisit the childhood vaccine schedule.
That assessment argues that recent evidence behind annual flu shots for children, much of it from observational studies rather than randomized trials, is thin. The new study's authors said they believe the evidence base is stronger than the assessment suggests, noting that it includes many clinical trials. They also acknowledged, however, that observational studies of annual flu shots can suffer from statistical biases.
The statistical problem is familiar: comparing children who got the shot with those who did not can be misleading, because the two groups may differ in ways unrelated to the vaccine, such as how cautious their parents are or how often they visit a doctor. Randomized trials solve this by assigning the shot purely by chance, so the only expected difference between the groups is whether they received it.
Randomized data are not always available, but sometimes, as the researchers put it, the world randomizes people by accident. Young children tend to have their annual checkup around their birthday. Those with fall birthdays may see their pediatrician just as the season's vaccine arrives, making the shot convenient; children with summer birthdays have to make a separate appointment, which many families skip.
A prior study by the same researchers found that among children aged two to five, those with fall birthdays are more likely to be vaccinated, less likely to be diagnosed with the flu, and less likely to have a family member catch it than children with summer birthdays. For the new analysis, the team tracked vaccination and influenza rates among two- to five-year-olds with fall versus summer birthdays over five seasons. In a season in which fall-born children were vaccinated more but did not get the flu less often, the shot would appear ineffective, possibly because the strains it targeted were not circulating. Instead, the pattern consistently pointed to effectiveness.
To test whether something other than the vaccine explained the results, the researchers compared rates of non-influenza infections, such as stomach viruses and common colds, and found no difference between the two birthday groups. That result made it less likely that one group was simply seeing the doctor more often or was more health-conscious than the other.
The authors stress that randomized controlled trials remain the most rigorous form of evidence. But trials are slow, expensive, and logistically challenging. When it comes to existing treatments, trials can also be unethical, because researchers cannot withhold treatment believed to be effective just to keep proving the point.
If the federal government's concern is a lack of fresh randomized evidence for a long-established treatment, the study's authors argue that stopping the treatment and waiting for a trial that may never come is not the right solution. Instead, they call for making fuller use of the enormous quantity of data the health care system already generates, much of which sits idle and unexamined. The birth-date experiment, they note, required no new patient enrollment, no millions of dollars, and no years of waiting, and the same measurement could be repeated every season. «With a bit of creativity and rigorous statistical methods,» they wrote, «that evidence can be drawn from the data we already have.»