After two lost statistical years, redrawing meaning from the numbers that have always told the story of our lives

In early March 2020, while the rest of the country fought over toilet paper and pasta, Brenda Bugge sat in her Ottawa office worrying about numbers.

As the assistant director of Statistics Canada’s National Economics Account division, Bugge and her team build the models that measure how Canada’s economy performs. They are based on international standards that measure gross domestic product (GDP). Canada tweaks the formula locally so it accurately reflects an enormous range of variables.

The dawn of the pandemic and the panic that ensued presented a fascinating challenge for Bugge and her team. How would they accurately and practically measure the economy while social and economic rules that had been written over generations were being thrown aside?

As far as the numbers that tell the story of our lives, what remained relevant and what was not?

“It’s been this massive rollercoaster,” Bugge told The Pointer in August. For almost two years, she has led a team that has been forced to constantly reevaluate its most basic assumptions.

Flight bans have disrupted tourism patterns and capacity limits moved the restaurant industry to a mere shell of its usual self. Theatres spent months completely empty and athletic arenas were transformed into vaccination centres.

The challenge has been reflecting new economic activity, sports records, academic data and all the other dramatic changes to life without creating statistics that will be useless when compared to non-pandemic times. 

What should the Toronto Raptors, the entire NBA and the rest of the professional sports world do about the statistics over the last two years? 

(Image Wikimedia Commons)

“We have been nimble and been able to change … just our weighting methodology,” she said. “For example, in a restaurant, there are certain inputs with regards to food, labour, capital, all of those inputs have a certain weighting methodology in our models. What we needed to do was we needed to change that, so when restaurants changed from doing eating in to doing takeout, we modified our models to take into account that things were changing.”

Even with the type of disciplined statistical balancing act that Bugge and her team perform daily, there are many inevitable factors that will dramatically skew how we are fed numbers coming out of the pandemic.

Politicians around the world, for example, have already been using year-over-year economic data to claim amazing performance under their leadership, failing to point out impressive gains in 2021—while historically better than the crippled previous year—were still not close to the markers of global (and national) economic performance in 2019.

The statistical anomalies distorting the story of the past two years are found across the board.

The pandemic had an immeasurable impact on our lives. We can count the concerts that were cancelled or the birthdays celebrated over Zoom without being able to quantify missing the birth of a child or the premature passing of a friend. The virus changed how we greet each other and how we date. The sports teams we follow were turned upside down and inside out. We may not be able to measure the true consequence, but an entire industry of statisticians is trying. 

COVID-19 impacted our lives in so many ways and statisticians are trying to understand how they can accurately document it.

(Image from fusion_medical_animation/Unsplash)

The result of the latest variant, Omicron, is yet to be truly understood. While doctors try to comprehend what the latest case counts and hospitalizations mean for restrictions, men and women are picking through the broader numbers that track our last two years on Earth. They’re working out how to contextualize it for current analysis and future historians to gain value and insight from the chaos, while trying to ensure entire sectors are not thrown upside down for decades.

Take for example school performance data and various university entry benchmarks. Can student achievement over the past two years be folded into longitudinal statistical analysis to develop new standards? Do we need to throw the last two years of standardized testing out the window? Is there a way to weight the recent performance of students so the last two years do not dramatically alter expectations?

And what about sports?

The NBA, for example, has seen a revolving door of players and coaches sitting out during the pandemic due to health and safety protocols over the past three seasons. Should playing against inferior competition or under dramatic conditions—the Toronto Raptors had to play all their “home” games in Tampa, Florida last year where fans often cheered for the opposing team and the last two seasons did not even include the usual 82-game schedule (some teams even played fewer games during the 2019-2020 pandemic-shortened season)—mean all statistics over recent years need to have an asterisk beside them?

How can the records and statistics for professional sports over the last two years around the world be folded into the historic record?

What about the Olympics held during the global crisis?

The gatekeepers of the numerical world we live in are doing their best.

“You can work with all data, [for] statisticians, the more data we have, the better,” Jeffrey Rosenthal, a professor of statistics at the University of Toronto, explained. “Really, our profession is trying to make the most out of whatever data we can get. So we would never say, ‘Oh, we're just going to ignore that data because it doesn't quite fit in with our model.’”

Data helps measure trends and performance, it will be used by future historians to preserve our age. How it is recorded and filtered during the pandemic could directly dictate how this period is viewed by generations to come.

Rosenthal worked on an article that will appear in a March 2022 edition of the Lancet Regional Health journal. The paper considered non-natural death in Ontario during the pandemic using data from the Office of the Chief Coroner, tracking assumptions for how lockdowns may have impacted road traffic fatalities, deaths due to drug overdoses or suicide.

The data show some basic assumptions about the fallout of lockdowns may be off-base. “There was no major change to the rate of homicides during 2020 compared to 2009-2019,” the paper states, adding that road traffic accidents only decreased marginally and the rate of suicides “did not show an overall major change in 2020”.

One area that saw a significant increase was in deaths from drug overdoses. Carefully peeling away the noise from COVID-19, Rosenthal says he and his colleagues concluded the pandemic made drug overdoses even worse.

“That was an example that was challenging because that's already been going up,” he explained. “The overall rates of opioid deaths have been going up quite a bit in recent years. And then, well, in the year 2020, what happened? They went up again.”

But again, whether numbers went up or down, how should they be placed into historical context to understand our collective reality, when the global health crisis clearly had a skewing impact?

The team used something called interrupted time series testing to evaluate if the rise in 2020 was higher as a result of the pandemic. Their analysis showed that, even with deaths increasing every year, the pandemic caused an even larger spike than might otherwise have taken place. While it is possible to pull data from the wreckage of COVID-19, the task is incredibly complex.

Drug overdoses have increased in the pandemic at an even higher rate than predicted, including in Peel.

(Image from Isaac Callan/The Pointer)

Other patterns are harder to tease out at a macro level.

Ridership on public transit has still not come close to a pre-pandemic recovery, while the arts sector has been decimated. Trends can be coaxed from this data using smaller chunks but it is perhaps less useful at a broader level. As Rosenthal explains, the data does not help us understand how popular going to the cinema was in general during 2020 since it was banned for months at a time; we can learn things like what genre of film was the most popular during the brief times movie trips were permitted.

In the world of sport, the impact of COVID-19 may be even harder to track.

Lockdowns kept fans out of arenas for much of the pandemic and massively disrupted fixture schedules. Players in sports like soccer and hockey found themselves isolated from families to finish the 2019/20 season. The physical effects may also have played a part — Newcastle United winger Allan Saint-Maximin, for example, was unable to play for more than two months after he contracted the virus.

How these factors change the individual performance of different people can vary massively. As a result, plucking the impacts of the pandemic from the data can be devilishly tough.

This is made even more convoluted by the fact players have been subjected to a range of brutal conditions for years.

Hockey and basketball players flit between time zones on a near-constant basis. The Winnipeg Jets will play in Las Vegas on January 2, then in Arizona and Colorado before hosting the Seattle Kraken in Manitoba less than a week later, on January 8. Elite soccer players in England compete in multiple competitions, playing in the Champions League, the Premier League and two domestic cups.

Professional athletes were impacted by COVID-19 but how much did it translate into performance changes?

(Image from Kaz Andrew licensed under CC BY-SA 2.0)

The result of travel and fixture congestion is played off against breakthroughs in sports conditioning through technological advancements. Stars are using the best equipment and technology-assisted training in history, with teams of medical experts planning their meals and monitoring everything from blood oxygen levels to heart rates and sleep. These changes are relatively recent: it was not until 2003 that UEFA banned smoking on the touchline of European soccer matches.

Despite the way sport has morphed through history — and the fact modern sports stars would almost certainly be unable to play against the men and women who paved the way — sports records are viewed as constant through time. The record for most goals in a season does not depend on who made the puck, nor does the presence of undersoil heating discredit an unbeaten run of home games in soccer.

Should COVID-19 be any different when we look back at records and results in years to come?

“There’s such joy in winning … so I don’t think you’re going to volunteer very many asterisks,” Bruce Kidd, a former Canadian Olympian, Order of Canada member and professor at the University of Toronto, told The Pointer.

Whether fans are willing to admit it or not, he said it’s hard to pretend the pandemic hasn’t had some impact on how sport is consumed and played. Whether that is different from other changes that have taken place throughout history is less clear.

“They talk about the crowd as a fifth man or a sixth man, depending on the sport,” he said. “Clearly during the Raptors’ great year in Toronto, again and again, and again they talked about having the audience on their side. It is buoyant, it is like being swept along by a wave.”

Without that wave, basketball, soccer, hockey and many other arena sports were patently different.

There is not a consensus over whether it was different enough to merit a disclaimer—or if it must be taken at face value.

In the world of economics, which is inextricably linked to our politics, Bugge and her team are doing both. They’re carefully adjusting the way they measure GDP to accurately depict what is going on, painfully aware they will still need to add asterisks in the future as well.

“Our drop in GDP during the pandemic, we’ve never seen such a drop,” she said. “I think that 2020 is unprecedented, I think it’ll always be something that we need to refer to. Whenever we’re talking about the GDP numbers, we will always need to refer to 2020 or take 2020 into context or [issue a note] excluding 2020.”

Whether we all do the same, when we read about or remember the pandemic through the numbers that serve as our collective comparator for where we stand, remains to be seen.

Email: [email protected]

Twitter: @isaaccallan

Tel: 647 561-4879

COVID-19 is impacting all Canadians. At a time when vital public information is needed by everyone, The Pointer has taken down our paywall on all stories relating to the pandemic and those of public interest to ensure every resident of Brampton and Mississauga has access to the facts. For those who are able, we encourage you to consider a subscription. This will help us report on important public interest issues the community needs to know about now more than ever. You can register for a 30-day free trial HERE. Thereafter, The Pointer will charge $10 a month and you can cancel any time right on the website. Thank you