This blog is a (much!) less-than-formal outlining of recent travels, events, happenings, thoughts and comments which tend to have some occupational relevance, but are on occasion nothing more than a means of passing the time while waiting for trains, planes & automobiles...
Showing posts with label metrics. Show all posts
Showing posts with label metrics. Show all posts

Thursday, 13 August 2020

Cherry Picked Information

Only 56 Million Cases to go until we reach H1N1 (Swine Flu) Cases from 2019!
Remind me again how long were schools shut down for?
What type of masks were popular back then?

Not having anything like as much data analysis to do as would be the case were we not in a pandemic, I've investigated the above meme which has appeared in my Facebook scroller due to posting by a Friend and getting liked/commented by enough other people to make it the first thing I saw.

The first thing that occurred to me was "is this data accurate"? I am not sure which data sources, location, or how recent it is, but as the poster has an intertest in US politics, it's probably about the US even though there are no citations to back up the statement.

However, it is consistent with the US CDC’s data. Over a period of one year (from April 12, 2009 to April 10, 2010), the CDC estimated there were 60.8 million cases of H1N1 in the United States [1]. Using the same source, the current cases of Covid-19 in the US are 4,920,369 (about 4.9 million) [2]. So as of today, we need roughly 55.9 million more Covid-19 cases (almost 56 million) to catch up on the 2009 H1N1 cases. So, yes, the data in the meme seems accurate.

Now, what about the argument? Although the conclusion is implied, the argument appears to be:
Premise 1: There were more 2009 H1N1 cases than the current pandemic
Premise 2: But we wear masks and close schools for this pandemic, but not for the 2009 pandemic.

Conclusion: we (or the US if my initial assumption is correct) are overreacting and don’t need to wear masks and close schools for this pandemic.

So on first reading, and without any sort or critical thought, it seems to make a sensible point, but a closer look shows it to be spurious. The argument rests on the hidden assumption that the case numbers are the primary determining factor for deciding if masks should be worn and schools should be closed.

But that doesn't make much sense as the average adult probably gets the common cold a few times a year, and given the US population, the cases of such a mild illness would be several hundred million a year. Clearly, other factors must be considered when deciding how to mitigate disease, whether that mitigation is with masks, closing schools, social distancing, or whatever else.

There are a number of important metrics in epidemiology such as incident rates, morbidity rates, mortality rates, hospitalisation rates, and many others, so only focusing on the cases is ridiculous, especially when these two diseases, H1N1 and Covid-19 are very different.

Going over each metric would be take way too long when there's cricket on the television, but we can clearly see that these two diseases are different by looking at the deaths in conjunction with the cases. The US coronavirus deaths to date attributed to COVID-19 is about 160,220 since January 2020, with an estimated 4.9 million cases.

However, H1N1 had substantially fewer (12,469 deaths from April 2009 to April 2010), from a whopping 60.8 million cases. But there is something else to think about. The death data for H1N1 is for one year, whereas the COVID-19 numbers are for only 8 months—and it may not slow down for several more months. Moreover, the H1N1 pandemic had relatively few precautions to attenuate its spread. Few people wore masks, there was little social distancing, etc.

So what does this mean? This means that over a period of 8 months, with social distancing, Covid-19 has more than 12 times the deaths (with social distancing and masks) from less than 1/12 the number of cases H1N1 had over one year. Though the 2009 pandemic was concerning, it was clearly nowhere near as deadly as this current pandemic. Therefore, at least concerning how society should react to the two diseases, it doesn’t make sense to compare these two diseases as equal.

So therefore this meme, which probably took seconds to compose, and maybe a couple of hours to pull apart with cited sources, makes a fallacious argument by using cherry-picked data from two very different diseases, and hyper focuses on case numbers as the sole metric for these decisions.

Who would have that sort thing would be all over T'Internet eh? :)


1. CDC website. 2009 H1N1 Pandemic (H1N1pdm09 virus). Last reviewed: June 11, 2019. Accessed Aug 12, 2020.

2. CDC website. Coronavirus Disease 2019 (COVID19). Cases, Data, and Surveillance. Last updated. Aug. 8, 2020, 07:00 PM. Accessed Aug 12, 2020.


Thursday, 12 March 2020

Sanity and Sanitisers

A friend of mine has asked if I'll send him some hand sanitiser as all the shops where he lives have run out, and there is still some on the shelves in my local ASDA. At first, I didn't think he was serious as he's way too intelligent to get caught up in the Coronavirus panic, but apparently not: he genuinely wants me to post him as many bottles as I can get.

It may be that his family are pressuring him, but mine are far more grounded, as are all my friends living locally. Maybe it's a Yorkshire thing? Maybe we'd rather risk death than pay inflated prices for scented squirty soap? Or maybe as my daughter put it, Yorkshiremen would rather die than suffer the embarrassment of being thought panic shoppers?

I accept that Coronavirus (Covid-19) is a pretty virulent virus, but not in the contagious-imminent-danger-to-everybody's-health way.

As I type this there are 125,743 confirmed cases worldwide, almost 81,00 of which were or are in China, and 3,169 deaths in China plus another 1441 elsewhere in the world. So 4,610 deaths in total and half the planet is in lockdown.

Why? What has driven this hysteria? Two years ago, all the way back in 2017-18 the Office for National Statistics recorded 50,100 ‘excess winter deaths’ but it was business as usual. The explanation (for the deaths, not the lack of hysterical reaction), according to the ONS, was probably ‘the predominant strain of flu, the effectiveness of the influenza vaccine, and below average winter temperatures’. Across the pond where Donald Trump has just announced a flight ban from mainland Europe and the golf I'm watching on TV is discussing the ban on spectators from tomorrow onwards, according to the US-based Centre for Disease Control and Prevention, over 80,000 died.

In both countries most of the victims were geriatric, many with compromised immune systems, as is the case with the UK Coronavirus deaths - or as I prefer to refer to them as a statistician, deaths of people with Coronavirus (not necessarily from, due to their age, health and underlying conditions)

And seasonal flu? According to an estimate by the CDCP, it causes somewhere between 291,000 and 646,000 deaths globally a year. To put it another way, if the number of deaths from coronavirus rises a hundredfold in the next few weeks or months, it will only have reached the lower bound of the estimate for existing strains of flu. How many of us wear face masks because of winter flu? How many planes and trains are cancelled? Does the stock market slump?

There is some justification for being more wary of Covid-19 than the flu as the former is an unknown quantity which we've not evolved with and don’t yet have a vaccine. But we know more about it by the day, its death rate is under 3 per cent and it is mostly killing people with pre-existing health conditions.

And probably more importantly, if it can, or does, spread quickly and easily, why don't we all have it already? In the past week, I've been on 8 trains (all crowded) and four buses (two full), attended a conference, a rock concert and a football match (1000, 2000 and 30000 people respectively) and been to the gym where nobody has cleaned down anything, sanitised between exercises, or avoided personal contact four times. And I'm ok. As is everybody I know and know of, but if the virus is virulent then surely this wouldn't be the case?

So where is the evidence that closing large gatherings of people makes anyone safer or slows the spread of disease? Is it logical to assume that Covid-19 can be passed on easier at a football match or the now-cancelled Australian Grand Prix than in a gym?

Maybe I'm missing something about Coronavirus and the attitude of people like me will only compound the problem, but it feels like this is just the latest "end of the world" phenomenon to trouble the populations of developed countries along with other apocalyptic portents such as climate alarmism, nuclear Armageddon and financial collapse.

At the end of January, Brexit had just been completed (sort of anyway) without incident, the standoff between the US and Iran had fizzled into nothing, the Australian bush fires had largely gone out, so did the media need something else to worry us about?

Reactionary hysteria has taken hold all over the world. Saudi Arabia has suspended religious pilgrimage trips to Mecca (oh the irony that the praying won't make the most devout any safer!), the Italian Prime Minister has ordered the lockdown of the country’s northern region, Ireland has closed schools and colleges, and Scotland's First Minister Nicola Sturgeon said today it was 'inappropriate that we continue as normal' and will recommend the cancellation of gatherings of more than 500 people to protect front-line services from Monday. She didn't supply any information as to why Monday and not tomorrow if the measure is so necessary, or why allowing gatherings at the weekend was appropriate, but common sense does seem to be in short supply.

As the great Homer Simpson once said "you can prove anything with facts" so why haven't the governments of the world provided some which justifies their extreme actions?

Or is that there aren't any and this hysteria is the result of a media frenzy which has caused the "leaders" of some countries to be scared of not being seen to "do something" whether or not it makes sense?

Wednesday, 11 March 2020

Political Mathematics


Many years of interacting with MPs, advisors and policy makers has made me very sceptical when any of them start to use numbers to illustrate a point: the latest of which is the London Mayoral hopeful Rory Stewart who claims in this week's Sunday Times to have slept in 50 different homes around London as part of his ‘Come Kip With Me’ scheme.

However, as he apparently only launched his campaign in the second week of February, a whole 27 days before Sunday, this means that Mr Stewart would have to have slept in almost two homes a night every night, never spending an evening at home, for his claim to be true.

He also says that “you learn an amazing amount because you’re spending 14 hours with somebody”, which means he wants us to believe that he's spent 700 hours in other people’s homes over the last 27 days.

Which is an admirable amount of effort when there are only 648 hours in 27 days ….

Monday, 9 November 2015

Going Green with Envy

Since the Green Paper was released last week, I've found that it required so much careful reading that getting a full understanding of the Teaching Excellence Framework proposals it contains is taking up way too much of my time due to, and this is me being kind, the document being as poorly thought out as some of the content.

Fortunately the considerate types at WonkHE have far more patience than I do (hence the post title) and have put together a visual interpretation of the TEF with inputs, processing, and outputs, and associating each set of components with different rules and conditions, most which I've put here for my future reference, but if you want to read the whole thing it's at http://wonkhe.com/blogs/the-incredible-machine-our-visual-guide-to-the-tef

Reproduced from WonkHE

KEY (with references to the Green Paper in parentheses)

The inputs:

a. In 2016-17, a satisfactory quality assurance review from QAA, ISI for course designation or equivalent, in place by February 2016, will lead directly to a TEF Level 1 award (Chapter 1, Paras. 26,27). Candidacy for these awards will not go to the Independent Panel, as far as we can tell.

b. Applications for higher TEF awards will be subject to three ‘pre-conditions’ that will be assessed by the Independent Panel: •The provider will need an Access Agreement or similar device (C1, P19) •The provider will need to show it is compliant with ‘market practice’ guidelines set out by the Competition and Markets Authority (C2, P3) •The provider will need to state whether or not they use a Grade Point Average assessment system (C1, P40); but note the requirement is only to state their position, and actual use of such a system is not to be a ‘prerequisite’ for higher TEF awards

c. Applications for higher TEF awards will be informed by ‘common metrics’ initially drawn from a set of three (C3, P12) to include measures of employment and earnings (starting with DLHE but going on to use data from the HMRC data match), retention and continuation (from HESA’s performance indicators), and student satisfaction (derived from the NSS). These metrics will change over time; in particular, note the NSS is itself under review and several suggestions for others have been put forward (C3, P14). As metrics will presumably change every year, and providers will be on different assessment cycles (no ‘gathered field’ as in the REF; C2, P6) then they will routinely be judged using differently constructed common metrics depending on when they are assessed or re-assessed – they are therefore not really ‘common’ metrics at the point of use.

d. Providers will be able to supplement these metrics with additional evidence, both quantitative and qualitative (C3, Ps. 13, 17), of various types.

e. Reporting of metrics will be dis-aggregated by student background (C3, P4) to show performance in the context of student profile differences.

The processor:
f. Applications for higher TEF levels will be conducted by an Independent Panel, comprised of ‘academic experts in learning and teaching, student representatives, and employer/professional representatives’ (C2, P9); note, there is no proposal to include provider representatives on this panel. The assessment framework will include •teaching quality; •learning environment; •student outcomes and learning gain;

…and various sub-factors are also sketched (C3, Ps. 5,7,8,9). Re-assessments are envisaged to take place on a 3-5 cycle, with trigger events for sooner re-assessment (C2, P5).

The outputs:
g. TEF award level one might best be described as ‘baseline quality assured’, as it is dependent only on the QA review input module. The Independent Panel will make higher TEF awards at either two or three additional levels. Levels 2 and 3 are not further defined, but are indicated to be ‘differentiation levels’ (C2, P15). Level 4 is further defined as ‘requiring performance significantly above expectations’ and/or ‘compelling evidence of excellence’ (C2, P15). This implies that providers can win the ultimate TEF prize by being “better than they really ought to be – if you know what we mean”, and that any provider who doesn’t get to Level 4 may be deemed “excellent alright, but not quite compellingly so – if you know what we mean”. The problem is that we don’t know what they mean.

h. Presumably, it will also be possible to fail a panel assessment and get pushed back out with a Level 1 award, though this isn’t explicitly stated.

i. At some point it is envisaged that these award levels may be given differentially for different subject areas within all providers and that these would then be aggregated to form an award for the provider as a whole (C1, P23); multiple independent panels would then be formed, presumably feeding into a ‘lead panel’ of some kind – needless to say, we haven’t even tried to put any of this in the diagram.