Going Green with Envy

Since the Green Paper was released last week, I've found that it required so much careful reading that getting a full understanding of the Teaching Excellence Framework proposals it contains is taking up way too much of my time due to, and this is me being kind, the document being as poorly thought out as some of the content.

Fortunately the considerate types at WonkHE have far more patience than I do (hence the post title) and have put together a visual interpretation of the TEF with inputs, processing, and outputs, and associating each set of components with different rules and conditions, most which I've put here for my future reference, but if you want to read the whole thing it's at http://wonkhe.com/blogs/the-incredible-machine-our-visual-guide-to-the-tef

Reproduced from WonkHE

KEY (with references to the Green Paper in parentheses)

The inputs:

a. In 2016-17, a satisfactory quality assurance review from QAA, ISI for course designation or equivalent, in place by February 2016, will lead directly to a TEF Level 1 award (Chapter 1, Paras. 26,27). Candidacy for these awards will not go to the Independent Panel, as far as we can tell.

b. Applications for higher TEF awards will be subject to three ‘pre-conditions’ that will be assessed by the Independent Panel: •The provider will need an Access Agreement or similar device (C1, P19) •The provider will need to show it is compliant with ‘market practice’ guidelines set out by the Competition and Markets Authority (C2, P3) •The provider will need to state whether or not they use a Grade Point Average assessment system (C1, P40); but note the requirement is only to state their position, and actual use of such a system is not to be a ‘prerequisite’ for higher TEF awards

c. Applications for higher TEF awards will be informed by ‘common metrics’ initially drawn from a set of three (C3, P12) to include measures of employment and earnings (starting with DLHE but going on to use data from the HMRC data match), retention and continuation (from HESA’s performance indicators), and student satisfaction (derived from the NSS). These metrics will change over time; in particular, note the NSS is itself under review and several suggestions for others have been put forward (C3, P14). As metrics will presumably change every year, and providers will be on different assessment cycles (no ‘gathered field’ as in the REF; C2, P6) then they will routinely be judged using differently constructed common metrics depending on when they are assessed or re-assessed – they are therefore not really ‘common’ metrics at the point of use.

d. Providers will be able to supplement these metrics with additional evidence, both quantitative and qualitative (C3, Ps. 13, 17), of various types.

e. Reporting of metrics will be dis-aggregated by student background (C3, P4) to show performance in the context of student profile differences.

The processor:
f. Applications for higher TEF levels will be conducted by an Independent Panel, comprised of ‘academic experts in learning and teaching, student representatives, and employer/professional representatives’ (C2, P9); note, there is no proposal to include provider representatives on this panel. The assessment framework will include •teaching quality; •learning environment; •student outcomes and learning gain;

…and various sub-factors are also sketched (C3, Ps. 5,7,8,9). Re-assessments are envisaged to take place on a 3-5 cycle, with trigger events for sooner re-assessment (C2, P5).

The outputs:
g. TEF award level one might best be described as ‘baseline quality assured’, as it is dependent only on the QA review input module. The Independent Panel will make higher TEF awards at either two or three additional levels. Levels 2 and 3 are not further defined, but are indicated to be ‘differentiation levels’ (C2, P15). Level 4 is further defined as ‘requiring performance significantly above expectations’ and/or ‘compelling evidence of excellence’ (C2, P15). This implies that providers can win the ultimate TEF prize by being “better than they really ought to be – if you know what we mean”, and that any provider who doesn’t get to Level 4 may be deemed “excellent alright, but not quite compellingly so – if you know what we mean”. The problem is that we don’t know what they mean.

h. Presumably, it will also be possible to fail a panel assessment and get pushed back out with a Level 1 award, though this isn’t explicitly stated.

i. At some point it is envisaged that these award levels may be given differentially for different subject areas within all providers and that these would then be aggregated to form an award for the provider as a whole (C1, P23); multiple independent panels would then be formed, presumably feeding into a ‘lead panel’ of some kind – needless to say, we haven’t even tried to put any of this in the diagram.

Food for thought

Sat on the train home from today's HN Examiner meeting, I've just read in the free paper handed to me at King's Cross this little gem from a recently published collection of the Queen Mother's letters about some South Sea Island:

"The natives are very diseased and are rapidly dying out.  Instead of being strong, healthy cannibals with strange religions and no clothes, they are now half-hearted Roman Catholics, with European clothes.  It seems all wrong but that is what happens."

David Icke probably takes this as confirmation that the QM was indeed a blood-drinking lizard keen on human flesh, but is it an example of anti-Catholic prejudice or an indication that the royal family have a progressive appreciation of cultural diversity just as long as the natives aren't eating anybody you might know?

Alphabet Soup, Acronym Crackers

Once upon a time, when I was way too young to know anything about education policies, principles & pass rates, every school child took either 'O' levels if you had some sort of demonstrable academic ability, or CSEs if you didn't (or possibly a mixture, but that's digressing..). The system approximately expressed the division between grammar schools and secondary moderns, but tended to make less sense as more schools went comprehensives, so the introduction of GCSEs in the late 1980s was long overdue.

However, this soon translated to GCSEs effectively becoming CSEs for everyone to the extent that the proportion of students obtaining top grades is now three times what it was when they were introduced (and looking through old papers from my sister's year to my own 10 years previously shows how dumbed down they were to begin with).

This means that Michael Gove's desire to ditch GCSEs in favour of a restored 'O' level-type qualification is somewhat idealistic, but its also unlikely that the latest scheme would survive the inevitable avalanche of criticism from the educational establishment even if his coalition partners weren't dead set against it.

It raises the conundrum that any exam possessing sufficient rigour to be worthwhile, will by definition be too difficult for a minority of pupils to pass, regardless of any improvements that are made in teaching, and so requires some sort of different provision for those less able.

So if we take the position that GCSEs are now discredited with their grades inflated into meaninglessness, is the best solution to replace them with a new examination system that would, in a few years' time, fall victim to exactly the same process of grade inflation and declining confidence?

Far better then to just get rid of them, particularly as post-16 qualifications make no sense when the normal school leaving age will be 18 from 2015 and it is surely logical for the main assessment of academic achievement to coincide with the final year of school? Taking GCSEs out of the picture would then leave a convenient opening for AS levels to be introduced to slightly younger students with minimum disruption and cost as teachers are familiar with them and they integrate with 'A'-levels in a way that GCSEs never have.

Money Talks

I've just arrived in Crawley to deliver a day full of assessment workshops and am happy to discover that not only is there apparently no recession here, the local authority has such a cash surplus that it must surely be reducing the business rates and community charges for the coming year. Not convinced? Well the local paper is carrying the headline that West Sussex county council has spent over £100,000 making information videos on such important subjects as "how to wash your hands", and "how to use a mobile phone". Therefore, simple logic suggests a surfeit of cash, as they surely can't think that this kind of thing is in anyway necessary. Can they?
For example, any user of the national rail service could have told them that 90% of the population already know how to "get on train/switch on phone/call someone/use loud voice to tell that someone you're on a train .....".

Disreputable Behaviour

I'm currently watching Sky News in a London hotel prior to tomorrow's assessment conference and wondering how exactly Fred Goodwin has "brought the honours system into disrepute"? He didn't award himself the knighthood that the queen has just withdrawn; all he did was accept it. You could probably argue that he brought himself into disrepute by graceless behaviour and reckless management of the Royal Bank of Scotland. And you could also make a case that he brought the banking industry into disrepute (admittedly in conjunction with many other supposedly leading lights). But the honours system? That charge really ought to be levelled at those who recommended and confirmed the award for "services to banking" for someone who was effectively just gambling with someone else's money.

Now, it's not that I'm particularly advocating that (the now) Mr Goodwin deserves to keep his knighthood, but unlike previously dispossessed knights, he is not a criminal or a traitor, just someone who didn't turn out to have been quite so brilliant a banker as they thought him to be when bestowing the award in the first place. So it occurs to me that if you consider the many still-knighted leading bankers who must be grateful to Fred Goodwin for effectively taking the public flak on their behalf, you might conceivably say that he's performed his fellow money men a great service. And doesn't great service eventually lead to honours? A knighthood perhaps?....

Mass homeopathic overdose - none dead!

Setting off to Derby on a 10.23 train to deliver an assessment workshop has prompted me to declare my admiration for all those who took part in Saturday's mass homeopathic overdose (at 10.23am: various locations around the country). Or not as it happens, since nobody succeeded in taking their own lives despite hundreds of people downing entire packets of homeopathic pills.

Hopefully the 10:23 campaign has succeeded in raising awareness (a form of education, just by another means?) of the inefficacy of homeopathic medicine, and the double-standards of leading pharmacists who happily stock them while admitting that they don't work.

Enhancing the Student Experience conference

After the "star billing" of Monday, today was the rather more usual relegation to "support act" on the "Enhancing the Student Experience" conference bill at Nottingham's Basford Hall. First up was HEFCE who told us that times are hard, there's no money, and their continued role in UK education is not guaranteed etc.
The delegates then chose from many workshops: by me (3 times: "Involving students in Assessment"), Wendy Stubbs from QAA, JISC RSC, Confetti Institute, Nottingham Law School, Trent Uni, Castle College, Leicester College etc and also heard from the NUS about what students want (subject knowedge that will get them a job, social experiences so they're not alone, feedback so they can improve, etc : no surprises, although one did say that their degree certificate would make a good table mat if it was laminated, but while unlaminated served no other purpose! Times are hard apparently.....).