Going Green with Envy

Since the Green Paper was released last week, I've found that it required so much careful reading that getting a full understanding of the Teaching Excellence Framework proposals it contains is taking up way too much of my time due to, and this is me being kind, the document being as poorly thought out as some of the content.

Fortunately the considerate types at WonkHE have far more patience than I do (hence the post title) and have put together a visual interpretation of the TEF with inputs, processing, and outputs, and associating each set of components with different rules and conditions, most which I've put here for my future reference, but if you want to read the whole thing it's at http://wonkhe.com/blogs/the-incredible-machine-our-visual-guide-to-the-tef

Reproduced from WonkHE

KEY (with references to the Green Paper in parentheses)

The inputs:

a. In 2016-17, a satisfactory quality assurance review from QAA, ISI for course designation or equivalent, in place by February 2016, will lead directly to a TEF Level 1 award (Chapter 1, Paras. 26,27). Candidacy for these awards will not go to the Independent Panel, as far as we can tell.

b. Applications for higher TEF awards will be subject to three ‘pre-conditions’ that will be assessed by the Independent Panel: •The provider will need an Access Agreement or similar device (C1, P19) •The provider will need to show it is compliant with ‘market practice’ guidelines set out by the Competition and Markets Authority (C2, P3) •The provider will need to state whether or not they use a Grade Point Average assessment system (C1, P40); but note the requirement is only to state their position, and actual use of such a system is not to be a ‘prerequisite’ for higher TEF awards

c. Applications for higher TEF awards will be informed by ‘common metrics’ initially drawn from a set of three (C3, P12) to include measures of employment and earnings (starting with DLHE but going on to use data from the HMRC data match), retention and continuation (from HESA’s performance indicators), and student satisfaction (derived from the NSS). These metrics will change over time; in particular, note the NSS is itself under review and several suggestions for others have been put forward (C3, P14). As metrics will presumably change every year, and providers will be on different assessment cycles (no ‘gathered field’ as in the REF; C2, P6) then they will routinely be judged using differently constructed common metrics depending on when they are assessed or re-assessed – they are therefore not really ‘common’ metrics at the point of use.

d. Providers will be able to supplement these metrics with additional evidence, both quantitative and qualitative (C3, Ps. 13, 17), of various types.

e. Reporting of metrics will be dis-aggregated by student background (C3, P4) to show performance in the context of student profile differences.

The processor:
f. Applications for higher TEF levels will be conducted by an Independent Panel, comprised of ‘academic experts in learning and teaching, student representatives, and employer/professional representatives’ (C2, P9); note, there is no proposal to include provider representatives on this panel. The assessment framework will include •teaching quality; •learning environment; •student outcomes and learning gain;

…and various sub-factors are also sketched (C3, Ps. 5,7,8,9). Re-assessments are envisaged to take place on a 3-5 cycle, with trigger events for sooner re-assessment (C2, P5).

The outputs:
g. TEF award level one might best be described as ‘baseline quality assured’, as it is dependent only on the QA review input module. The Independent Panel will make higher TEF awards at either two or three additional levels. Levels 2 and 3 are not further defined, but are indicated to be ‘differentiation levels’ (C2, P15). Level 4 is further defined as ‘requiring performance significantly above expectations’ and/or ‘compelling evidence of excellence’ (C2, P15). This implies that providers can win the ultimate TEF prize by being “better than they really ought to be – if you know what we mean”, and that any provider who doesn’t get to Level 4 may be deemed “excellent alright, but not quite compellingly so – if you know what we mean”. The problem is that we don’t know what they mean.

h. Presumably, it will also be possible to fail a panel assessment and get pushed back out with a Level 1 award, though this isn’t explicitly stated.

i. At some point it is envisaged that these award levels may be given differentially for different subject areas within all providers and that these would then be aggregated to form an award for the provider as a whole (C1, P23); multiple independent panels would then be formed, presumably feeding into a ‘lead panel’ of some kind – needless to say, we haven’t even tried to put any of this in the diagram.