Mooconomics 101 exam


Venue: anywhere with wifi that lets you use Google

Date: whenever you feel like it

Instructions to candidates:  You may not use any external information, unless you can get away with it, nor should you use your free hand to shoo the cat, pick cornflakes from your navel, or scratch unhygienically. Answer any questions which appeal to you. Pictures of kittens will only be considered as a tiebreaker …

If MOOCs are to be of any use in educating students to degree standard, something has to give. Around this time of year, many academics are invigilating and marking exams, or working through the pile of student work handed in at the end of the semester. The exams were set three or four months ago, so that they could be sent to external examiners, with sample solutions, to maintain a consistent and proper standard across similar degrees in different institutions. After marking, the standard is checked by a second examiner, the entry of marks into the IT system is checked, and any units whose averages lie outside the expected range are given extra scrutiny. The outcome is a set of numbers which can be printed on transcripts and used to calculate degree classifications. All of this marking, and resulting discussion with students, takes time, hours per student over a semester, and, in consequence, money, about £25 per hour of an academic’s time.

The assigning of a mark to a piece of work, and to a student’s performance on an academic unit, has two main purposes. The first is to check that the student has reached the minimum standard required for them to be considered properly educated in that part of the discipline (the pass mark); the second is to order students by performance and show where they lie in the distribution of abilities in that part of the discipline. The final number which emerges, whether GPA or degree classification, is taken as a more or less reliable guide to a student’s standard and is then used by employers and graduate schools as a first criterion on which graduates may be accepted for entry.

If MOOCs are not just In Our Time for snobs, or TED talks with grades, the assessment must be credible. Daphne Koller of Coursera suggest that “students can grade each other’s papers even on issues of critical reasoning and grammar”, also known as the magic rubric. If you have enough contempt for the humanities to think this is acceptable there, feel free to have your children’s history and English teachers graded by their fellow students. On the other hand, do you want to be treated by a doctor whose knowledge of the circulatory system was tested by a cast of unknown thousands, or defended in court by a barrister whose command of language was assessed by internet users working from a crib sheet?

At present, the tangible outcome of a degree programme is a degree certificate, accompanied by a transcript giving a breakdown of marks in various elements of the programme. Like a banknote, this certificate takes its formal authority from the institution which issued it, backed by the requirements which the institution had to meet in order to offer an `accredited’ degree. Accreditation is the process by which professional bodies establish a framework for degree courses in particular areas, and check that the right things are being taught to the right standard. In engineering terms, there is a traceable calibration from the accrediting body to the degree, with quality control at each stage. If the accrediting bodies continue to establish standards for entry to the professions-somebody has to-the apostolic succession must be maintained, by transmission through identifiable, accountable, individuals. Should student grading become the standard, we risk having biology exams being marked by creationists, without our even knowing. Frightening though the prospect of students being failed for mentioning evolution is, the idea of creationists being passed is even worse.

For degrees to have credibility, then, they must be marked by academics of some kind: people with the knowledge, experience, and authority (`authoritative’ rather than `authoritarian’) to form a proper judgement on a piece of work, and to justify that judgement if required to. Without the capacity to form a proper judgement, an examiner will not be able to assess the student who goes beyond the basic reading for a course, because they will not able to mark outside the rubric, and the best students will not be properly recognized (in both senses). This standard of marking is expensive, and cannot be made cheap.

If you want to reduce the cost of a labour intensive process, there are two obvious methods: sweat the workers you have, or hire cheaper ones. Clearly, if the lecturing is no longer being done in-house, because the university has bought a course in, and the exams are set by ExamCorp, academics will have plenty of time for assessment. We could even sell it as the value-added element of the course, which sets it apart from the inferior offering at some lowlier institution which uses the same MOOCs but doesn’t give you a free iPad with the university logo. Even academics, however, will eventually resist the assembly line: year-round, nine-to-five, marking of exams is not a credible strategy.

There is a reserve army of intellectual labour, however, ready to be paid in hope, if they think they might make partner at ExamCorp, or in money if they live in a country where the alternative is working in a call centre. Like call-centre workers, though, neither group will have much incentive to care about their work, beyond doing it sufficiently well to keep their jobs.

The marginal cost of transmitting a MOOC is zero, so the price charged to the consumer will fall rapidly. Once that happens, cost must be reduced everywhere and the next big cost is proper, competent assessment. On past experience, we can expect the creation of a new lumpenintelligentsia.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s