How to be a bastard: crowd control in the academy

When I had my first academic job, new staff spent a couple of days of induction learning about how to teach. On the afternoon of the last day, which was the Friday before teaching started, the course tutor asked if there was anything he hadn’t covered which we would like to learn about.

A colleague said he had 150 first year engineering students on Monday. I laughed. The course tutor laughed. Nobody else laughed. The tutor explained to the other academics, mainly from the humanities, that engineering students, at least in Ireland, are a notorious shower of animals, liable to throw things at the lecturer, including, on one occasion, a frozen chicken.

“So you want to know how to be a bastard?”
Read the rest of this entry »

What’s wrong with numerical feedback?

In short: the numbers are wrong, and the statistics are worse.

Like other academics in the UK and elsewhere, I am judged as a teacher on the basis of feedback from students taking my courses. In some institutions, not doing well enough on this feedback can lead to dismissal. The problem is that this feedback is largely meaningless.

In my university, as in others, the feedback takes the form of comments (valuable and useful for thinking about teaching practice) and a numerical score between 1 and 5 under a number of headings. These scores are then averaged and, in my department, any score under 3.5 is reason to fill in a form explaining what action will be taken to make sure it does not happen next year.

The first problem with the numerical feedback is that students are not good judges of teaching. Insofar as there is evidence from proper trials, it seems that the numerical scores awarded by students do not reflect how well they have learned from their teachers. In other words, the numbers going in are unreliable, especially since with low return rates the results are dominated by students who are disgruntled or very gruntled.

Secondly, the final score is unreliable. As you will know from following opinion polls before elections, when you take a small sample of a group, there is an inevitable error in the resulting estimate of the average. This is especially true when the sample is biased towards the extremes. In my university, students give scores between 1 and 5: averages are presented to three significant figures 1.00 to 5.00.

To see what is wrong with this, think of the distinction between precision and accuracy, something every first engineering student must learn: precision is the number of decimal places, accuracy is the number of decimal places you can believe.

A typical class size might be 40 students. On a 25% submission rate (typical), ten students put numbers in to be averaged. Doing the sums, if one student changes a mark by one, say from 3 to 4, the average changes by 0.1. The academic is assessed on the basis of a difference of 0.01: 3.5 good, 3.49 is a problem. In other words, decisions are made by believing the noise in the signal. A single anonymous student, cheesed off because he has been set an exam question he has never seen before, can ruin a career.

We have numbers which are probably wrong to start with, in biased samples too small to be statistically valid, forced through an averaging process to give a spurious precision, and a management prepared to use these numbers as an `objective’ measure of teaching `quality’.

Paying for education

As seems likely, we in the UK are about to see the price of education hiked by another few grand a year. It turns out that the problem of what to pay for education was solved by William Blake a couple of centuries ago:

What is the price of Experience? Do men buy it for a song?
Or wisdom for a dance in the street? No, it is bought with the price
Of all that a man hath, his house, his wife, his children
Wisdom is sold in the desolate market where none come to buy
And in the wither’d field where the farmer ploughs for bread in vain

or, if you prefer, fame costs:

Theodor Adorno and Michael Gove

So there I was, minding my own business, when I discovered that the Frankfurt School was responsible for giving gay men and women in Ireland the right to marry.

“Well now,” says I, “what else might they be responsible for, these gnomes of Frankfurt.” It turns out they have been the source of the “ideas” for education policy in the United Kingdom for decades. From Dialectic of Enlightenment:

Cultural education spread with bourgeouis property. It forced paranoia into the dark corners of society and the soul. But since the real emancipation of mankind did not take place with the enlightenment of the mind, education itself became diseased. The greater the distance between the educated consciousness and social reality, the more it was itself exposed to the process of reification. Culture became wholly a commodity disseminated as information without permeating the individuals who acquired it. Thought became restricted to the acquisition of isolated facts. Conceptual relationships were rejected as uncomfortable and useless effort. The aspect of development in thought, all that is genetic and intensive in it, is forgotten and leveled down to the immediately given, to the extensive. Today the order of life allows no time for the ego to draw spiritual or intellectual conclusions. The thought which leads to knowledge is neutralized and used as a mere qualification on specific labour markets and to heighten the commodity value of the personality. And so that self-examination of the mind which works against paranoia is defeated. Finally, under the conditions of modern capitalism, half-education has become objective spirit. In the totalitarian phase of domination, it calls upon the provincial charlatans of politics, and with them the system of delusion as the ultima ratio: forcing it upon the majority of the ruled, who are already deadened by the culture industry. The contradictions of rule can be seen through by the healthy consciousness so easily today that it takes a diseased mind to keep them alive. Only those who suffer from a delusion of persecution accept the persecution to which domination must necessarily lead, inasmuch as they are allowed to persecute others.