120 Questions With Correct Answers

Mexico's most prestigious university has cancelled around 3,000 entrance exams. They suspect a mix of AI and leaked questions. Everyone's arguing about the detection. I'd look at the exam.

UNAM's entrance test is 120 multiple-choice questions in three hours, one point each. There's no pass mark either. Places go to the highest scorers until they run out, so it's a ranking machine, and every inflated score pushes a real candidate down the list.

This year UNAM put it online for the first time. 158,000 people sat it in May and June. The scores came back looking wrong.

Students flagged by the AI proctoring system are now disputing it. So the detection is under suspicion too.

Some places have gone the other way and redesigned the assessment rather than the detection.

The University of Sydney have quite an interesting approach. They split assessment into two lanes:

• Lane one is secured and supervised. It exists to form a trustworthy judgement about what someone can do.
• Lane two is open, and AI is encouraged. It exists to help them learn.

One assessment can't do both jobs, which is why detection became the battleground everywhere else.

UNAM's entrance exam is lane one. They put it online.

What replaces it at that scale, I don't know. Oral exams and portfolios are lovely, and you can't run them for 158,000 people in six weeks.

My instinct is that the honest answer here is a boring one. At this scale you can't design an exam that AI won't solve. You can only put people in a room and watch them do the test.

Which is deflating, when designing the alternative is what I do for a living.

If you're teaching or examining right now, I'd like to know how you're thinking about this stuff?