Panos Ipeirotis

I need to go through this material and they don't get distracted.

They get back on topic and so on.

So if it worked for 70,000 job interviews, I thought that they would work for 35 oral exams in my classroom.

Actually, we had been studying AI grading for quite a while.

I think that finally we're reaching a point where AI is really good in not only grading, but actually giving very detailed feedback.

To tell you the truth, I have graded every single exam myself before giving it to AI to make sure that everything works.

And

To make sure that whatever they are submitting is actually correct and makes sense.

Yes, I agree.

So, yes, we also used a council of LLMs.

We gave the same transcript to three different LLMs to grade.

Then they consulted each other.

And they came up with a deliberation and then the final grade.

And I compared all my grades against the council and I realized that the council was actually stricter than me, but very consistent and had a lot of evidence for their grades and very good feedback for the students.

They found it a little bit more stressful than the regular exam.

And, yeah.

They didn't like the voice.

We borrowed the voice of my colleague, Foster Provo.

He has a very distinct and very imposing voice.

They found it very frightening and very stressful during the exam.