Panos Ipeirotis

I think that finally we're reaching a point where AI is really good in not only grading, but actually giving very detailed feedback.

To tell you the truth, I have graded every single exam myself before giving it to AI to make sure that everything works.

And

To make sure that whatever they are submitting is actually correct and makes sense.

Yes, I agree.

So, yes, we also used a council of LLMs.

We gave the same transcript to three different LLMs to grade.

Then they consulted each other.

And they came up with a deliberation and then a final grade.

And I compared all my grades against the council and I realized that the council was actually stricter than me, but very consistent and had a lot of evidence for their grades and very good feedback for the students.

They found it a little bit more stressful than the regular exam.

And, yeah.

They didn't like the voice.

We borrowed the voice of my colleague, Foster Provo, who has a very distinct and very imposing voice.

They found it very frightening and very stressful during the exam.

They found it very entertaining, but on a real note, probably will change it next time.

And they were

But on the other hand, the scooters, they found the experience more stressful, but they also agreed that it tested the real understanding of the material.

So we didn't get any complaints about fairness or accuracy of the grading and so on.

I hope so.