Atrioc

So in both angles, we're like suffering.

And then there's just

going to be this massive slop that is being pushed out as everybody becomes dumber, but is better at producing slop.

We're getting really efficient at being dumb slop generators.

prime prime said this he reviewed the two most recent models and he's like the people who weren't producing good code are just producing bad code faster but at a gargantuan rate yeah and then one final thing i'll say there's this excellent youtube video called can ai pass freshman cs so this is graduate ta at cornell i guess we can put links to these youtube videos in the description but graduate ta at cornell um last semester did a study where he basically had

the three of the best coding models, Gemini, Claude, and ChachiBT, just follow his computer science class that he helps teach to see, is it actually like PhD level coding?

Because Sam Altman keeps saying they're now like PhD level coders.

So the synopsis of this hour and a half video, these AIs ace a whole bunch of really hard things and screw up things that no student would ever get wrong and no human would ever get wrong.

And the final grades were Gemini and Claude both got a C plus in this class.

ChatGPT got a B plus.

All three of those models, which are the newest models that are mega hyped, that are told, like we're told constantly, these are PhD level smartness.

Look at all these graphs.

Every single one scored below the median student grade.

These are supposedly PhD level CS coding capabilities and they're doing worse than a freshman computer science student.

And so he ends with a quote that I really like.

He talks about how there's all these mistakes and then he says, but the complete void of creativity that permeates everything that AI touches makes the output so much less than anything we got from the real people we got taking the course.

If you put the bland, broken output from the LLMs alongside the magic the students worked, it really isn't even a comparison.

So can an AI pass a first semester CS class?

Sure, it can technically.

But looking at our students, I'm still feeling OK.