Dr. Alexander Wissner-Gross
👤 SpeakerAppearances Over Time
Podcast Appearances
That's easier said than done.
Friends don't let friends start non-profits, I know.
I would say, taking the high road, I would say if that's your experience, you have almost an obligation to humanity to create benchmarks to encode this knowledge that you have that you think isn't being accurately or fully or effectively reflected right now in AIs.
If you were to create a Cutler bench that encodes all of your wisdom, all of your writing knowledge.
But it's not my wisdom.
It's just basic writing skills.
Let me take a quick poll here in the room.
OK, that's fair.
Stephen, I would just, again, encourage you as a scientist, as you say, to be rigorous in how you measure progress or lack thereof.
If you're going to assert that there's been a regression in terms of creative writing capabilities, for example, create your own benchmark and then show the world.
Show the world that there's been a regression, and I will promise you, if you construct it well, you will get the Frontier Labs interested in including your benchmark in every other eval suite.
As an optimization function.
That's right.
And you will get your better creative writing.
Saleem, do you want to have a final point here?
I think this falls under the category of don't feed the trolls.
It's probably right.
I've done the same extrapolations others have.
You extrapolate the humanoids curve here, you find that in the early 2030s, the number of humanoids is predicted to cross the number of wheeled robots.
I would assume there's relatively little alpha left at this point in