Joscha Bach
๐ค SpeakerAppearances Over Time
Podcast Appearances
Solomonov induction, where you are trying to understand intelligence as predicting future observations from past observations, which is intrinsic to data compression. Mm-hmm. And predictive coding is a paradigm with this boundary between neuroscience and physics and computer science.
So it's not something that is completely alien, but this radical thing that you only do next token prediction and see what happens is something where most people, I think, were surprised that this works so well.
So it's not something that is completely alien, but this radical thing that you only do next token prediction and see what happens is something where most people, I think, were surprised that this works so well.
So it's not something that is completely alien, but this radical thing that you only do next token prediction and see what happens is something where most people, I think, were surprised that this works so well.
The idea that compression is sufficient to produce all the desired behaviors is a very radical idea.
The idea that compression is sufficient to produce all the desired behaviors is a very radical idea.
The idea that compression is sufficient to produce all the desired behaviors is a very radical idea.
It's something that wouldn't work in biological organisms, I believe. Biological organisms have something like next-frame prediction for our perceptual system, where we try to filter out principal components out of the perceptual data and build hierarchies over them to track the world.
It's something that wouldn't work in biological organisms, I believe. Biological organisms have something like next-frame prediction for our perceptual system, where we try to filter out principal components out of the perceptual data and build hierarchies over them to track the world.
It's something that wouldn't work in biological organisms, I believe. Biological organisms have something like next-frame prediction for our perceptual system, where we try to filter out principal components out of the perceptual data and build hierarchies over them to track the world.
But our behavior ultimately is directed by hundreds of physiological and probably dozens of social and a few cognitive needs. that are intrinsic to us, that are built into the system as reflexes and direct us until we can transcend them and replace them by instrumental behavior that relates to our higher goals.
But our behavior ultimately is directed by hundreds of physiological and probably dozens of social and a few cognitive needs. that are intrinsic to us, that are built into the system as reflexes and direct us until we can transcend them and replace them by instrumental behavior that relates to our higher goals.
But our behavior ultimately is directed by hundreds of physiological and probably dozens of social and a few cognitive needs. that are intrinsic to us, that are built into the system as reflexes and direct us until we can transcend them and replace them by instrumental behavior that relates to our higher goals.
Yes. Of course, there is not this degree of simultaneity in the biological system. But again, I don't know whether this is actually an optimization if we imitate biology here. Because creating something like simultaneity is necessary for many processes that happen in the brain.
Yes. Of course, there is not this degree of simultaneity in the biological system. But again, I don't know whether this is actually an optimization if we imitate biology here. Because creating something like simultaneity is necessary for many processes that happen in the brain.
Yes. Of course, there is not this degree of simultaneity in the biological system. But again, I don't know whether this is actually an optimization if we imitate biology here. Because creating something like simultaneity is necessary for many processes that happen in the brain.
And you see the outcome of that by synchronized brainwaves, which suggests that there is indeed synchronization going on, but the synchronization creates overhead. And this overhead is going to make the cells more expensive to run, and you need more redundancy, and it makes the system slower.
And you see the outcome of that by synchronized brainwaves, which suggests that there is indeed synchronization going on, but the synchronization creates overhead. And this overhead is going to make the cells more expensive to run, and you need more redundancy, and it makes the system slower.
And you see the outcome of that by synchronized brainwaves, which suggests that there is indeed synchronization going on, but the synchronization creates overhead. And this overhead is going to make the cells more expensive to run, and you need more redundancy, and it makes the system slower.
So if you can build a system in which the simultaneous need gets engineered into it, maybe you have a benefit that you can exploit that is not available to the biological system and that you should not discard right away.