Joscha Bach
speaker
510 appearances
2 recordings
2 series
first heard Aug 2023
last heard 23 Jan
Joscha Bach’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsRecordings per month over the last 12 months — 1 in all, peaking in Jan 2026 with 1.
Appearances
I think it's totally okay to be sad about goodbyes, because that indicates that there was something that you're going to miss.
But you have to say goodbye before you say hello again.
And I'm no stranger to melancholy, and sometimes it's difficult to bear to be alive. Sometimes it's just painful to exist.
It can also kill you.
I find that large language models do help with coding, right? So it's an extremely useful application that is for a lot of people taking stack overflow out of their life in exchange for something that is more efficient. I feel that ChatGPT is like an intern that I have to micromanage. I have been working with people in the past who were less capable than ChatGPT.
and, uh, I'm not saying this because I hate people, but they personally, as human beings, there was something present that was not there in Chet Chibiti, which was why I was covering for them. But, uh, ChatGPT has an interesting ability. It does give people superpowers. And the people who feel threatened by them are the prompt completers.
They are the people who do what ChatGPT is doing right now. So if you are not creative, if you don't build your own thoughts, if you don't have actual plans in the world, and your only job is to summarize emails and to expand simple intentions into emails again, then ChatGPT might look like a threat.
But I believe that it is a very beneficial technology that allows us to create more interesting stuff and make the world more beautiful and fascinating if we find to build it into our life in the right ways. So I'm quite fascinated by these large language models, but I also think that they are by no means the final development. And it's interesting to see how this development progresses.
One thing that the out-of-the-box vanilla language models have as a limitation is that they have still some limited coherence and ability to construct complexity. And even though they exceed human abilities to do what they can do one shot,
Typically, when you write a text with a language model or using it, or when you write code for the language model, it's not one shot because they're going to be bugs in your program and design errors and compiler errors and so on. And your language model can help you to fix those things. But this process is out of the box, not automated yet.
So there is a management process that also needs to be done. And there are some interesting developments, baby AGI and so on, that are trying to automate this management process as well. And I suspect that soon we are going to see a bunch of cognitive architectures where every module is in some sense a language model or something equivalent.
And between the language models, we exchange suitable data structures, not English, and produce compound behavior of this whole thing.
There are limitations in a language model alone. I feel that part of my mind works similarly to a language model, which means I can yell into it a prompt and it's going to give me a creative response. But I have to do something with this response first. I have to take it as a generative artifact that may or may not be true. It's usually a confabulation. It's just an idea.
And then I take this idea and modify it. I might build a new prompt that is stepping off this idea and develops it to the next level or put it into something larger. Or I might try to prove whether it's true or make an experiment. And this is what the language models right now are not doing yet. But there's also no technical reason for why they shouldn't be able to do this.
So the way to make a language model coherent is probably not to use reinforcement learning until it only gives you one possible answer that is linking to its source data. But it's using this as a component in a larger system that can also be built by the language model or is enabled by language model structured components or using different technologies.
I suspect that language models will be an important stepping stone in developing different types of systems. And one thing that is really missing in the form of language models that we have today is real-time world coupling. It's difficult to do perception with a language model and motor control with a language model. Instead, you would need to have different type of thing that is...
working with it. Also, the language model is a little bit obscuring what its actual functionality is. Some people associate the structure of the neural network of the language model with the nervous system, and I think that's the wrong intuition. Neural networks are unlike nervous systems. They are more like
100-step functions that use differentiable linear algebra to approximate the correlation between adjacent brain states. It's basically a function that moves the system from one representational state to the next representational state.
If you try to map this into a metaphor that is closer to our brain, imagine that you would take a language model or a model like DALL-E, that you use, for instance, image-guided diffusion to approximate a camera image and use the activation state of the neural network to interpret the camera image, which in principle I think will be possible very soon. You do this periodically.
And now you look at these patterns, how, when this thing interacts with the world periodically, look like in time. And these time slices, they are somewhat equivalent to the activation state of the brain at a given moment.
Showing 241–260 of 510 · page 13 of 26
← Previous
Next →