Menu
Sign In Search Podcasts Charts People & Topics Add Podcast API Blog Pricing

Dylan Patel

๐Ÿ‘ค Speaker
See mentions of this person in podcasts
3551 total appearances

Appearances Over Time

Podcast Appearances

Lex Fridman Podcast
#459 โ€“ DeepSeek, China, OpenAI, NVIDIA, xAI, TSMC, Stargate, and AI Megaclusters

Their time per session is like two hours. Yeah. Character AI very likely could be optimizing this where it's like the way that this data is collected is naive or it's like you're presented a few options and you choose them. But there's that's not the only way that these models are going to be trained. It's naive stuff like talk to an anime girl.

Lex Fridman Podcast
#459 โ€“ DeepSeek, China, OpenAI, NVIDIA, xAI, TSMC, Stargate, and AI Megaclusters

I know where you're going. I mean, you can see it physiologically. Like I take three days if I'm like backpacking or something and you. You're literally breaking down addiction cycles.

Lex Fridman Podcast
#459 โ€“ DeepSeek, China, OpenAI, NVIDIA, xAI, TSMC, Stargate, and AI Megaclusters

I know where you're going. I mean, you can see it physiologically. Like I take three days if I'm like backpacking or something and you. You're literally breaking down addiction cycles.

Lex Fridman Podcast
#459 โ€“ DeepSeek, China, OpenAI, NVIDIA, xAI, TSMC, Stargate, and AI Megaclusters

I know where you're going. I mean, you can see it physiologically. Like I take three days if I'm like backpacking or something and you. You're literally breaking down addiction cycles.

Lex Fridman Podcast
#459 โ€“ DeepSeek, China, OpenAI, NVIDIA, xAI, TSMC, Stargate, and AI Megaclusters

I mean, there are already tons of AI bots on the internet. Right now, it's not frequent, but every so often, I have replied to one, and they're instantly replying. I'm like, crap, that was a bot. And that is just going to become more common. They're going to get good.

Lex Fridman Podcast
#459 โ€“ DeepSeek, China, OpenAI, NVIDIA, xAI, TSMC, Stargate, and AI Megaclusters

I mean, there are already tons of AI bots on the internet. Right now, it's not frequent, but every so often, I have replied to one, and they're instantly replying. I'm like, crap, that was a bot. And that is just going to become more common. They're going to get good.

Lex Fridman Podcast
#459 โ€“ DeepSeek, China, OpenAI, NVIDIA, xAI, TSMC, Stargate, and AI Megaclusters

I mean, there are already tons of AI bots on the internet. Right now, it's not frequent, but every so often, I have replied to one, and they're instantly replying. I'm like, crap, that was a bot. And that is just going to become more common. They're going to get good.

Lex Fridman Podcast
#459 โ€“ DeepSeek, China, OpenAI, NVIDIA, xAI, TSMC, Stargate, and AI Megaclusters

There's probably a few things to keep in mind here. One is the kind of Tiananmen Square factual knowledge. How does that get embedded into the models? Two is the Gemini, what you called the Black Nazi model. incident, which is when Gemini as a system had this extra thing put into it that dramatically changed the behavior.

Lex Fridman Podcast
#459 โ€“ DeepSeek, China, OpenAI, NVIDIA, xAI, TSMC, Stargate, and AI Megaclusters

There's probably a few things to keep in mind here. One is the kind of Tiananmen Square factual knowledge. How does that get embedded into the models? Two is the Gemini, what you called the Black Nazi model. incident, which is when Gemini as a system had this extra thing put into it that dramatically changed the behavior.

Lex Fridman Podcast
#459 โ€“ DeepSeek, China, OpenAI, NVIDIA, xAI, TSMC, Stargate, and AI Megaclusters

There's probably a few things to keep in mind here. One is the kind of Tiananmen Square factual knowledge. How does that get embedded into the models? Two is the Gemini, what you called the Black Nazi model. incident, which is when Gemini as a system had this extra thing put into it that dramatically changed the behavior.

Lex Fridman Podcast
#459 โ€“ DeepSeek, China, OpenAI, NVIDIA, xAI, TSMC, Stargate, and AI Megaclusters

And then three is what most people would call general alignment, RLHF post-training. Each of these have very different scopes in how they are applied. In order to do, if you're just going to look at the model weights, in order to audit specific facts is extremely hard because you have to chrome through the pre-training data and look at all of this.

Lex Fridman Podcast
#459 โ€“ DeepSeek, China, OpenAI, NVIDIA, xAI, TSMC, Stargate, and AI Megaclusters

And then three is what most people would call general alignment, RLHF post-training. Each of these have very different scopes in how they are applied. In order to do, if you're just going to look at the model weights, in order to audit specific facts is extremely hard because you have to chrome through the pre-training data and look at all of this.

Lex Fridman Podcast
#459 โ€“ DeepSeek, China, OpenAI, NVIDIA, xAI, TSMC, Stargate, and AI Megaclusters

And then three is what most people would call general alignment, RLHF post-training. Each of these have very different scopes in how they are applied. In order to do, if you're just going to look at the model weights, in order to audit specific facts is extremely hard because you have to chrome through the pre-training data and look at all of this.

Lex Fridman Podcast
#459 โ€“ DeepSeek, China, OpenAI, NVIDIA, xAI, TSMC, Stargate, and AI Megaclusters

And then that's terabytes of files and look for very specific words or hints of the words.

Lex Fridman Podcast
#459 โ€“ DeepSeek, China, OpenAI, NVIDIA, xAI, TSMC, Stargate, and AI Megaclusters

And then that's terabytes of files and look for very specific words or hints of the words.

Lex Fridman Podcast
#459 โ€“ DeepSeek, China, OpenAI, NVIDIA, xAI, TSMC, Stargate, and AI Megaclusters

And then that's terabytes of files and look for very specific words or hints of the words.

Lex Fridman Podcast
#459 โ€“ DeepSeek, China, OpenAI, NVIDIA, xAI, TSMC, Stargate, and AI Megaclusters

So if you want to get rid of facts in a model, you have to do it at every stage. You have to do it at the pre-training. So most people think that pre-training is where most of the knowledge is put into the model and then you can elicit and move that in different ways, whether through post-training or whether through systems afterwards.

Lex Fridman Podcast
#459 โ€“ DeepSeek, China, OpenAI, NVIDIA, xAI, TSMC, Stargate, and AI Megaclusters

So if you want to get rid of facts in a model, you have to do it at every stage. You have to do it at the pre-training. So most people think that pre-training is where most of the knowledge is put into the model and then you can elicit and move that in different ways, whether through post-training or whether through systems afterwards.

Lex Fridman Podcast
#459 โ€“ DeepSeek, China, OpenAI, NVIDIA, xAI, TSMC, Stargate, and AI Megaclusters

So if you want to get rid of facts in a model, you have to do it at every stage. You have to do it at the pre-training. So most people think that pre-training is where most of the knowledge is put into the model and then you can elicit and move that in different ways, whether through post-training or whether through systems afterwards.

Lex Fridman Podcast
#459 โ€“ DeepSeek, China, OpenAI, NVIDIA, xAI, TSMC, Stargate, and AI Megaclusters

I almost think it's practically impossible. Because you effectively have to remove them from the internet.