Dylan Patel

hell yeah that's that's what i'm saying and yeah and that's why it's like we want this whole open language models thing the olmo thing is to try to keep the model where everything is open with the data as close to the frontier as possible so we're compute constrained we're personnel constrained we're

hell yeah that's that's what i'm saying and yeah and that's why it's like we want this whole open language models thing the olmo thing is to try to keep the model where everything is open with the data as close to the frontier as possible so we're compute constrained we're personnel constrained we're

We rely on getting insights from people like John Schulman tells us to do RL on outputs like we can make these big jumps, but it just takes a long time to push the frontier of open source. And fundamentally, I would say that that's because open source AI does not have the same feedback loops as open source software. We talked about open source software for security.

We rely on getting insights from people like John Schulman tells us to do RL on outputs like we can make these big jumps, but it just takes a long time to push the frontier of open source. And fundamentally, I would say that that's because open source AI does not have the same feedback loops as open source software. We talked about open source software for security.

We rely on getting insights from people like John Schulman tells us to do RL on outputs like we can make these big jumps, but it just takes a long time to push the frontier of open source. And fundamentally, I would say that that's because open source AI does not have the same feedback loops as open source software. We talked about open source software for security.

Also, it's just because you build something once and you can reuse it. If you go into a new company, there's so many benefits. But if you open source a language model, you have this data sitting around, you have this training code. It's not like that easy for someone to come and build on and improve because you need to spend a lot on compute. You need to have expertise.

Also, it's just because you build something once and you can reuse it. If you go into a new company, there's so many benefits. But if you open source a language model, you have this data sitting around, you have this training code. It's not like that easy for someone to come and build on and improve because you need to spend a lot on compute. You need to have expertise.

Also, it's just because you build something once and you can reuse it. If you go into a new company, there's so many benefits. But if you open source a language model, you have this data sitting around, you have this training code. It's not like that easy for someone to come and build on and improve because you need to spend a lot on compute. You need to have expertise.

So until there are feedback loops of open source AI, it seems like mostly an ideological mission. Like people like Mark Zuckerberg, which is like America needs this. And I agree with him, but In the time where the motivation ideologically is high, we need to capitalize and build this ecosystem around what benefits do you get from seeing the language model data. And there's not a lot about that.

So until there are feedback loops of open source AI, it seems like mostly an ideological mission. Like people like Mark Zuckerberg, which is like America needs this. And I agree with him, but In the time where the motivation ideologically is high, we need to capitalize and build this ecosystem around what benefits do you get from seeing the language model data. And there's not a lot about that.

So until there are feedback loops of open source AI, it seems like mostly an ideological mission. Like people like Mark Zuckerberg, which is like America needs this. And I agree with him, but In the time where the motivation ideologically is high, we need to capitalize and build this ecosystem around what benefits do you get from seeing the language model data. And there's not a lot about that.

We're going to try to launch a demo soon where you can look at an OMO model and a query and see what pre-training data is similar to it, which is like legally risky and complicated, but it's like... what does it mean to see the data that the AI was trained on? It's hard to parse. It's terabytes of files. It's like, I don't know what I'm going to find in there.

We're going to try to launch a demo soon where you can look at an OMO model and a query and see what pre-training data is similar to it, which is like legally risky and complicated, but it's like... what does it mean to see the data that the AI was trained on? It's hard to parse. It's terabytes of files. It's like, I don't know what I'm going to find in there.

We're going to try to launch a demo soon where you can look at an OMO model and a query and see what pre-training data is similar to it, which is like legally risky and complicated, but it's like... what does it mean to see the data that the AI was trained on? It's hard to parse. It's terabytes of files. It's like, I don't know what I'm going to find in there.

But that's what we need to do as an ecosystem if people want open source AI to be financially useful.

But that's what we need to do as an ecosystem if people want open source AI to be financially useful.

But that's what we need to do as an ecosystem if people want open source AI to be financially useful.

I had a while to think about this while listening to Dylan's beautiful response. He didn't listen to me. He was so dumb. No, I knew this was coming. And it's like, realistically, training models is very fun because there's so much low-hanging fruit. And the thing that makes my job entertaining, I train models. I write analysis about what's happening with models.

I had a while to think about this while listening to Dylan's beautiful response. He didn't listen to me. He was so dumb. No, I knew this was coming. And it's like, realistically, training models is very fun because there's so much low-hanging fruit. And the thing that makes my job entertaining, I train models. I write analysis about what's happening with models.

I had a while to think about this while listening to Dylan's beautiful response. He didn't listen to me. He was so dumb. No, I knew this was coming. And it's like, realistically, training models is very fun because there's so much low-hanging fruit. And the thing that makes my job entertaining, I train models. I write analysis about what's happening with models.