Mark Blyth
speaker
294 appearances
3 recordings
3 series
first heard Aug 2024
last heard 3 Jul
Mark Blyth’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsRecordings per month over the last 12 months — 1 in all, peaking in Jul 2026 with 1.
Appearances
A lot of people say, well, there's a lot of data that we haven't mined yet, which the obvious example that many have suggested is kind of YouTube, which has obviously, I think, 150 billion hours of video. And then secondarily to that, synthetic data, the creation of artificial data that isn't in existence yet. To what extent are those effective pushbacks?
What about the creation of new data that doesn't exist yet?
While we're on utility value of data, when we look at effectiveness of agents, I've had Alex Wang at Scale.ai on the show, and he said the hardest thing about building effective agents is most of the work that one does in an organization, you don't actually codify down in data. You remember when you were at school and it says, show your thinking or show your work.
You don't do that in an organization. You draw on the whiteboard, you map it out, and then you put down what you think in the document. The whiteboard is often not correlated in a data source. To what extent do we have the data of showing your work for models, agents to actually do in a modern enterprise?
To what extent do you think enterprises today are willing to let passive AI products into their enterprises to observe, to learn, to test? And is there really that willingness, do you think?
You said about smaller models. Help me just understand again. I'm sorry. The show is very successful, Arvind, because I think I asked the questions that everyone asked, but they're too afraid to actually admit they don't know the answers to. Why are we seeing this trend towards smaller models?
And why do we think that is the most likely outcome in the model landscape to have a world of many smaller models?
Will Moore's law not mean cost goes down dramatically in actually a relatively short three to five year period?
Where does it become a barrier and where does it not?
So we have smaller models, and they're effective, as we said, because of cost, and they're popular because of cost. What does that do to the requirements in terms of compute?
When we think about the alignment in compute and models, we had David Kahn from Sequoia on the show and he said that you would never train a frontier model on the same data center twice. Meaning that essentially there is now a misalignment in the development speed of models and that is much faster than the development speed of new hardware and compute. How do you think about that?
So we are releasing new models so fast that computers are unable to keep up with them. And as a result, you won't want to train your new model on old H100 hardware that is 18 months old. You need continuously the newest hardware for every single new frontier model.
Speaking of that commoditization, the thing that I'm interested by there is kind of the benchmarking or the determination that they are suddenly commoditized or kind of equal performance. You said before LLM evaluation is a minefield. Help me understand why is LLM evaluation a minefield?
We mentioned that, you know, some of the early use cases in terms of passing the bar, some real kind of wild applications in terms of how models are applied. I do just want to kind of move a layer deeper to the companies building the products and the leaders leading those companies. You've got Zach and Demis who are saying that AGI is further out than we think.
And then you have Sam Altman and you have Dario and Elon in some cases saying it's sooner than we think. What are your reflections and analysis on company leader predictions on AGI?
Is it possible to have a dual strategy of chasing AGI and superintelligence, as OpenAI very clearly are, and creating valuable products at the same time that can be used in everyday use? Or is that balance actually mutually exclusive?
If I push you, if you think about your priority, your priority at OpenAI is, say, achieving superintelligence and AGI. Their best researchers, their best developers, the core of their budgets will go to that. When you have dual priorities, one takes the priority. And so there is that conflict.
What did you mean when you said to me that AI companies should pivot from creating gods to building products?
Do you think it's even possible for companies to compete in any level of AGI pursuit? When you look at the players and the cash that they're willing to spend, you know, Zuck has committed $50 billion over the next three years. When you look at how much OpenAI has raised over the last three years and they carry on that run rate, it's something crazy like that.
It'd still be $38 billion short of a Zuck spend over a three-year period. Can you create AGI-like products or God-like products unless you are Google, Amazon, Apple, or Facebook?
Showing 241–260 of 294 · page 13 of 15
← Previous
Next →