Menu
Sign In Search Podcasts Charts People & Topics Add Podcast API Blog Pricing

Eve Bodnia

๐Ÿ‘ค Speaker
495 total appearances

Appearances Over Time

Podcast Appearances

The Neuron: AI Explained
Why Energy-Based Models Could Be the Next Big Shift in AI

Um, no, it's, um, so the first, when we started this company, I just had like some theoretical idea, right?

The Neuron: AI Explained
Why Energy-Based Models Could Be the Next Big Shift in AI

And then I was surrounded by talented engineers who just brought this idea into a form of proof of concept.

The Neuron: AI Explained
Why Energy-Based Models Could Be the Next Big Shift in AI

And that was like a few months ago.

The Neuron: AI Explained
Why Energy-Based Models Could Be the Next Big Shift in AI

And the natural question was like, oh, can the proof of concept be the actual, like a toy model for the model we have today?

The Neuron: AI Explained
Why Energy-Based Models Could Be the Next Big Shift in AI

So the answer was yes, but to get there, we had to perform like a series of experiments to evaluate like what's right, what works, what doesn't.

The Neuron: AI Explained
Why Energy-Based Models Could Be the Next Big Shift in AI

So when the architecture was fully designed, the next step would be, oh, is it compatible with LLMs or with transformers in general?

The Neuron: AI Explained
Why Energy-Based Models Could Be the Next Big Shift in AI

Because it's so fundamentally different.

The Neuron: AI Explained
Why Energy-Based Models Could Be the Next Big Shift in AI

We didn't even know like

The Neuron: AI Explained
Why Energy-Based Models Could Be the Next Big Shift in AI

it's possible to do.

The Neuron: AI Explained
Why Energy-Based Models Could Be the Next Big Shift in AI

So the first step was to attach Transformer and try to scale it a little bit and then kind of shrink it back to the toy model version.

The Neuron: AI Explained
Why Energy-Based Models Could Be the Next Big Shift in AI

So we successfully done so.

The Neuron: AI Explained
Why Energy-Based Models Could Be the Next Big Shift in AI

And then we're like, oh, can we like not even have an LLM, but the most simplest version of something related to LLM attached to Transformant, which is small and we understand.

The Neuron: AI Explained
Why Energy-Based Models Could Be the Next Big Shift in AI

So we attach that, we also scale it and then scale it back and like, okay, that works.

The Neuron: AI Explained
Why Energy-Based Models Could Be the Next Big Shift in AI

How about we just attach the real LLM to the EBM and see how it is as a user interface.

The Neuron: AI Explained
Why Energy-Based Models Could Be the Next Big Shift in AI

Can it prompt the EBM in the way we want?

The Neuron: AI Explained
Why Energy-Based Models Could Be the Next Big Shift in AI

And the answer was yes.

The Neuron: AI Explained
Why Energy-Based Models Could Be the Next Big Shift in AI

So we again scale it and then test it.

The Neuron: AI Explained
Why Energy-Based Models Could Be the Next Big Shift in AI

We have a set of benchmarks, which is related to spatial thinking and hierarchical planning.

The Neuron: AI Explained
Why Energy-Based Models Could Be the Next Big Shift in AI

So we had baselines for the smallest version of

The Neuron: AI Explained
Why Energy-Based Models Could Be the Next Big Shift in AI

smallest version of the model, then proof of concept, then the real version of the model and kind of like compare it back and forth and it seems to be working.