James Zou
speaker
371 appearances
1 recordings
1 series
first heard Dec 2024
last heard Dec 2024
James Zou’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsNo recordings in the last 12 months.Older appearances are listed below; set an alert to hear about the next one.
Appearances
written down in words in description, right?
If I look at all the protein databases, people have tried annotated different concepts, but everybody recognizes that we're limited in what we have annotated.
There are a lot of things that we just haven't annotated.
And I think to your point, this is a really exciting opportunity where these protein language models, they I think they have learned new concepts.
They have filled in some of the gaps in our knowledge.
And now it's our job to see can we extract those out using techniques like SAEs.
That's right.
Yeah.
That's right for these ESM models.
It's an interesting question.
In some ways, I I think the the the proteins is a more constrained space than the the entirety of human language, right?
Like, you know, with human language there there's so many things you can talk about outside of science, uh but in in all these other domains and it could be code, right?
It could be talk about different images.
So th I think there there's actually a lot of diversity of information.
Uh but if I just look at only the language of proteins, in some ways it is the more narrow domain, right?
That being said, I think we are in some sense sort of scratching the tip of the iceberg in that I do think that's and something that we're actively working on now is also
um applying these approaches to larger protein language models.
For example, we're also applying that now to like alpha fold models where there's actually more structural information and those models are larger.
And I think there's actually going to be even some additional concepts that we'll be able to extract out from these different models.
Well, I I'm glad that you are uh excited about this work.
Showing 321–340 of 371 · page 17 of 19
← Previous
Next →