AI Realized Podcast

Evaluating AI Models: Cost, Latency, and Quality: A conversation with Ivan Lee, founder/CEO of Datasaur

14 Feb 2025

Audio

Description

In this podcast episode of AI Realized, Ivan Lee, founder and CEO of Datasaur, discusses his company's mission to democratize access to natural language processing (NLP). Datasaur introduced a data labeling platform used by major organizations like Netflix and the FBI and launched LLM Labs to facilitate the customization of large language models (LLMs). Ivan elaborates on the importance of evaluating AI models in production, focusing on cost, latency, and quality, and introduces the concept of prompt unit testing to ensure consistent model performance. He highlights the need for data scientists to understand business-side ROI and notes the significant unit costs associated with LLMs. Ivan explores the potential for cost reduction driven by innovations like OpenAI's GPT 4o Mini and open-source models like Lama. He also considers the implications of running LLMs on-device for industries with strict data privacy needs. Lastly, Ivan advises enterprise executives to start small, piloting AI solutions to demonstrate their effectiveness, and emphasizes the future of a multi-model AI solution landscape within organizations.00:00 Introduction to AI Realized Podcast01:04 Meet Ivan Lee, CEO of Datasaur01:18 Datasaur's Mission and Products02:31 Ensuring Quality in AI Models03:51 Calculating ROI for AI Use Cases06:41 Cost Reduction in AI07:47 Future of AI Deployment09:46 Advice for Enterprise Executives12:22 Key Takeaways and Conclusion

Transcription

This episode hasn't been transcribed yet

Help us prioritize this episode for transcription by upvoting it.

0 upvotes

🗳️ Sign in to Upvote

Popular episodes get transcribed faster

Other recent transcribed episodes

Transcribed and ready to explore now

13:00H | 21 DIC 2025 | Fin de Semana

01 Jan 1970

Fin de Semana

10:00H | 21 DIC 2025 | Fin de Semana

01 Jan 1970

Fin de Semana

12:00H | 20 DIC 2025 | Fin de Semana

01 Jan 1970

Fin de Semana

2ª PARTE | 06 ENE 2026 | EL PARTIDAZO DE COPE

01 Jan 1970

El Partidazo de COPE

3ª PARTE | 22 ENE 2026 | EL PARTIDAZO DE COPE

01 Jan 1970

El Partidazo de COPE

3ª PARTE | 04 MAR 2026 | EL PARTIDAZO DE COPE

01 Jan 1970

El Partidazo de COPE

Comments

There are no comments yet.

Please log in to write the first comment.

Report any issue

AI Realized Podcast

Evaluating AI Models: Cost, Latency, and Quality: A conversation with Ivan Lee, founder/CEO of Datasaur

This episode hasn't been transcribed yet

Other recent transcribed episodes

13:00H | 21 DIC 2025 | Fin de Semana

10:00H | 21 DIC 2025 | Fin de Semana

12:00H | 20 DIC 2025 | Fin de Semana

2ª PARTE | 06 ENE 2026 | EL PARTIDAZO DE COPE

3ª PARTE | 22 ENE 2026 | EL PARTIDAZO DE COPE

3ª PARTE | 04 MAR 2026 | EL PARTIDAZO DE COPE

Sign in to Audioscrape

Share this moment