OpenObservability Talks

Shopify’s Journey to Planet-Scale Observability - OpenObservability Talks S5E09

27 Feb 2025

Audio

Description

Shopify operates at massive scale, running thousands of services and processing billions of events per second. To tackle the challenges of observability at this scale, they built Observe—an in-house observability stack that makes use of open-source tools and specifications. In fact, they replaced an older vendors-based system, in an awe-inspiring migration project. But why build their own stack? Which open source tools did they use? How did they shape the user experience to their needs?Joining us to unpack Shopify’s journey is Elijah McPherson, an engineering leader with deep expertise in observability and distributed systems. Elijah led the complete rebuild of Shopify’s observability stack and now also oversees jobs, caching, search, and ClickHouse infrastructure. Tune in to hear firsthand insights from one of the most innovative purpose-built observability implementations in production today!The episode was live-streamed on 11 February 2025 and the video is available at https://www.youtube.com/watch?v=rBfTjlXKJW0OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube.We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and chime in with your comments and questions on the live chat.⁠⁠https://www.youtube.com/@openobservabilitytalks⁠ https://www.twitch.tv/openobservability⁠Show Notes:00:46 - Episode and guest intro03:43 - Why rebuild the observability stack in house 05:47 - Cost and vendor lock-in07:09 - Tailoring observability for the organizational processes10:27 - How to build a team to build in-house observability 13:37 - The importance of product sense in internal platforms18:05 - The functionality of Shopify’s observability platform 25:15 - The Open Source stack used at Shopify observability 29:50 - Extending open source Grafana to Shopify’s needs36:23 - Adopting open standards 42:26 - observability into business health45:16 - how to run a migration project for a live production platform53:15 - final tips and best practices 56:41 - which organizations should develop in-house observabilityResources: Episode: Scaling Platform Engineering: Shopify’s Blueprint: https://medium.com/p/f18e97140681 Shopify Observe - lectures: https://www.linkedin.com/posts/elijahmcpherson_observe-activity-7258195493657223168-mOGS/ Socials:Twitter:⁠ https://twitter.com/OpenObserv⁠YouTube: ⁠https://www.youtube.com/@openobservabilitytalks⁠Dotan Horovits============Twitter:@horovitsLinkedIn:www.linkedin.com/in/horovitsMastodon: @horovits@fosstodonBlueSky: @horovits.bsky.socialElijah McPherson===============Twitter: https://twitter.com/ElijahMcPhersonLinkedIn: https://www.linkedin.com/in/elijahmcpherson/

Transcription

This episode hasn't been transcribed yet

Help us prioritize this episode for transcription by upvoting it.

0 upvotes

🗳️ Sign in to Upvote

Popular episodes get transcribed faster

Other recent transcribed episodes

Transcribed and ready to explore now

13:00H | 21 DIC 2025 | Fin de Semana

01 Jan 1970

Fin de Semana

10:00H | 21 DIC 2025 | Fin de Semana

01 Jan 1970

Fin de Semana

12:00H | 20 DIC 2025 | Fin de Semana

01 Jan 1970

Fin de Semana

2ª PARTE | 06 ENE 2026 | EL PARTIDAZO DE COPE

01 Jan 1970

El Partidazo de COPE

3ª PARTE | 22 ENE 2026 | EL PARTIDAZO DE COPE

01 Jan 1970

El Partidazo de COPE

3ª PARTE | 04 MAR 2026 | EL PARTIDAZO DE COPE

01 Jan 1970

El Partidazo de COPE

Comments

There are no comments yet.

Please log in to write the first comment.

Report any issue

OpenObservability Talks

Shopify’s Journey to Planet-Scale Observability - OpenObservability Talks S5E09

This episode hasn't been transcribed yet

Other recent transcribed episodes

13:00H | 21 DIC 2025 | Fin de Semana

10:00H | 21 DIC 2025 | Fin de Semana

12:00H | 20 DIC 2025 | Fin de Semana

2ª PARTE | 06 ENE 2026 | EL PARTIDAZO DE COPE

3ª PARTE | 22 ENE 2026 | EL PARTIDAZO DE COPE

3ª PARTE | 04 MAR 2026 | EL PARTIDAZO DE COPE

Sign in to Audioscrape

Share this moment