Real-time data and AI are converging—and companies that have already solved the data pipeline problem are pulling ahead fast. Wix processes over 40 billion interactions every day across hundreds of millions of websites, and the architecture behind that scale didn't happen by accident. It was built, lane by lane, around the principle that your upstream data must be at least as fast as your fastest use case. In this episode, Josef Goldstein, Head of R&D for the Big Data Platform at Wix.com, joins Joseph Morais to unpack how Wix evolved from a batch-based Hadoop architecture to a fully streaming-first platform on Confluent Cloud. The conversation covers Wix's multi-lane data architecture—from petabyte-scale data lakes to sub-second algorithmic decisions—how they approach data contracts and governance at distributed scale, and why Confluent Freight Clusters became a strategic unlock for cost and elasticity. Josef also explains how Wix is now wiring real-time stream processing directly into context layers for agentic AI systems. You'll Learn: How to architect a multi-lane data streaming system that serves use cases from sub-second ML inference to petabyte-scale analytics Why upstream data quality, governance, and metadata are the real prerequisites for production-grade agentic AI How Confluent Freight Clusters reduced costs and unlocked elastic scale for Wix's highest-volume, latency-tolerant data streams About the Guest: Josef Goldstein is an Israeli software engineer and engineering manager specializing in big data infrastructure and real-time analytics. With over 15 years of experience in developing data-intensive SaaS applications, Goldstein has built and led high-performing teams in the technology sector. Since 2021, he has served as the Head of R&D for Wix's Big Data and Analytics Platform, where he oversees the infrastructure and tools enabling data-driven decisions for the company's users and operations. Guest Highlight: "You understand that you need to be on the upstream side of the house as fast as your fastest lane. Otherwise, none of that is possible." Chapters: [00:00] Wix.com Overview and Customer Base [07:08] Segment 1: Data Streaming Goodness [24:24] Segment 2: Beyond the Stream [45:37] Segment 3: Quick Bytes [47:08] Joseph’s Top Takeaways Dive Deeper into Data Streaming: EP1—Stream On: Unleashing Innovation With Data Streaming | Life Is But A Stream EP2—Processing Without Pause: Continuous Stream Processing and Apache Flink® | Life Is But A Stream EP3—The Connective Tissue: Shift Left to Turn Data Chaos to Clarity | Life Is But A Stream Get Connected: Connect with Joseph: @thedatagiant Joseph’s LinkedIn: http://linkedin.com/in/thedatagiant Josef’s LinkedIn: https://www.linkedin.com/in/josefgoldstein/ Our Sponsor: Your data shouldn’t be a problem to manage. It should be your superpower. The Confluent data streaming platform transforms organizations with trustworthy, real-time data that seamlessly spans your entire environment and powers innovation across every use case. Create smarter, deploy faster, and maximize efficiency with a true data streaming platform from the pioneers in data streaming. Learn more at confluent.io. Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.

Ask this episode

“What did the council decide on item 42?” — answered with the speaker and the moment.

Transcript

Related