Budapest Data 2015 has ended
Back To Schedule
Thursday, June 4 • 11:35 - 12:15
The Evolution of Big Data at Spotify - Through Failures and Pain

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

The quickest way to learn and evolve infrastructure is by encountering obstacles and being forced to overcome limitations that keep you inches away from project goals. At Spotify, we’ve encountered many of these obstacles and frustrations as we grew our Big Data systems from a few machines in an office closet aggregating played song events for financial reports, to our current 1300 node Hadoop cluster and a complex architecture that plays a large role in many features that you see in our application today and processes PBs of data every week.

A member of Spotify’s Hadoop ‘squad’ will weave in war stories, failures, frustrations and lessons learned to describe the Hadoop/Big Data architecture at Spotify and talk about how that architecture has evolved over time.

avatar for Josh Baer

Josh Baer

Hadoop Product Owner, Spotify
Josh ‘joined the band’ at Spotify in early 2013 and has worked on a small team focusing on stabilizing and enhancing the Hadoop infrastructure: performing multiple migrations, upgrades and growing the cluster from 190 nodes to over 1300+. Josh holds a BS in Computer Science/Philosophy... Read More →

Thursday June 4, 2015 11:35 - 12:15 CEST
Mátyás I-II.

Attendees (0)