Loading…
Budapest Data 2015 has ended
Back To Schedule
Wednesday, June 3 • 16:50 - 17:20
Interactive Graph Analytics with Spark

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

The Spark community has a lot of experience using Spark for offline batch analysis tasks coming from a broad range of use cases. But creating an interactive web application which aims for sub-second response times using Spark as the computation backend is still a somewhat unexplored territory. We at Lynx Analytics wandered into this territory when we built LynxKite, our big graph analysis tool. The tool enables users to interactively explore graphs of hundreds of millions of vertices and billions of edges. Exploration includes global and local views of the graph featuring visualization of attributes, connections and distributions. This talk is about the technical challenges — general and domain specific — we faced during building this software and about our solutions. We will talk about problems like scheduler delay, GC pauses, interoperability with other Akka based libraries and solutions like sorted RDDs, prefix sampling, and column based attribute representation.

Speakers
avatar for Darabos Dániel

Darabos Dániel

Programozó, Lynx Analytics
Dániel has been member of the LynxKite developers team in Budapest since the very beginning of the project. Prior to this he worked at Google SRE team in Dublin.


Wednesday June 3, 2015 16:50 - 17:20 CEST
Mátyás I

Attendees (0)