邵赛赛

Hortonworks技术专家

Saisai Shao works as a member of technical staff at Hortonworks, mainly focused on Spark area, especially Spark core, Spark on Yarn and Spark Streaming. He is an active Apache Spark contributor and Apache Chukwa committer. Prior to Hortonworks, he was a software engineer at Intel working on performance tuning and optimization of Hadoop and Spark.

Nowadays Spark is adopted more and more extensively, with its flexible design of framework, it can be run on public cloud, dedicated or multi-tenant cluster with different cluster manager: Standalone, Mesos and YARN. In this talk, we will focus on Spark running on YARN, to introduce how does Spark run on YARN and why Spark is better on YARN. Also we will cover best practice for Spark running on YARN, especially for in-production environment. Finally future works for this area will be discussed, like container resizing, better long-running supports, ATS integration.