Kurz je určen pro ...

Audience

• Software developers
• IT professionals

Lektoři kurzu

Lektoři z firmy: DataScript s.r.o.

[Kurz] Program kurzu (obsah přednášky/semináře/rekvalifikace/studia) ...

Goals Mastering Apache Spark is an extended workshop designed to familiarize attendees with Apache Spark - a well-known engine for fast Big Data processing. From zero to hero means that attendes do not need to have prior knowledge of how Apache Spark works, but after this workshop will be able to spin efficient, robust jobs running on Spark cluster starting from day one. They will not only gain general understanding of what Apache Spark is, what are its modules and how they work. Attendee will get detailed, in-depth insight of Apache Spark internals. This makes this workshop globally unique it prepares its graduates not only to start working with the framework, but also makes them right candidates to tackle any problems that might occur on production - A true heros that each distributed project needs.
Instructor Instructor has more then 10 years of profesional experience as software engineer - during which he worked on numerous number of projects for veriety of clients. He often
* Mastering Apache Spark is an extended workshop designed to familiarize attendees with Apache Spark - a well-known engine for fast Big Data processing. From zero to hero means that attendes do not need to have prior knowledge of how Apache Spark works, but after this workshop will be able to spin efficient, robust jobs running on Spark cluster starting from day one. They will not only gain general understanding of what Apache Spark is, what are its modules and how they work. Attendee will get detailed, in-depth insight of Apache Spark internals. This makes this workshop globally unique it prepares its graduates not only to start working with the framework, but also makes them right candidates to tackle any problems that might occur on production - A true heros that each distributed project needs.
Instructor Instructor has more then 10 years of profesional experience as software engineer - during which he worked on numerous number of projects for veriety of clients. He often call him self a veteran of early Apache Spark adoptions - his first big data project (based on Apache Spark) dates back to 2014. Back then Spark (version 0.9) was still under heavy development, where every problem encountered requried reading Spark s codebase - googling the question was pointless. That being a pain then, prove to be a great asset after a while. Experience gathered was invaluable, allowing Pawel to challange other Spark-based projects he worked on ever since. The knowledge he is now happy to share with you! Recently he was hired to write Apache Spark connector between HDFS & SlamData s Quasar project.
Outline

• Introduction
  
  • What is Apache Spark?
  • What was before?
  • Challenges, proble & issues with MapReduce
  • The Big Picture
  
  
• Spark Core
  
  • RDD
  • Transforamtions vs Actions
  • Partitions & Tasks
  • RDDs for key-value
  • Pipelining & Shuffeling
  • DAG & Stages
  • Resilience
  • Performance issues and how to handle them
    • Common pitfals (groupBy)
    • Deepr knowledge of how cluster works
    • Classpath & Serialization issues
    • Spark configuration
    • Spark UI & Spark history server
  • Caching & Checkpointing
  • Broadcasts & Accumulators
  • Joins
  
  
• Spark Core - internals
  
  • The 5 things definig RDD
  • Shuffeling algorithms
  • Spark Memory Model
  
  
• Spark SQL
  
  • Adventages of semi-structured data
  • SQL
    
    • Introduction
    • Joins
    • Hive
    
    
  • Dataframes
    
    • Introduction
    • Joins
    • JSON
    • Parquet
    • Avro
    
    
  • DataSets
    
    • Problems with Dataframes
    • Dataset to the rescue!
    
    
  • What makes Spark SQL run faster
    
    • Structure vs Expression
    • Catalyst Optimizer
    • Predicate Push Down
    • Project Tungsten
    
    
  
  
• Spark

[Kurz] Cíl školení / poznámka ke kurzu...

[dotaz na školení/kurz] Formulář pro komentáře, dotazy a odpovědi

Dotaz ke kurzu: Mastering Apache Spark

Předmět:

Text (Maximální počet znaků: 5.000, telefonní číslo vložte až do další kolonky) Ano, zobrazit    Nezobrazovat    text dotazu, ostatní data (tel/email) se samozřejmě nezveřejňují a nezobrazují, pouze budou přeposlána školící firmě

Váš email   ... a pro kontrolu zadejte prosím znova Váš email Někdy je zadaná špatná emailová adresa a odpověď se vrátí, proto prosím zadejte email dvakrát Vaše jméno             Příjmení Váš kontaktní telefon Chci poslat přihlašovací údaje na server skoleni-kurzy.eu [Registrovat jako uživatele Ano/Ne] Pokud jste přihlášeni, email nemusíte zadávat Vložte potvrzovací kód               Pro změnu potvrzovacího kódu je možno znovu načíst stránku