Big Data Analytics Lecture BigData Analytics Julian M. Kunkel julian.kunkel@googlemail.com University of Hamburg / German Climate Computing Center (DKRZ) Disclaimer: Big Data software is constantly updated, code samples may be outdated. BigData Challenges Gaining Insight with Analytics Use Cases Programming Summary Outline 1 Lecture 2 Big Data Analytics 3 BigData Challenges 4 Gaining Insight with Analytics 5 Use Cases 6 Programming 7 Summary Julian M. Kunkel Lecture BigData Analytics, WiSe 17/18 2 / 59 Lecture Big Data Analytics BigData Challenges Gaining Insight with Analytics Use Cases Programming Summary About DKRZ German Climate Computing Center (DKRZ) Partner for Climate Research Maximum Compute Performance. Sophisticated Data Management. Competent Service. Julian M. Kunkel Lecture BigData Analytics, WiSe 17/18 3 / 59 Lecture Big Data Analytics BigData Challenges Gaining Insight with Analytics Use Cases Programming Summary Scientific Computing Research Group of Prof. Ludwig at the University of Hamburg Embedded into DKRZ Research Analysis of parallel I/O Alternative I/O interfaces I/O & energy tracing tools Data reduction techniques Middleware optimization Cost & energy efficiency Julian M. Kunkel Lecture BigData Analytics, WiSe 17/18 4 / 59 Lecture Big Data Analytics BigData Challenges Gaining Insight with Analytics Use Cases Programming Summary Lecture Concept of the lecture The lecture is focusing on applying technology and some theory Theory Data models and data processing Statistics and machine learning System architectures Algorithms and data structures Applying technology Learning about various state-of-the art technology Hands-on for understanding the key concepts Languages: R and Python, (Java is important but not used) The domain of big data is overwhelming, especially technology It is a crash course for several topics such as statistics and d