Columnar Access with HBase Lecture BigData Analytics Julian M. Kunkel julian.kunkel@googlemail.com University of Hamburg / German Climate Computing Center (DKRZ) Disclaimer: Big Data software is constantly updated, code samples may be outdated. Introduction Excursion: ZooKeeper Architecture Accessing Data Summary Outline Introduction Excursion: ZooKeeper Architecture Accessing Data Summary Julian M. Kunkel Lecture BigData Analytics, 2016 Introduction Excursion: ZooKeeper Architecture Accessing Data Summary Overview of HBase [29, 30] Column-oriented key-value database for structured data Based on Google’s BigTable Simple data and consistency model Row “Hans” “Julian” column1 bla NULL column2 Scalable for billion of rows with millions of columns Sharding of tables: distribute keys automatically among servers Stretches across data centers Custom query language Real-time queries Compression, in-memory execution Bloom filters and block cache to speed up queries Use HDFS and supports MapReduce Uses ZooKeeper for configuration, notification and synchronization Interactive shell (invoke hbase shell) Julian M. Kunkel Lecture BigData Analytics, 2016 Introduction Excursion: ZooKeeper Architecture Accessing Data Summary Data Model [29] Namespace: Logical grouping of tables for quota, security Table: A table (namespae:table) consists of multiple rows Row: Consists of a row key and (many) columns with values Key/values are binary (converted from any data type) WARNING: hbase shell stores all data as STRING Column: Consists of a column family and a qualifier (cf:q) Column family: string with printable characters Cell: Combination of row, column Contains value (byte array) and timestamp (last modification) Timestamp: versions that change upon update Student grading table (timestamps are not shown) Row=Matrikel stud/4711 stud/4712 Julian M. Kunkel a:name Hans Julian a:age 19 20

Lire le document entier