Description

Lecture 3: Business Intelligence: OLAP, Data Warehouse, and Column Store 1 Why we still study OLAP/Data Warehouse in Big Data? Understand the Big Data history How does the requirement of (big) data analytics/business intelligence evolve over the time? What are the architecture and implementation techniques being developed? Will they still be useful in Big Data? Understand their limitation and what factors have changed from 90’s to now? NoSQL is not only SQL☺ Hive/Impala aims to provide OLAP/BI for Big Data using Hadoop 2 Highlights OLAP Multi-relational Data model Operators SQL Data warehouse (architecture, issues, optimizations) Join Processing Column Stores (Optimized for OLAP workload) 3 Let’s get back to the root in 70’s: Relational Database Basic Structure Formally, given sets D1, D2, …. Dn a relation r is a subset of D1 x D2 x … x D n Thus, a relation is a set of n-tuples (a1, a2, …, an) where each ai  Di Example: customer_name = {Jones, Smith, Curry, Lindsay} customer_street = {Main, North, Park} customer_city = {Harrison, Rye, Pittsfield} Then r = { (Jones, Main, Harrison), (Smith, North, Rye), (Curry, North, Rye), (Lindsay, Park, Pittsfield) } is a relation over customer_name , customer_street, customer_city Relation Schema A1, A2, …, An are attributes R = (A1, A2, …, An ) is a relation schema Example: Customer_schema = (customer_name, customer_street, customer_city) r(R) is a relation on the relation schema R Example: customer (Customer_schema) Relation Instance The current values (relation instance) of a relation are specified by a table An element t of r is a tuple, represented by a row in a attributes table customer_name customer_street customer_city(or columns) Jones Main Harrison Smith North Rye tuples Curry North Rye (or rows) Lindsay Park Pittsfield customer Database A database consists of multiple relations Information about an enterprise is broken up into parts, with each relation stor

AI summary