Big Data Analytics Big Data Analytics Lars Schmidt-Thieme Information Systems and Machine Learning Lab (ISMLL) Institute for Computer Science University of Hildesheim, Germany Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Germany 1 / 40
Big Data Analytics Outline 1. What is Big Data? 2. Overview 3. Organizational Stuff Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Germany 2 / 40
Big Data Analytics 1. What is Big Data? Outline 1. What is Big Data? 2. Overview 3. Organizational Stuff Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Germany 1 / 40
Big Data Analytics 1. What is Big Data? What is Big Data? Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Germany 1 / 40
Big Data Analytics 1. What is Big Data? What is Big Data? “Big data is like teenage sex: — Dan Ariely Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Germany 2 / 40
Big Data Analytics 1. What is Big Data? What is Big Data? “Big data is like teenage sex: everyone talks about it, — Dan Ariely Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Germany 2 / 40
Big Data Analytics 1. What is Big Data? What is Big Data? “Big data is like teenage sex: everyone talks about it, nobody really knows how to do it, — Dan Ariely Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Germany 2 / 40
Big Data Analytics 1. What is Big Data? What is Big Data? “Big data is like teenage sex: everyone talks about it, nobody really knows how to do it, everyone thinks everyone else is doing it, — Dan Ariely Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Germany 2 / 40
Big Data Analytics 1. What is Big Data? What is Big Data? “Big data is like teenage sex: everyone talks about it, nobody really knows how to do it, everyone thinks everyone else is doing it, so everyone claims they are doing it.” — Dan Ariely Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Germany 2 / 40
Big Data Analytics 1. What is Big Data? What is Big Data? Some definitions: I “data sets that are so voluminous and complex that traditional data processing application software are inadequate to deal with them.” [http://en.wikipedia.org/wiki/Big_data] I “Big data is high-volume, high-velocity and/or high-variety information assets that demand cost-effective, innovative forms of information processing for enhanced insight and decision making.” [www.gartner.com/it-glossary/big-data/] Note: The “3 Vs” go back to Laney [2001]. Often a 4th V “veracity” and a 5th V “value” is used. Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Germany 3 / 40
Big Data Analytics 1. What is Big Data? Big Data — Dimensions (the “4 Vs”) Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Germany 4 / 40
Big Data Analytics 1. What is Big Data? What is Big Data? Big Data is about: I Storing and accessing I large amounts of I (complex/unstructured) data I Processing high volume data streams I Making sense of the data I Making predictions based on the data Note: Unstructured data in this context means, data that is not already a vector. Some of this data, e.g., relational data and graph data, confusingly often also is called “structured”. Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Germany 5 / 40
Big Data Analytics 1. What is Big Data? Where to find Big Data? I 1.52 billion daily active users (2.32 billion monthly active users, Dec. 2018) I size of user data stored by Facebook: 300 Petabytes I average amount of data that Facebook takes in daily: 600 terabytes I size of Facebook’s graph search database: 700 Terabytes [source: https://newsroom.fb.com/company-info/; online source for points 2–4 vanished] Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Germany 6 / 40
Big Data Analytics 1. What is Big Data? Where to find Big Data? I 3.3 billion searches per day (on average)1 I 30 trillion unique URLs identified on the Web1 I 20 billion sites crawled a day1 I In 2008 Google processed more than 20 Petabytes of data per day2 1http://searchengineland.com/google-search-press-129925 (2012) 2Jeffrey Dean and Sanjay Ghemawat. 2008. MapReduce: simplified data processing on large clusters. Commun. ACM 51, 1 (January 2008), 107-113. Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Germany 7 / 40
Big Data Analytics 1. What is Big Data? Where to find Big Data? I tweets per day: 58 million1 I Twitter search engine queries per day: 2.1 billion1 I registered/active Twitter users: 695 million / 342 million1 [1http://www.statisticbrain.com/twitter-statistics/] (9/2016) Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Germany 8 / 40
Big Data Analytics 1. What is Big Data? Where to find Big Data? I Ensembl database contains the genome of humans and 50 other species I “only” 250 GB1 [1http://www.ensembl.org/] Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Germany 9 / 40
Big Data Analytics 1. What is Big Data? Where to find Big Data? I CERN Large Hadron Collider has collected data from over 300 trillion proton-proton collisions I Approx. 25 Petabytes per year Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Germany 10 / 40
Big Data Analytics 1. What is Big Data? Big Data — Public Datasets 1000 Genomes Project DNA of 1700 humans 200 TB Common Crawl Corpus 5G web pages 81 TB Wikipedia / Freebase 1.9G subject/predicate/object triples 250 GB Million Song Dataset audio features of 1M songs 280 GB OpenStreetMap a map of earth 90 GB 2000 US Census US census data 200 GB PubChem library biological activities of small molecules 230 GB NCDC weather data daily measurements from 9000 stations 20 GB Open Library metadata of 20M books 7 GB Twitter 1.6G tweets 0.6 GB CD 700 MB, DVD 4.7–17 GB, Blu-ray 25–100 GB, hard disc: 10 TB. Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Germany 11 / 40
Big Data Analytics 1. What is Big Data? How Large is 1 Petabyte I 1 PB = 1000 TB = 1015 B Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Germany 12 / 40
Big Data Analytics 1. What is Big Data? How Large is 1 Petabyte I 1 PB = 1000 TB = 1015 B I 35.7M years counted one byte per second, Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Germany 12 / 40
Hiển thị 20 trang đầu. Tải tài liệu để xem đầy đủ.
Chưa có bình luận nào. Hãy là người đầu tiên!
Bình luận (0)
Chưa có bình luận nào. Hãy là người đầu tiên!