Big Data Analytics Big Data Analytics A. Parallel Computing / A.1 Threads Lars Schmidt-Thieme Information Systems and Machine Learning Lab (ISMLL) Institute for Computer Science University of Hildesheim, Germany Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Germany 1 / 52
Big Data Analytics Syllabus Tue. 9.4. (1) 0. Introduction A. Parallel Computing Tue. 16.4. (2) A.1 Threads Tue. 23.4. (3) A.2 Message Passing Interface (MPI) Tue. 30.4. (4) A.3 Graphical Processing Units (GPUs) B. Distributed Storage Tue. 7.5. (5) B.1 Distributed File Systems Tue. 14.5. (6) B.2 Partioning of Relational Databases Tue. 21.5. (7) B.3 NoSQL Databases C. Distributed Computing Environments Tue. 28.5. (8) C.1 Map-Reduce Tue. 4.6. — — Pentecoste Break — Tue. 11.6. (9) C.2 Resilient Distributed Datasets (Spark) Tue. 18.6. (10) C.3 Computational Graphs (TensorFlow) D. Distributed Machine Learning Algorithms Tue. 25.6. (11) D.1 Distributed Stochastic Gradient Descent Tue. 2.7. (12) D.2 Distributed Matrix Factorization Tue. 9.7. (13) Questions and Answers Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Germany 1 / 52
Big Data Analytics Outline 1. Threads Basics 2. Starting and Interrupting Threads 3. Synchronization I: Monitors 4. Synchronization II: Locks 5. Starting Threads II: Thread Pools and Dependency Graphs 6. Open MP 7. More Examples Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Germany 1 / 52
Big Data Analytics 1. Threads Basics Outline 1. Threads Basics 2. Starting and Interrupting Threads 3. Synchronization I: Monitors 4. Synchronization II: Locks 5. Starting Threads II: Thread Pools and Dependency Graphs 6. Open MP 7. More Examples Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Germany 1 / 52
Big Data Analytics 1. Threads Basics Processes and Threads I process: a running program I each process has its exclusive memory I managed by the operating system, heavy weight I distributed computing: running on different machines I multitasking: run several processes in parallel I OS switches between processes I multiprocessing: run in parallel on several processors Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Germany 1 / 52
Big Data Analytics 1. Threads Basics Processes and Threads I process: a running program I each process has its exclusive memory I managed by the operating system, heavy weight I distributed computing: running on different machines I may have several threads I multitasking: run several processes in parallel I OS switches between processes I multiprocessing: run in parallel on several processors I thread: a running subprogram I threads can share memory (shared address space) I managed by the program, light weight I multithreading: run several threads in parallel I may be switched between processors or run on several in parallel Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Germany 1 / 52
Big Data Analytics 1. Threads Basics Threads APIs I POSIX threads (Pthreads; IEEE standard 1995): C I C++ Standard library thread (2011): C++ I Java Standard library (1995, 2004): Java I Thread (java.lang; 1995) I Lock, ThreadPoolExecutor etc. (java.util.concurrent; 2004) I Open Multi-Processing (OpenMP; 1997): C, C++, Fortran I Java ports: JOMP, omp4j Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Germany 2 / 52
Big Data Analytics 1. Threads Basics Threads APIs I POSIX threads (Pthreads; IEEE standard 1995): C I C++ Standard library thread (2011): C++ I Java Standard library (1995, 2004): Java I Thread (java.lang; 1995) I Lock, ThreadPoolExecutor etc. (java.util.concurrent; 2004) I Open Multi-Processing (OpenMP; 1997): C, C++, Fortran I Java ports: JOMP, omp4j I Python Standard Library (2015; 3.5): Python I packages threading, multiprocessing, concurrent I default interpreter and jit compiler cpython and pypy: Global Interpreter Lock (GIL) I time-sliced execution on a single core only I the GIL can be released in python by many compilers such as cython, numba and possibly nuitka Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Germany 2 / 52
Big Data Analytics 2. Starting and Interrupting Threads Outline 1. Threads Basics 2. Starting and Interrupting Threads 3. Synchronization I: Monitors 4. Synchronization II: Locks 5. Starting Threads II: Thread Pools and Dependency Graphs 6. Open MP 7. More Examples Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Germany 3 / 52
Big Data Analytics 2. Starting and Interrupting Threads Starting Threads Runnable: I interface in java.lang I only method: void run(). I models a subprogram / procedure that can be run. Thread: I class in java.lang I constructor Thread(Runnable): a thread to run a given Runnable. I Thread.start(): begin to execute this thread. Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Germany 3 / 52
Big Data Analytics 2. Starting and Interrupting Threads Starting Threads / Example Output A C 1 public class HelloWorld3 implements Runnable { B 2 String msg; A 3 public HelloWorld3(String msg) { A 4 this .msg = msg; D 5 } . 6 public void run() { . . 7 while ( true ) 8 System.out. println (msg); 9 } 10 public static void main(String [] args ) { 11 new Thread(new HelloWorld3("A")).start(); 12 new Thread(new HelloWorld3("B")).start(); 13 new Thread(new HelloWorld3("C")).start(); 14 new Thread(new HelloWorld3("D")).start(); 15 } 16 } Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Germany 4 / 52
Big Data Analytics 2. Starting and Interrupting Threads Interrupting Threads I Thread.interrupt(): I set the thread’s interrupted property to true. I if the thread is sleeping or waiting, an InterruptedException will be thrown. I Thread.interrupted(): I get the value of the thread’s interrupted property. I Thread.isAlive(): I a thread is alive if it has been started, but not yet died. I Thread.currentThread() (static): I get the thread executing the current code. I Thread.sleep(long) (static): I the current thread sleeps for the given number of milliseconds. I There is no way to stop a thread externally. I There is no way to interrupt a thread that does not cooperate. Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Germany 5 / 52
Big Data Analytics 2. Starting and Interrupting Threads Interrupting Threads / Example / Example Computation 1 import java . util .∗; 2 3 public class Primes { 4 ArrayList <Long> primes = new ArrayList<>(); 5 6 public void compute(long max) { 7 primes .add(2L); 8 for (long n = 3; n < max; n = n+2) { 9 boolean isPrime = true; 10 for (Long prime: primes) 11 if (n % prime == 0) { 12 isPrime = false ; 13 break ; 14 } 15 if (isPrime) 16 primes .add(n); 17 } 18 } 19 } Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Germany 6 / 52
Big Data Analytics 2. Starting and Interrupting Threads Interrupting Threads / Example (non coop./broken) Output 1 import java . util .∗; 2 Work step 0 3 public class Worker implements Runnable { Work step 1 4 public void run() { Work step 2 5 for ( int i = 0; i < 1000; ++i) { interrupt 6 System.out. println ("Work step " + i ); Work step 3 7 new Primes().compute(100000); . . 8 } . 9 } 10 11 public static void main(String [] args ) { 12 Thread worker = new Thread(new Worker()); 13 worker. start (); 14 while (worker. isAlive ()) { 15 String input = System.console().readLine (); 16 if (input . equals(" interrupt ")) { 17 worker. interrupt (); 18 break ; 19 } 20 } 21 } 22 } Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Germany 7 / 52
Big Data Analytics 2. Starting and Interrupting Threads Interrupting Threads / Example (coop./fixed) Output 1 import java . util .∗; 2 Work step 0 3 public class Worker2 implements Runnable { Work step 1 4 public void run() { Work step 2 5 for ( int i = 0; i < 1000; ++i) { interrupt 6 if (Thread.currentThread (). isInterrupted ()) 7 break ; 8 System.out. println ("Work step " + i ); 9 new Primes().compute(100000); 10 } 11 } 12 13 public static void main(String [] args ) { 14 Thread worker = new Thread(new Worker2()); 15 worker. start (); 16 while (worker. isAlive ()) { 17 String input = System.console().readLine (); 18 if (input . equals(" interrupt ")) { 19 worker. interrupt (); 20 break ; 21 } 22 } 23 } 24 } Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Germany 8 / 52
Big Data Analytics 2. Starting and Interrupting Threads Interrupting Threads / Example (sleeping) 1 import java . util .∗; 2 3 public class Worker3 implements Runnable { 4 public void run() { 5 for ( int i = 0; i < 1000; ++i) { 6 System.out. println ("Work step " + i ); 7 try { 8 Thread.sleep (1000); 9 } catch ( InterruptedException ex) { 10 break ; 11 } 12 } 13 } 14 15 public static void main(String [] args ) { 16 Thread worker = new Thread(new Worker3()); 17 worker. start (); 18 while (worker. isAlive ()) { 19 String input = System.console().readLine (); 20 if (input . equals(" interrupt ")) { 21 worker. interrupt (); 22 break ; 23 } 24 } 25 } 26 } Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Germany 9 / 52
Big Data Analytics 3. Synchronization I: Monitors Outline 1. Threads Basics 2. Starting and Interrupting Threads 3. Synchronization I: Monitors 4. Synchronization II: Locks 5. Starting Threads II: Thread Pools and Dependency Graphs 6. Open MP 7. More Examples Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Germany 10 / 52
Big Data Analytics 3. Synchronization I: Monitors Synchronization I Several threads running in parallel may need to exchange some information. I can be accomplished through shared variables Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Germany 10 / 52
Big Data Analytics 3. Synchronization I: Monitors Synchronization I Several threads running in parallel may need to exchange some information. I can be accomplished through shared variables I Several threads running in parallel may need to coordinate, e.g., I a thread needs to wait until another is terminated I a thread requires exclusive access to some variable I e.g., to increment a counter or to edit an array I a thread requires some condition to hold to continue I e.g., further input in a stream being available Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Germany 10 / 52
Big Data Analytics 3. Synchronization I: Monitors Synchronization I Several threads running in parallel may need to exchange some information. I can be accomplished through shared variables I Several threads running in parallel may need to coordinate, e.g., I a thread needs to wait until another is terminated I a thread requires exclusive access to some variable I e.g., to increment a counter or to edit an array I a thread requires some condition to hold to continue I e.g., further input in a stream being available I Called synchronization. Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Germany 10 / 52
Hiển thị 20 trang đầu. Tải tài liệu để xem đầy đủ.
Chưa có bình luận nào. Hãy là người đầu tiên!
Bình luận (0)
Chưa có bình luận nào. Hãy là người đầu tiên!