Introduction of Generative AI and Large Langue Models Prof. Ching-Yung Lin Nov 10, 2023 Overview of Large Language Models THE EVOLUTION OF NATURAL LANGUAGE PROCESSING EECS E6893 BIG DATA ANALYTICS 2 The Evolution of LLMs 1. In 2017, Google released the "Transformer Model", which can be used in question-answering systems, reading comprehension, sentiment analysis, instant translation of text or speech, and more 2. In 2018, OpenAI proposed "GPT" and Google proposed the "BERT" model, widely used in search engines, speech recognition, machine translation, questionanswering systems, and more. 3. From 2018 to 2022, most of the research focused on BERT-related algorithms, when GPT performance was inferior to BERT 4. Text Image In 2023, ChatGPT (GPT3.5) was proposed by OpenAI, which significantly improves NLU's ability to understand most texts and surpasses humans in some area In NLU CNN RNN Self-Attention Local feature Front and Back Dependency Issues One to all attention, more flexible and trainable need large datasets EECS E6893 BIG DATA ANALYTICS 3 The speed of development of Generative AI EECS E6893 BIG DATA ANALYTICS 4 Generative AI Basics CREATING ARTIFICIAL CREATIVITY EECS E6893 BIG DATA ANALYTICS 5 Generative AI Application Condition Model Generative Model Multi-Model NLU + Image Generator Conditional image Generator Image Generator Audio Generator Speech Generator NLU + Robot Chat Bot text Generator Summarization and Translation Pose Generator Robot EECS E6893 BIG DATA ANALYTICS 6 Generative AI Methodology Variational AutoEncoder (VAE) Diffusion Denoise Model Training Generating Generative Adversarial Network (GAN) Large Language Model (LLM) EECS E6893 BIG DATA ANALYTICS 7 What is Generative AI EECS E6893 BIG DATA ANALYTICS 8 Multimedia Generation EECS E6893 BIG DATA ANALYTICS 9 Generating Text using Large Language Models EECS E6893 BIG DATA ANALYTICS 10 How LLM works EECS E6893 BIG DATA ANALYTICS 11