Gen AI: Going Beyond Determinism
The power of Gen AI comes from significant advancement in artificial intelligence that differentiates itself from traditional deterministic models
To gain a clear understanding of any concept, it is essential to understand the fundamental building blocks. The term ‘AI’ is being thrown around everything related to tech or tech-adjacent. This is no different from surge in popularity around machine learning or block chain experienced a few years ago. Therefore, let’s take a closer look and define the key distinctions between AI, Machine Learning, Deep Learning and Data Science.
Artificial Intelligence (AI): The fundamental idea with AI is to create systems or machines capable of performing tasks that typically require human intelligence. That could be recognizing patterns, understanding natural language, problem-solving and decision-making.
Let’s use a running analogy to explain the concept of AI: Think of the marathon runner as the “Artificial Intelligence” in this analogy. The runner is like the brain of the operation, making decisions and orchestrating the entire training process. The runner adapts to terrains and weather conditions, to achieve specific goals. If it’s too hot and humid, the runner would make a decision to slow down. If it’s downhill, the runner can potentially adapt and pick up the speed. AI, in a similar way, is the overarching concept that involves making real time decisions and taking actions based on information and learning.
Machine Learning (ML): Machine learning is a subfield of AI with a focus on teaching machines to learn from data and improve their performance over time. It involves developing algorithms and models that can make predictions or decisions without being explicitly programmed. Machine learning is the driving force behind many AI applications, from recommendation systems to image recognition.
Using the same running analogy, consider that the runner follows a training plan that includes a mix of workouts, distances, and paces to reach a specific goal, call it as ‘X’. As the runner logs their training data, they can use this information to track their performance against ‘X’ goal, and adjust their future training sessions accordingly. This is same as “Machine Learning” - the runner is learning from their training data to optimize their future performance.
Deep Learning: Deep learning is a class of machine algorithms that focuses on artificial neural networks, particularly deep neural networks with multiple layers (hence the term "deep"). These neural networks attempt to mimic the structure and function of the human brain by processing and learning from vast amounts of data. Deep learning has gained significant attention and success in tasks like image and speech recognition, natural language processing, and recommendation systems.
Back to our running analogy, Deep learning is like a relay race where a team of runners, representing layers in a neural network, passes information (the baton) to achieve a common goal. Each layer processes data, specializes in features, and contributes to the overall task's accuracy, much like runners in a relay race optimize their performance. Just as deep learning involves neural networks with multiple layers, a relay race may have multiple runners, each contributing to the overall performance. The depth of the neural network corresponds to the number of layers in the relay team.
Data Science: Data science is an interdisciplinary field that combines domain knowledge, programming skills, and statistical analysis to extract meaningful insights from data. Data Analysts and scientists collect, clean, and analyze data to inform decision-making, often using machine learning techniques. Data science plays a crucial role in AI development.
Here, the "Data Analyst/ Scientist" is like the coach or trainer who designs the training program for the runner. The data scientist collects data on the runner's performance, tracks their progress, and analyzes factors like nutrition, sleep, and weather conditions that may affect their performance. Data science is about managing and interpreting the data to help the runner (AI) make informed decisions and improve their running.
Back to our running analogy, the runner represents the AI, training plan represents ML, and the Coach or trainer leverages data science to power meaningful insights. In short, AI is a discipline, ML & Deep Learning are subfields in AI, and Data Sciences is an overlapping field that plays a critical role in AI development.
So, what is Gen AI?
Generative modeling is a branch of machine learning that involves training a model to produce new data that is similar to a given dataset - David Foster, author of “Generative Deep Learning”
The key here is generating new data. Your traditional deterministic models including deep learning are primarily focused on learning from input data and making either predictions or classifying the new data. Whereas, Gen AI generates entirely new content based on patterns it has learned.
Discriminative Models
In the above example, we are trying to predict if a given image is pained by Van Gosh. This is a classic deterministic model example, where we have an input dataset with observations that has labels, 1 being painted by Van Gosh and 0 as non- Van Gosh. As the new data comes in, our model provides a probability of whether the observation is 1 or 0.
Whereas, Generative Model doesn’t need data to be labeled because the core purpose of generative models is to output entirely new images, rather than predicting a label of a given image.
In this example, Generative model captures the distribution and rules that govern the relationships between pixels in images of horses. So, it must be probabilistic rather than deterministic, because we want to produce many variations of the output, instead of just classification. The goal of Generative models is to mimic the distribution of training dataset as closely as possible and then sample from it to generate new dataset that looks as if they were present in training dataset.
With Generative models, creativity and novelty becomes the core selling point versus traditional deterministic models. Similar to ML, it also requires less human intervention once the model is trained, and has high adaptability to different data inputs.
I hope that this explanation goes beyond the common buzzwords, providing a deeper understanding.
https://www.oreilly.com/library/view/generative-deep-learning/9781098134174/ & AI Co Pilot