First, what is an LLM?
LLM stands for Large Language Model. Let's break it down: We can describe it as a system that takes input, performs some operations on it, and gives a generated text output. This is what an LLM does; it sits between the input and the output.
Some of the most popular LLMs are OpenAI's GPT, Google's Gemini, Anthropic's Claude, and Meta's Llama. We use common applications like ChatGPT or Claude in our day-to-day life to complete various tasks.
What is GPT?
GPT - Generative Pre-trained Transformer GPT is a transformer that generates output based on some pre-trained data. It takes text as input, which is broken down into tokens, and the LLM responds with the next token prediction based on the input. This happens in a loop until a natural stop occurs.
WHAT HAPPENS WHEN YOU SEND A MESSAGE TO GPT?
Everything starts with a prompt, which is a message or instruction we give to ChatGPT. It is the input we provide to any LLM. In technical terms, this input is known as tokens.
Example: "best gen ai cohort available out there"
This input undergoes the following process to give an output
This is the entire workflow.
Why responses are not copied from the internet
The responses are not copied from the internet because every token is generated by the transformer based on some pre-trained data, using the context of the input text.
WHY A COMPUTER DOESN'T UNDERSTAND HUMAN LANGUAGE?
It is as simple as you think, because we can't communicate with a person who doesn't understand our language.
Computers only understand 0s and 1s and can only work based on the specific rules they are programmed with. That is the reason text must be converted to numbers so that a computer can perform operations on it.
This brings us closer to the concept of a token.
Token - It is a number given to a word or character in any given input to an LLM.
Tokenization
A token is a number assigned to the words present in the input text. These numbers are taken from the vocabulary of the particular GPT. There is always a unique vocabulary set for each GPT.
Transformer
The transformer is the brain behind this big tech, where the entire process of predicting the next token based on the present tokens happens.
-
Vector embedding - It is a way of mapping tokens on a 3D plane so that we can find similar-meaning tokens nearby.
-
Positional encoding - Based on the position of the token and the way they are placed, the meaning can change. Positional encoding makes vector embeddings very meaningful according to the position of the token.
-
Self-attention - In this step, the tokens can talk to each other so that they can become more aware of the input's context.
-
Feed-forward - The main neural network runs here, which returns the probability distribution of the next token.
-
Softmax or temperature - These are the values used to pick the next token, which brings randomness to the next token prediction.
Bonus
Transformers have 2 phases:
-
Training - A looping process where it takes the input and gives an output. Based on the difference between the actual output and the model's output, the model gets updated with new values. It runs until the correct output is returned.
-
Inferencing - Talking to the model. Here, the learning loop doesn't work; we only get the output for the provided input. Updating the values is not done.
The Conclusion
So this is under the hood: AI or GPT doesn't know words, but they can talk using these mathematical processes.