Shape of everything inside a transformer

While learning about GPT-2 architecture I constantly come across the same issue - shapes, dimensions and every further problems related to that. The purpose of this article is to go through the whole GPT model and thoroughly think of any dimension and its purpose

Everything about tokenization

• • Part 1. Beginners understanding
• • Part 2. Tokenizer types

"Vibe coding" notes

Everyone wonders how the world is going to look like after emerging of LLMs. My main goal was to check how good LLM can follow instructions, how well can make tech decisions, what are the main limitations and what is a human part in such type of coding.