Large Language Models (LLMs)¶

Welcome to the Large Language Models section of the AI Engineering Academy. This module provides a comprehensive understanding of LLMs and their practical applications in AI engineering.

Repository Structure¶

Category	Topic	Resource
Introduction	Overview	Introduction to LLMs
Theory Behind Fine-tuning	Pre-Training	Pre-Training
	Supervised Fine-Tuning (SFT)	SFT Theory
	Proximal Policy Optimization (PPO)	PPO Theory
	Direct Preference Optimization (DPO)	DPO Theory
	Observation-Regularized Policy Optimization (ORPO)	ORPO Theory
	Gated Regularized Policy Optimization (GRPO)	GRPO Theory
Hands-On SFT	Overview	SFT Implementation Guide
	Implementation	SFT Notebook
Hands-On GRPO	Guide	Hacker Guide to GRPO
	Implementation	Qwen 0.5B GRPO
Gemma	Overview	Gemma Guide
	Implementation	Gemma Fine-tuning
Llama2	Overview	Llama2 Guide
	Implementation	Llama2 Fine-tuning
	Advanced	QLora Fine-tuning
Llama3	Implementation	Llama3 Fine-tuning
Mistral-7B	Overview	Mistral Guide
	Implementation	Mistral Fine-tuning
	Evaluation	Evaluation Harness
	DPO	DPO Fine-tuning
	SFT	SFT Trainer
	Inference	ChatML Inference
Mixtral	Implementation	Mixtral Fine-tuning
Visual Language Models	Florence2	Florence2 Fine-tuning
	PaliGemma	PaliGemma Fine-tuning
Architecture	Parameter Analysis	Parameter Count

Learning Roadmap¶

Level	Steps	Resources
Beginner	1. Introduction to LLMs	Introduction
	2. Understanding core theory	Pre-Training, SFT Theory
	3. First implementation	SFT Guide
	4. Practical application	Llama2 Fine-tuning
Intermediate	1. Advanced techniques	DPO Theory, PPO Theory
	2. Model implementation	Mistral Fine-tuning
	3. Architecture concepts	Parameter Count
Advanced	1. Cutting-edge methods	ORPO Theory, GRPO Theory
	2. Advanced implementation	GRPO Implementation
	3. Multimodal models	Florence2, PaliGemma

Contributing¶

We welcome contributions to expand this repository. Please follow the standard pull request process and ensure your contributions align with the overall structure.

License¶

This project is licensed under the MIT License - see the LICENSE file for details.

---

AI Engineering Academy - Advancing the frontier of language model understanding and implementation