AWS Sagemaker AI Crash Course-Customize and Deploy Any LLM
This video provides a complete end-to-end tutorial on fine-tuning, aligning, and deploying Large Language Models (LLMs) serverlessly using Amazon SageMaker AI. Here we demonstrates four levels of LLM customization—Prompt Engineering, Supervised Fine-Tuning (SFT) with LoRA, Direct Preference Optimization (DPO), and Reinforcement Learning (RLVR & RLAIF). Using real-world hands-on demonstrations, a base model is customized on legal contracts (Contract NLI dataset), evaluated against expert benchmarks, and served via an OpenAI-compatible API endpoint. Finally, the video covers model alignment using DPO, advanced reinforcement learning notebooks, and low-code customization workflows using an AI agent (Kiru CLI) and the SageMaker Studio UI. Workshop Link: fandf.co/4gOfvHD Code github: https://github.com/sourangshupal/sagemaker-model-customization-demo Timestamp: 00:00:00 - Introduction to LLM Customization and Deployment using Amazon SageMaker AI1 00:00:35 - Why Fine-Tuning is Necessary & Benefits of Serverless SageMaker AI2 00:01:48 - AWS Partnership Disclosure & Overview of Practical Demonstration3 00:02:45 - 4 Levels of LLM Customization (Prompt Engineering, SFT + LoRA, DPO, RL)4 00:03:50 - Key Features of Amazon SageMaker AI (JumpStart, Serverless, AI Agents)5 00:05:07 - Setting Up Amazon SageMaker Studio Domain & JupyterLab Environment6 00:07:22 - Cloning the GitHub Repository & Overview of Notebook Steps7 00:10:28 - Overview of Contract NLI Data Set (Stanford NLP)8 00:12:12 - Installing Dependencies & Setting Up SageMaker Session9 00:13:37 - Step 1: Formatting Data into JSONL Prompt-Completion Pairs10 00:18:40 - Step 2: Uploading Data to S3 and Registering in SageMaker AI Registry11 00:20:43 - Step 3: Setting Up Serverless Supervised Fine-Tuning (SFT with LoRA)12 00:25:28 - Reviewing Fine-Tuning Job Status & Metrics in MLflow13 00:28:10 - Step 4: Deploying Fine-Tuned Model & Base Qwen 3 4B Model Endpoints14 00:33:39 - Evaluating Side-by-Side Performance: Base Model vs. Fine-Tuned Model15 00:38:43 - Calculating Accuracy & Evidence F1 Scores via Automated Scorer16 00:41:13 - Step 5: Inferencing via Production OpenAI-Compatible API Stream17 00:44:26 - Tearing Down Deployed Resources & Endpoints to Avoid Charges18 00:45:20 - Step 6: Direct Preference Optimization (DPO) Setup & Data Preparation19 00:49:45 - Launching & Monitoring the Serverless DPO Training Job20 00:53:27 - Deploying the DPO Model & Evaluating Casual Human-like Tone Response21 00:55:54 - Tearing Down DPO Deployment Resources22 00:56:05 - Exploring Additional RLVR & RLAIF Workshop Notebooks23 00:58:17 - Demonstration: Customizing LLMs using AI Agent-Guided Workflow (Kiru CLI)24 01:02:38 - Demonstration: Customizing LLMs via SageMaker Studio UI Interface25 01:03:59 - Final Recap, Workshop Link Details, and Conclusion
