LLaMA 3 Deep Dive! (Thomas Scialom - Meta)

Become a Patreon: https://www.patreon.com/theaiepiphany 👨‍👩‍👧‍👦 Join our Discord community: https://discord.gg/peBrCpheKE Thomas joined us for the second time to talk about their latest work: LLaMA 3! We cover synthetic data for pre/post training, why didn't they go with MoE, privacy (was it trained on Facebook user data?), and much more. ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ LLaMA 3: https://ai.meta.com/research/publications/the-llama-3-herd-of-models/ ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ ⌚️ Timetable: 00:00 - 00:27 Intro 00:27 - 02:08 Hyperstack GPUs platform! (sponsored) 02:08 - 06:40 What is new in new Llama? 06:40 - 13:30 Synthetic data 13:30 - 15:35 Privacy - training on Facebook user data? 15:35 - 19:10 Scaling and distillation 19:10 - 25:35 MoE, new architectures? 25:35 - 37:15 Upper boundary for the quality of SX data? 37:15 - 45:10 Context length 45:10 - 46:40 What framework does Meta use for Llama 46:40 - 51:20 Playing with smaller Llamas 51:20 - 53:20 Multilingual capabilities ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ 💰 SPONSOR The AI Epiphany - https://www.patreon.com/theaiepiphany One-time donation - https://www.paypal.com/paypalme/theaiepiphany Huge thank you to these AI Epiphany patreons: Eli Mahler Petar Veličković ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ 💼 LinkedIn - https://www.linkedin.com/in/aleksagordic/ 🐦 Twitter - https://twitter.com/gordic_aleksa 👨‍👩‍👧‍👦 Discord - https://discord.gg/peBrCpheKE 📺 YouTube - https://www.youtube.com/c/TheAIEpiphany/ 📚 Medium - https://gordicaleksa.medium.com/ 💻 GitHub - https://github.com/gordicaleksa 📢 AI Newsletter - https://aiepiphany.substack.com/ ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ #llama3 #llms #meta #opensource