OpenAI CLIP | Machine Learning Coding Series
❤️ Become The AI Epiphany Patreon ❤️ https://www.patreon.com/theaiepiphany 👨👩👧👦 Join our Discord community 👨👩👧👦 https://discord.gg/peBrCpheKE Kicking off a series of videos where I'll be going through the actual code of many of the papers I've covered over the last few years! In this video I do a code walkthrough of OpenAI's CLIP model from the "Learning Transferable Visual Models From Natural Language Supervision" paper. Let me know what you'd like me to cover next! ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ ✅ GitHub: https://github.com/openai/CLIP ✅ Paper: https://arxiv.org/abs/2103.00020 Learn about Byte-Pair Encoding: ✅ https://en.wikipedia.org/wiki/Byte_pair_encoding ✅ https://leimao.github.io/blog/Byte-Pair-Encoding/ ✅ Video: https://www.youtube.com/watch?v=tOMjTCO0htA&ab_channel=FromLanguagestoInformation ✅ https://www.youtube.com/watch?v=zjaRNfvNMTs&ab_channel=AbhishekThakur Unicode: ✅ https://dmitripavlutin.com/what-every-javascript-developer-should-know-about-unicode/ ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ ⌚️ Timetable: 00:00:00 Intro 00:02:00 High level overview: Interacting with CLIP 00:26:11 High level overview: Prompt engineering for ImageNet 00:40:25 Deep dive starts: vocabulary and byte-pair encoding 00:49:00 Vision Transformer & Text Transformer explained 01:02:00 Tokenization walkthrough 01:09:25 Encoding the image 01:15:15 Encoding the text 01:23:15 Learning a linear probe 01:27:00 Tokenization of the (brain emoji) 01:29:56 Outro ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ 💰 BECOME A PATREON OF THE AI EPIPHANY ❤️ If these videos, GitHub projects, and blogs help you, consider helping me out by supporting me on Patreon! The AI Epiphany - https://www.patreon.com/theaiepiphany One-time donation - https://www.paypal.com/paypalme/theaiepiphany Huge thank you to these AI Epiphany patreons: Eli Mahler Kevin Stone Petar Veličković ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ 💼 LinkedIn - https://www.linkedin.com/in/aleksagordic/ 🐦 Twitter - https://twitter.com/gordic_aleksa 👨👩👧👦 Discord - https://discord.gg/peBrCpheKE 📺 YouTube - https://www.youtube.com/c/TheAIEpiphany/ 📚 Medium - https://gordicaleksa.medium.com/ 💻 GitHub - https://github.com/gordicaleksa 📢 AI Newsletter - https://aiepiphany.substack.com/ ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ #CLIP #contrastive #codewalkthrough
