How to Build Data Pipelines for ML Projects (w/ Python Code)
🤝 Your team not maximizing Claude? I run 1:1 and team AI workshops for companies doing $10M+ per year: https://aibuilder.academy/yt/OnIQrDiTtRM This is the 3rd video in a series on Full Stack Data Science. Here, discuss key aspects of building data pipelines for machine learning and share Python code for pulling transcripts from all my YouTube videos. 🎥 Series Playlist: https://www.youtube.com/playlist?list=PLz-ep5RbHosWmAt-AMK0MBgh3GeSvbCmL More Resources: 📰 Read more: https://medium.com/towards-data-science/how-to-build-data-pipelines-for-machine-learning-b97bbef050a5?sk=4823c18cab0a6225b0be8773c5427704 💻 Example Code: https://github.com/ShawhinT/YouTube-Blog/tree/main/full-stack-data-science/data-engineering References: [1] How Data Engineering Works: https://www.youtube.com/watch?v=qWru-b6m030 [2] ETL vs ELT: https://aws.amazon.com/compare/the-difference-between-etl-and-elt/ [3] YouTube Search API: https://developers.google.com/youtube/v3/docs/search Introduction - 0:00 Data Engineering - 0:34 Data Pipelines - 1:19 2 Types of Pipelines (ETL vs ELT) - 2:18 Extract - 4:30 Transform - 6:07 Load - 7:22 Orchestration - 9:06 Example Code: ETL of My YouTube Video Transcripts - 10:47 What's Next? - 21:34
