Streaming for LangChain Agents + FastAPI
In chapter 10 of the LangChain series we'll work from LangChain streaming 101 through to developing streaming for LangChain Agents and serving it through FastAPI. With what we cover here, you'll be able to go from never having used streaming to deploying it in production in no time. We'll focus on using OpenAI's GPT-3.5-turbo model via LangChain's ChatOpenAI object. Learning how to do simple terminal (StdOut) streaming with LLMs, up to parsing stream outputs with Async Iterator streaming. 📌 Code notebook: https://github.com/pinecone-io/examples/blob/master/learn/generation/langchain/handbook/09-langchain-streaming/09-langchain-streaming.ipynb 📌 FastAPI template code: https://github.com/pinecone-io/examples/blob/master/learn/generation/langchain/handbook/09-langchain-streaming/main.py 🌲 Subscribe for Latest Articles and Videos: https://www.pinecone.io/newsletter-signup/ 👋🏼 AI Development: https://aurelio.ai 👾 Discord: https://discord.gg/c5QtDB9RAP Twitter: https://twitter.com/jamescalam LinkedIn: https://www.linkedin.com/in/jamescalam/ 00:00 Streaming for LLMs and Agents 01:38 Simple StdOut Streaming in LangChain 05:03 Streaming with LangChain Agents 07:56 Final Output Streaming 10:02 Custom Callback Handlers in LangChain 14:16 FastAPI with LangChain Agent Streaming 22:56 Confirming we have Agent Streaming 24:32 Custom Callback Handlers for Async 26:26 Final Things to Consider #artificialintelligence #langchain #openai #chatgpt #nlp #deeplearning
