Spaces:
Runtime error
Langchain
Thanks for the example, I was looking for something like this.
I saw that you are creating a Chain module, did you consider using Langchain? works pretty well with chainlit
Yeah, we've used Langchain before and it works fine, but there's often quite a bit of boilerplate that is needed for simple tasks--e.g., you need to instantiate a Prompt and a Chain to send a single LLM message. Langchain has some value for more complex tasks, but these have their own issues as well--here are some off the top of my head:
- Embedding vector databases: Most of the wrappers provided by Langchain don't leverage the full capabilities of each database backend (e.g., embedding searches with filter queries, persisting in-memory embeddings) because the functionality doesn't fit into Langchain's interfaces or because Langchain's wrappers are not always up-to-date. The Langchain package is constantly being updated to try to catch up to all the backends it tries to support, but those constant updates are an issue in themselves.
- Memory: Works well until you start needing to decouple the chain input/output from the values saved in memory. E.g., with Chainlit, it's possible to output additional text/actions with each message, but how should those map to the memory? It may take some custom handling beyond the default memory provided by Langchain.
- Agents/Executors: Langchain has some decent prompts for these, but because the tasks are so complex (and our current LLMs are quite finicky), it may be necessary to customize these prompts, or even to change the executor behavior (e.g. to better handle exceptions, to handle cases where the agent output has the wrong format, or to completely change the agent output format)
Since Langchain is designed quite decently, it's possible to customize each of these aspects via subclassing, but when we get to the point that we're subclassing every single class, there really isn't much reason to use Langchain--we might as well just build a custom framework that's more tailored to our needs and requires less boilerplate... And we can see with the custom chain module that it really doesn't take that much code to build a basic and fairly-pythonic framework that also integrates well with Chainlit.
You are right, I'm kinda a fan of Langchain, but I saw all the problems that you mention.
I found that working with chains instead of agents could give you more control of the input/outputs.
But all you said is true, I struggled with the memory, with the vector stores, and many more problems.
In my case, I'm using boilerplates with langchain, but it's for fast development, maybe in some productive development I should not use langchain
dumb question but how did you get the history tab/button on the space? I can't see where you do anything that causes it in the files, and I don't see a modified chainlit package loaded. Seems like magic :-)