Our great sponsors
-
haystack
:mag: LLM orchestration framework to build customizable, production-ready LLM applications. Connect components (models, vector DBs, file converters) to pipelines or agents that can interact with your data. With advanced retrieval methods, it's best suited for building RAG, question answering, semantic search or conversational agent chatbots.
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
Depends on what exactly you are trying to do. If you are trying open-domain QA, you have a bigger issue that you may need to interact with millions of documents - where each document can have 100s of pages. For that, you can try the standard techniques of having an IR mechanism to retrieve relevant documents and then some form of retrieval augmented generation with GPT (https://github.com/deepset-ai/haystack). Now for multi-hop reasoning, you may also need a kind of feedback loop where you want to interact with IR multiple times based on retrieved content from previous turn. So there is already that level of complexity. For that probably you can build from self-ask. The other level of difficulty is - say you have retrieved top-k documents, now what? What if each document has 100s of unseen pages? One thing you can do here is just put it all in. I am not sure about GPT exactly, but there are Transformer models that can model long context with Transformer XL or some build-up on that eg. block recurrent transformer. That may require some fine-tuning. The other thing you can do is use another IR, to retrieve the most relevant content (eg. Top-k passages) WITHIN the retrieved document page (and keep on spamming IR hierarchally until you truncate down the content from prompt satisfactorily). These may be doable without fine tuning. You can probably engineer some way to combine the different passages in a single prompt. You can play around or think about more sophisticated strategies (1, 2) but those strategies may require architectural modifications and additional training.