private-gpt VS PrivateGPT4Linux

PrivateGPT is a nice tool for this. It's not exactly what you're asking for, but it gets part of the way there.

https://github.com/zylon-ai/private-gpt

Further details available at: https://docs.privategpt.dev/api-reference/api-reference/ingestion

Run https://github.com/imartinez/privateGPT

Then

make ingest /path/to/folder/with/files

Then chat to the LLM.

Done.

Docs: https://docs.privategpt.dev/overview/welcome/quickstart

PrivateGPT repository in case anyone's interested: https://github.com/imartinez/privateGPT . It doesn't seem to be linked from their official website.

I’m preparing a small internal tool for my work to search documents and provide answers (with references), I’m thinking of using GPT4All [0], Danswer [1] and/or privateGPT [2].

The RAG technique is very close to what I have in mind, but I don’t want the LLM to “hallucinate” and generate answers on its own by synthesizing the source documents. As stated by many others, we’re living in interesting times.

[0] https://gpt4all.io/index.html

[1] https://www.danswer.ai/

[2] https://github.com/imartinez/privateGPT

https://www.reddit.com/r/LocalLLaMA/comments/14niv66/using_a...

https://github.com/imartinez/privateGPT

I've been playing around with https://github.com/imartinez/privateGPT and https://github.com/simonw/llm and wanted to create a simple Python package that made it easier to run ChatGPT-like LLMs on your own machine, use them with non-public data, and integrate them into practical applications.

This resulted in Python package I call OnPrem.LLM.

In the documentation, there are examples for how to use it for information extraction, text generation, retrieval-augmented generation (i.e., chatting with documents on your computer), and text-to-code generation: https://amaiya.github.io/onprem/

Enjoy!

I found things like this dataset and LocalAI and I followed the article to get PrivateGPT and the GPT4ALL groovy.bin but I'm completely lost and it feels like the more I research the internet or ask BingAI for answers, the more questions I get instead. At this stage I don't know what goes where, if there's a difference between source documents and datasets, should I run this from my 2tb SSD? Should I have the data on my 8tb HDD? Will all this even work on my PC?

No, as the same as the last question, It does not have access to anything except the model data itself. However, there are some approaches that can let LLMs have access LOCAL documents, which means if you can have a program that extracts data from the database into a local folder which contains TEXT files. This could also work for 2(I didn't mention it in 2 because online datas are REALLY big. It would take the model hours to give an answer. If the database is not large then there might be a shot. Check https://github.com/imartinez/privateGPT(Must be GPT4all compatible models sadly).

I've been doing exactly this with an open source repository called PrivateGPT imartinez/privateGPT: Interact privately with your documents using the power of GPT, 100% privately, no data leaks (github.com)

Not really, you will probably need to change the data location and the LLM provider in the example code to get it running. But you don't have to implement that yourself there are a couple projects that already do that like privateGPT. I use it for searching datasheets, got it up an running in a few hours and I'm pretty happy with it so far.

PrivateGPT can do that.