fast_vector_similarity
swiss_army_llama
fast_vector_similarity | swiss_army_llama | |
---|---|---|
7 | 11 | |
323 | 878 | |
- | - | |
7.2 | 8.8 | |
9 months ago | 4 days ago | |
Rust | Python | |
- | - |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
fast_vector_similarity
-
SentenceTransformers: Python framework for sentence, text and image embeddings
Yes, check out my library for vector similarity that has various other measures which are more discriminative:
https://github.com/Dicklesworthstone/fast_vector_similarity
pip install fast_vector_similarity
-
Show HN: Neum AI – Open-source large-scale RAG framework
Got it. I'd encourage you to expose more of that functionality at the level of your application if possible. I think there is a lot of potential in using more than just cosine similarity, especially when there are lots of candidates and you really want to sharpen up the top few recommendations to the best ones. You might find this open-source library I made recently useful for that:
https://github.com/Dicklesworthstone/fast_vector_similarity
I've had good results from starting with cosine similarity (using FAISS) and then "enriching" the top results from that with more sophisticated measures of similarity from my library to get the final ranking.
-
Some Reasons to Avoid Cython
You can see how I did something similar in my library here:
https://github.com/Dicklesworthstone/fast_vector_similarity/...
Basically you use ndarray instead of numpy, try to vectorize anything you can, and for the for loops that can’t be vectorized, you can use rayon to do them in parallel.
- FLaNK Stack Weekly 28 August 2023
- Fast Vector Similarity Library, Useful for Working With Llama2 Embedding Vectors
-
Show HN: Fast Vector Similarity Using Rust and Python
Yeah, like the other commenter said, everything is in this file here:
https://github.com/Dicklesworthstone/fast_vector_similarity/...
If you also make your project using Rust and Maturin, you can literally just copy and paste that into your project because it's totally generic, and if the repo is public, GitHub will just run it all for you for free.
The only thing is you need to create an account on PyPi (pip) and add 2-Factor Auth so you can generate an API key. Then you go into the repo settings and go to secrets, and create a Github Actions secret with the name PYPI_API_TOKEN and make the value your PyPi token. That's it! It will not only compile all the wheels for you but even upload the project to PyPi for you using the settings found in your pyproject.toml file, like this:
https://github.com/Dicklesworthstone/fast_vector_similarity/...
swiss_army_llama
-
Ask HN: Cheapest way to run local LLMs?
Depends what you mean by "local". If you mean in your own home, then there isn't a particularly cheap way unless you have a decent spare machine. If you mean "I get to control everything myself" then you can rent a cheap VPS on a value host like Contabo (you can get 8cores, 30gb of ram, and 1tb SSD on Ubuntu 22.04 for something like $35/month-- just stick the to US data centers).
Then if you want something that is extremely quick and easy to set up and provides a convenient REST api for completions/embeddings with some other nice features, you might want to check out my project here:
https://github.com/Dicklesworthstone/swiss_army_llama
Especially if you use Docker to set it up, you can go from a brand new box to a working setup in under 20 minutes and then access it via the Swagger page from any browser.
-
What's the difference between LangChain, llama indexand others like autollm?
I found all of them to be quite bloated and annoying to use directly, which is why I made my own FastAPI based one, Swiss Army Llama. I’m obviously biased, but I far prefer it:
https://github.com/Dicklesworthstone/swiss_army_llama
- Show HN: Swiss Army Llama – A Versatile, FastAPI-Based Multitool for Local LLMs
-
Show HN: Swiss Army Llama
I just added a very cool feature that lets you supply a sample JSON file and it will automatically generate a BNF grammar for it. You can also supply a pydantic data model description and it will generate the corresponding JSON BNF for you:
https://github.com/Dicklesworthstone/swiss_army_llama/blob/m...
And then you can add that grammar file and it will validate it with this:
https://github.com/Dicklesworthstone/swiss_army_llama/blob/5...
-
Show HN: Fast Vector Similarity Using Rust and Python
Cool, I also made a similar kind of tool recently that I also shared on HN a couple weeks ago. You might find it useful for generating and managing LLM embeddings locally:
https://github.com/Dicklesworthstone/llama_embeddings_fastap...
-
Show Show HN: Llama2 Embeddings FastAPI Server
Thanks for pointing out those models. I see from a quick Huggingface search that the bge model is available in GGML format. You can trivially add new GGML format models to the code by simply adding the direct download link to this line:
https://github.com/Dicklesworthstone/llama_embeddings_fastap...
So to add the base bge model, you could just add this URL to the list:
https://huggingface.co/maikaarda/bge-base-en-ggml/resolve/ma...
I will add that as an additional default.
- Llama2 Embeddings FastAPI Service
- Show HN: LLama2 Embeddings API Service Made with FastAPI
What are some alternatives?
simsimd
llama_embeddings_fastap
np-sims - numpy ufuncs for vector similarity
openembeddings - Self-hostable pay for what you use embedding server for bge-large-en and arbitrary embedding models using crypto