transformers
faiss
Our great sponsors
transformers | faiss | |
---|---|---|
171 | 69 | |
122,577 | 27,545 | |
2.7% | 3.7% | |
10.0 | 9.4 | |
7 days ago | 6 days ago | |
Python | C++ | |
Apache License 2.0 | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
transformers
-
Gemma doesn't suck anymore – 8 bug fixes
Thanks! :) I'm pushing them into transformers, pytorch-gemma and collabing with the Gemma team to resolve all the issues :)
The RoPE fix should already be in transformers 4.38.2: https://github.com/huggingface/transformers/pull/29285
My main PR for transformers which fixes most of the issues (some still left): https://github.com/huggingface/transformers/pull/29402
-
Paris-Based Startup and OpenAI Competitor Mistral AI Valued at $2B
If you want to tinker with the architecture Hugging Face has a FOSS implementation in transformers: https://github.com/huggingface/transformers/blob/main/src/tr...
If you want to reproduce the training pipeline, you couldn't do that even if you wanted to because you don't have access to thousands of A100s.
-
[D] What is a good way to maintain code readability and code quality while scaling up complexity in libraries like Hugging Face?
In transformers, they tried really hard to have a single function or method to deal with both self and cross attention mechanisms, masking, positional and relative encodings, interpolation etc. While it allows a user to use the same function/method for any model, it has led to severe parameter bloat. Just compare the original implementation of llama by FAIR with the implementation by HF to get an idea.
-
Self train a super tiny model recommendations
You can train it with the code provided in transformer repo: https://github.com/huggingface/transformers/blob/main/examples/pytorch/language-modeling/run_clm.py
-
Can we discuss MLOps, Deployment, Optimizations, and Speed?
transformers uses accelerate if you call it with device_map='auto'
-
Show HN: Phind Model beats GPT-4 at coding, with GPT-3.5 speed and 16k context
Too much money being thrown around on BS in the LLM space, hardly any of it is going to places where it matters.
For example, the researchers working hard on better text sampling techniques, or on better constraint techniques (i.e. like this https://arxiv.org/abs/2306.03081), or on actual negative prompting/CFG in LLMs (i.e. like this https://github.com/huggingface/transformers/issues/24536) are doing far FAR more to advance the state of AI than dozens of VC backed LLM "prompt engineering" companies operating today.
HN, and the NLP community have some serious blindspots with knowing how to exploit their own technology. At least someone at Anderson Howartz got a clue and gave some funding to Oogabooga - still waiting for Automatic1111 to get any funding.
-
🐍🐍 23 issues to grow yourself as an exceptional open-source Python expert 🧑💻 🥇
Repo : https://github.com/huggingface/transformers
-
Whisper prompt tuning
From what I know, Whisper already supports prompting (https://github.com/huggingface/transformers/pull/22496). Can I somehow freeze the whole model and tune exclusively the prompt or would I need to write an implementation from scratch?
-
A look at Apple’s new Transformer-powered predictive text model
https://github.com/huggingface/transformers/blob/0a55d9f7376...
To summarize how they work: you keep some number of previously generated tokens, and once you get logits that you want to sample a new token from, you find the logits for existing tokens and multiply them by a penalty, thus lowering the probability of the corresponding tokens.
-
Can LLMs learn from a single example?
Very cool. This came up in a huggingface transformers issue a while ago and we also determined memorization to be the likely reason. It's nice to see someone else reach the same conclusion.
faiss
-
You Shouldn't Invest in Vector Databases?
You can try txtai (https://github.com/neuml/txtai) with a Faiss backend.
This Faiss wiki article might help (https://github.com/facebookresearch/faiss/wiki/Indexing-1G-v...).
For example, a partial Faiss configuration with 4-bit PQ quantization and only using 5% of the data to train an IVF index is shown below.
faiss={"components": "IVF,PQ384x4fs", "sample": 0.05}
-
Approximate Nearest Neighbors Oh Yeah
If you want to experiment with vector stores, you can do that locally with something like faiss which has good platform support: https://github.com/facebookresearch/faiss
Doing full retrieval-augmented generation (RAG) and getting LLMs to interpret the results has more steps but you get a lot of flexibility, and there's no standard best-practice. When you use a vector DB you get the most similar texts back (or an index integer in the case of faiss), you then feed those to an LLM like a normal prompt.
The codifer for the RAG workflow is LangChain, but their demo is substantially more complex and harder-to-use than even a homegrown implementation: https://news.ycombinator.com/item?id=36725982
-
Can someone please help me with this problem?
According to this documentation page, faiss-gpu is only supported on Linux, not on Windows.
-
Code Search with Vector Embeddings: A Transformer's Approach
As the size of the codebase grows, storing and searching through embeddings in memory becomes inefficient. This is where vector databases come into play. Tools like Milvus, Faiss, and others are designed to handle large-scale vector data and provide efficient similarity search capabilities. I've wrtten about how to also use sqlite to store vector embeddings. By integrating a vector database, you can scale your code search tool to handle much larger codebases without compromising on search speed.
-
Unum: Vector Search engine in a single file
But FAISS has their own version ("FastScan") https://github.com/facebookresearch/faiss/wiki/Fast-accumula...
- Introduction to Vector Similarity Search
-
Vector Databases 101
If you want to go larger you could still use some simple setup in conjunction with faiss, annoy or hnsw.
- I'm an undergraduate data science intern and trying to run kmodes clustering. Did this elbow method to figure out how many clusters to use, but I don't really see an "elbow". Tips on number of clusters?
-
Disrupting the AI Scene with Open Source and Open Innovation
Facebook research for having open source licensed an amazing vector based indexing library
-
Meilisearch across the Semantic Verse
I just used huggingface sentence transformer to compute the embeddings and FAISS to get the nearest entries. This is very close to this tutorial on huggingface.
What are some alternatives?
annoy - Approximate Nearest Neighbors in C++/Python optimized for memory usage and loading/saving to disk
Milvus - A cloud-native vector database, storage for next generation AI applications
hnswlib - Header-only C++/python library for fast approximate nearest neighbors
fairseq - Facebook AI Research Sequence-to-Sequence Toolkit written in Python.
pgvector - Open-source vector similarity search for Postgres
Weaviate - Weaviate is an open-source vector database that stores both objects and vectors, allowing for the combination of vector search with structured filtering with the fault tolerance and scalability of a cloud-native database.
sentence-transformers - Multilingual Sentence & Image Embeddings with BERT
qdrant - Qdrant - High-performance, massive-scale Vector Database for the next generation of AI. Also available in the cloud https://cloud.qdrant.io/
hdbscan - A high performance implementation of HDBSCAN clustering.
llama - Inference code for Llama models
transformer-pytorch - Transformer: PyTorch Implementation of "Attention Is All You Need"
text-generation-webui - A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.