grub-2.0
MeiliSearch
grub-2.0 | MeiliSearch | |
---|---|---|
4 | 129 | |
19 | 43,397 | |
- | 1.5% | |
0.0 | 9.8 | |
over 1 year ago | 10 days ago | |
Python | Rust | |
- | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
grub-2.0
-
I want to dive into how to make search engines
Not finished, but the Selenium based crawler works pretty well to combat most blocks: https://github.com/kordless/grub-2.0
For IP blocks, try this: https://github.com/kordless/mitta-screenshot
-
Ask HN: Decent, open source search engine?
I started https://mitta.us as this, but am pivoting to prompt management for GPT-3. I've Open Sourced the code for the crawler here: https://github.com/kordless/grub-2.0. The entire system uses Google Vision for extracting text. I dislike fiddling with the DOM...
If you are interested in using Solr for this, I can provide instructions to you. I'm kordless at the gmails ... com.
-
How to Scrape and Extract Hyperlink Networks with BeautifulSoup and NetworkX
Depending on the use case you might try imaging the page, then send the image to an ML model for full text before indexing. If you need links extracted, Selenium also supports parsing the assembled DOM: https://github.com/kordless/grub-2.0/tree/main/aperture
-
Mastering Web Scraping in Python: Crawling from Scratch
I’ve found imaging the page and doing OCR on the image is quite good for text extraction. Many pages on the Internet render with JavaScript, which means BS may not see the text in the DOM.
Here is the code to do some of that: https://github.com/kordless/grub-2.0
MeiliSearch
-
Publish/Subscribe with Sidekiq
We needed to introduce a new service for search. As we settled on using meilisearch, we needed a way to sync updates on our models with the records in meilisearch. We could've continued to use callbacks but we needed something better.
-
The Mechanics of Silicon Valley Pump and Dump Schemes
Meilisearch
-
What is Hybrid Search?
In this case, a good strategy is to use vector search only when the keyword/prefix search returns none or just a small number of results. A good candidate for this is MeiliSearch. It uses custom ranking rules to provide results as fast as the user can type.
-
Create a ChatBot with VertexAI and LibreChat
With the VertexAI endpoint set up and tested, our next step is to work with LibreChat. LibreChat is an open-source ChatGPT clone that can integrate with various AI models, including the PaLM 2 models via the VertexAI API. It's built using React, MongoDB, and Meilisearch technologies.
-
Pg_bm25: Elastic-Quality Full Text Search Inside Postgres
Meilisearch seems like it is the best open source option.
https://www.meilisearch.com/
- Looking for an easy installable search engine for a shared hosting account? Any ideas?
- Meilisearch: Build an intuitive search experience in a snap
-
Vector storage is coming to Meilisearch to empower search through AI
Starting with v1.3, you can use Meilisearch as a vector store. Meilisearch allows you to store vector embeddings alongside your documents conveniently. You will need to create the vector embeddings using your third-party tool of choice (Hugging Face, OpenAI). As we published the first v1.3 release candidate, you can try out vector search today.
-
[N] Open-source search engine Meilisearch launches vector search
I work at Meilisearch, an open-source search engine built in Rust. 🦀
-
Creating search engine for your local network - Is it even possible?
https://www.meilisearch.com/ https://github.com/meilisearch
What are some alternatives?
ChromeController - Comprehensive wrapper and execution manager for the Chrome browser using the Chrome Debugging Protocol.
Typesense - Open Source alternative to Algolia + Pinecone and an Easier-to-Use alternative to ElasticSearch ⚡ 🔍 ✨ Fast, typo tolerant, in-memory fuzzy Search Engine for building delightful search experiences
skyscraper - Structural scraping for the rest of us.
zincsearch - ZincSearch . A lightweight alternative to elasticsearch that requires minimal resources, written in Go.
mitta-screenshot - Mitta's Chrome extension for saving the current view of a website.
Elasticsearch - Free and Open, Distributed, RESTful Search Engine
rod - A Devtools driver for web automation and scraping
Searx - Privacy-respecting metasearch engine
phalanx - Phalanx is a cloud-native distributed search engine that provides endpoints through gRPC and traditional RESTful API.
sonic - 🦔 Fast, lightweight & schema-less search backend. An alternative to Elasticsearch that runs on a few MBs of RAM.
markov - Materials for book: "Markov Chains for programmers"
rust-postgres - Native PostgreSQL driver for the Rust programming language