fscrawler
Elasticsearch
fscrawler | Elasticsearch | |
---|---|---|
3 | 91 | |
1,308 | 67,708 | |
- | 0.7% | |
9.4 | 10.0 | |
3 days ago | 2 days ago | |
Java | Java | |
Apache License 2.0 | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
fscrawler
-
Tools for converting assorted text files to JSON?
Or use the ingest Attachment plugin, or https://github.com/dadoonet/fscrawler and remove any doubt
-
How to index PDF files with AppSeach
Did you sir give FSCrawler a shot? it's also a file system crawler that automatically ingests files into an ElasticSearch index and it is built over Tika so it performs typically the same (multiple file formats, multiple languages, ...). It can also perform OCR while indexing (uses Tesseract) by just toggling it on its config (I didn't try the OCR that much for a lack of need and for the ridiculous amount of times it added to the indexation process on my small environment).
Elasticsearch
-
Elasticsearch Version 9
You could check out their GitHub and see what is going on https://github.com/elastic/elasticsearch/issues
- One .gitignore to rule them all
-
Who's hiring developer advocates? (October 2023)
Link to GitHub -->
-
Do we think about vector dbs wrong?
I believe the 1024 limit has been upped in recent versions of Elasticsearch
https://github.com/elastic/elasticsearch/issues/92458
-
Elasticsearch VS openobserve - a user suggested alternative
2 projects | 30 Aug 2023
- A dedicated Elasticsearch query language (ES|QL)
- Fleet datastreams: custom index templates
-
Integrating Elasticsearch with Node.js Applications
Elasticsearch is written in Java and its source code is available on Github.
-
Murmur3 hash plugin for nested objects?
I don't think the murmur3 hash implementation has changed since it was added as the default in version 2.0 (see the [changes](https://github.com/elastic/elasticsearch/commits/main/server/src/main/java/org/elasticsearch/cluster/routing/Murmur3HashFunction.java)). The plugin itself has seen [more changes](https://github.com/elastic/elasticsearch/commits/main/plugins/mapper-murmur3) but that's IMO because of internals and not visible changes in the calculations.
-
Mongo or Mysql for 10tb of JSON documents, I'm questioning my previous choice.
Mysql is not as open source as postgres (long story). And you can see how open elasticsearch is by just having access to the bugs database https://github.com/elastic/elasticsearch/issue
What are some alternatives?
mutad - Realtime Data Processing and Search Engine Implementation.
OpenSearch - 🔎 Open source distributed and RESTful search engine.
webmagic - A scalable web crawler framework for Java.
Apache Superset - Apache Superset is a Data Visualization and Data Exploration Platform [Moved to: https://github.com/apache/superset]
nifi-extracttext-processor - Apache NiFi Custom Processor Extracting Text From Files with Apache Tika
bleve - A modern text/numeric/geo-spatial/vector indexing library for go
analysis-ik - 🚌 The IK Analysis plugin integrates Lucene IK analyzer into Elasticsearch and OpenSearch, support customized dictionary.
pgvector - Open-source vector similarity search for Postgres
advanced-java - 😮 Core Interview Questions & Answers For Experienced Java(Backend) Developers | 互联网 Java 工程师进阶知识完全扫盲:涵盖高并发、分布式、高可用、微服务、海量数据处理等领域知识
Whoosh
url-collector - An application that crawls the Common Crawl corpus for URLs with the specified file extensions.
MeiliSearch - A lightning-fast search API that fits effortlessly into your apps, websites, and workflow