fscrawler
mutad
fscrawler | mutad | |
---|---|---|
3 | 7 | |
1,308 | 15 | |
- | - | |
9.4 | 0.0 | |
2 days ago | over 1 year ago | |
Java | Java | |
Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
fscrawler
-
Tools for converting assorted text files to JSON?
Or use the ingest Attachment plugin, or https://github.com/dadoonet/fscrawler and remove any doubt
-
How to index PDF files with AppSeach
Did you sir give FSCrawler a shot? it's also a file system crawler that automatically ingests files into an ElasticSearch index and it is built over Tika so it performs typically the same (multiple file formats, multiple languages, ...). It can also perform OCR while indexing (uses Tesseract) by just toggling it on its config (I didn't try the OCR that much for a lack of need and for the ridiculous amount of times it added to the indexation process on my small environment).
mutad
- My First Attempt to Implement Real-Time Data-Processing/Search Engine Based on Typescript FE and Java Backend.
- WIP: Realtime Data-Processing and Search Engine Implementation with Typescript FE and Java BE.
- My first attempt to implement a React/Next.js based realtime data-processing/search engine with Typescript + Java.
- WIP: End-to-End Realtime Data Processing and Search Engine Implementation using Elasticsearch
- WIP: End-to-End Realtime Data Processing and Search Engine Implementation using React
- WIP: End-to-End Realtime Data Processing and Search Engine Implementation using Next.js
- WIP: End-to-End Realtime Data Processing and Search Engine Implementation.
What are some alternatives?
Elasticsearch - Free and Open, Distributed, RESTful Search Engine
es-kafka-pagination - Let’s say you have an ElasticSearch Index of 1,000,000 documents and you need to run some operation on those documents.
webmagic - A scalable web crawler framework for Java.
book-project - Book tracker web app for book lovers
nifi-extracttext-processor - Apache NiFi Custom Processor Extracting Text From Files with Apache Tika
Light-Java - A fast, lightweight and more productive microservices framework
analysis-ik - 🚌 The IK Analysis plugin integrates Lucene IK analyzer into Elasticsearch and OpenSearch, support customized dictionary.
eartquake-streaming - Distributed system on AWS which extracts earthquakes in real time using Apache Kafka and displays them on a frontend with load balancing
advanced-java - 😮 Core Interview Questions & Answers For Experienced Java(Backend) Developers | 互联网 Java 工程师进阶知识完全扫盲:涵盖高并发、分布式、高可用、微服务、海量数据处理等领域知识
Springy-Store-Microservices - Springy Store is a conceptual simple μServices-based project using the latest cutting-edge technologies, to demonstrate how the Store services are created to be a cloud-native and 12-factor app agnostic. Those μServices are developed based on Spring Boot & Cloud framework that implements cloud-native intuitive, design patterns, and best practices.
url-collector - An application that crawls the Common Crawl corpus for URLs with the specified file extensions.
JHipster - JHipster, much like Spring initializr, is a generator to create a boilerplate backend application, but also with an integrated front end implementation in React, Vue or Angular. In their own words, it "Is a development platform to quickly generate, develop, & deploy modern web applications & microservice architectures."