Python ElasticSearch

Open-source Python projects categorized as ElasticSearch

Top 20 Python ElasticSearch Projects

  • GitHub repo TWINT

    An advanced Twitter scraping & OSINT tool written in Python that doesn't use Twitter's API, allowing you to scrape a user's followers, following, Tweets and more while evading most API limitations.

    Project mention: Webscraper Twitter using Scrapy | reddit.com/r/scrapy | 2021-03-31

    I won't recommend using Scrapy to scrape twitter. You might want to check out Twint. It's a lot better [Twint](https://github.com/twintproject/twint)

  • GitHub repo awesome-aws

    A curated list of awesome Amazon Web Services (AWS) libraries, open source repos, guides, blogs, and other resources. Featuring the Fiery Meter of AWSome.

  • GitHub repo dev-setup

    macOS development environment setup: Easy-to-understand instructions with automated setup scripts for developer tools like Vim, Sublime Text, Bash, iTerm, Python data analysis, Spark, Hadoop MapReduce, AWS, Heroku, JavaScript web development, Android development, common data stores, and dev-based OS X defaults.

    Project mention: MacOS Development workspace 2021 | dev.to | 2021-03-08

    donnemartin - dev setup

  • GitHub repo sigma

    Generic Signature Format for SIEM Systems

    Project mention: Splunk course for use cases development | reddit.com/r/Splunk | 2021-02-25
  • GitHub repo elasticsearch-dsl-py

    High level Python client for Elasticsearch

    Project mention: Building ES analyzers: Any recommend GUIs or workflows? | reddit.com/r/elasticsearch | 2021-02-06
  • GitHub repo nyaa

    Bittorrent software for cats

    Project mention: Nyaa.si's github repository disabled after DMCA takedown notice was filed by MPA. | reddit.com/r/trackers | 2021-01-19

    Right. So explain to me why the repo is back up again now? https://github.com/nyaadevs/nyaa

  • GitHub repo archivy

    Archivy is a self-hosted knowledge repository that allows you to safely preserve useful content that contributes to your own personal, searchable and extendable wiki.

    Project mention: An Emacs wallabag client - the Emacser way to manage web pages! | reddit.com/r/emacs | 2021-04-12

    [1] https://archivy.github.io/

  • GitHub repo haystack

    :mag: End-to-end Python framework for building natural language search interfaces to data. Leverages Transformers and the State-of-the-Art of NLP. Supports DPR, Elasticsearch, HuggingFace’s Modelhub, and much more!

    Project mention: Ask HN: Who is hiring? (April 2021) | news.ycombinator.com | 2021-04-01

    deepset | Python Engineers, DevOps, Frontend | Berlin, Remote (CET +/- 2) | https://deepset.ai/

    We build Haystack, an Open Source-framework that empowers developers to build NLP-powered search pipelines for various use cases: https://github.com/deepset-ai/haystack

    On our mission to bring the State-of-the-Art of NLP into every application, we look for different roles to join our team and our journey! If you want to work in one of the most exciting areas of Machine Learning and actively work with an engaged and fast-growing community, reach out to us!

    You find our open roles here: http://careers.deepset.ai/ In case you identify with our mission but do not find a suitable role, do not hesitate to still reach out to us at [email protected]

  • GitHub repo RedELK

    Red Team's SIEM - tool for Red Teams used for tracking and alarming about Blue Team activities as well as better usability in long term operations.

    Project mention: Documentation / Logging - what are you using? | reddit.com/r/redteamsec | 2021-01-25

    Redelk - https://github.com/outflanknl/RedELK

  • GitHub repo match

    :crystal_ball: Scalable reverse image search built on Kubernetes and Elasticsearch

    Project mention: Reverse image search in my own local database? | reddit.com/r/DataHoarder | 2021-04-11

    https://github.com/dsys/match might be useful. You can upload all your images there and store path to the file on your local pc in the metadata.

  • GitHub repo stocksight

    Stock market analyzer and predictor using Elasticsearch, Twitter, News headlines and Python natural language processing and sentiment analysis

    Project mention: I wrote a trading algo that buys BTC every time Peter Schiff Tweets | reddit.com/r/Bitcoin | 2021-03-02

    Here is a starting place for anyone that wants to build such a thing https://github.com/shirosaidev/stocksight

  • GitHub repo nagios-plugins

    450+ AWS, Hadoop, Cloud, Kafka, Docker, Elasticsearch, RabbitMQ, Redis, HBase, Solr, Cassandra, ZooKeeper, HDFS, Yarn, Hive, Presto, Drill, Impala, Consul, Spark, Jenkins, Travis CI, Git, MySQL, Linux, DNS, Whois, SSL Certs, Yum Security Updates, Kubernetes, Cloudera etc...

    Project mention: check_yum.py weird output | reddit.com/r/nagios | 2021-04-16

    YUM WARNING: Cannot find summary line in yum output. Please make sure you have upgraded to the latest version from https://github.com/harisekhon/nagios-plugins. If the problem persists, please raise a ticket at https://github.com/harisekhon/nagios-plugins/issues with the full -vvv output

  • GitHub repo Eliot

    Eliot: the logging system that tells you *why* it happened

  • GitHub repo bertsearch

    Elasticsearch with BERT for advanced document search.

  • GitHub repo FeedHQ

    FeedHQ is a web-based feed reader

  • GitHub repo ck

    Collective Knowledge framework (CK) helps to organize black-box research software as a database of reusable components and micro-services with common APIs, automation actions and extensible meta descriptions. See real-world use cases from Arm, General Motors, ACM, Raspberry Pi foundation and others: (by ctuning)

    Project mention: Research software code is likely to remain a tangled mess | news.ycombinator.com | 2021-02-22

    – Their solution product https://cknowledge.io/ and source code https://github.com/ctuning/ck\

    I guess it should be helpful to the researchers community.

  • GitHub repo wazuh-ruleset

    Wazuh - Ruleset

    Project mention: Windows events alerts with Wazuh | reddit.com/r/Wazuh | 2021-03-19

    In this repository https://github.com/wazuh/wazuh-ruleset you can find the decoders and rules that wazuh-manager has by default (all these files are being migrated to the repository wazuh/wazuh https://github.com/wazuh/wazuh).

  • GitHub repo kiri

    Backprop makes it simple to use, finetune, and deploy state-of-the-art ML models.

    Project mention: Show HN: Backprop – a simple library to use and finetune state-of-the-art models | news.ycombinator.com | 2021-03-24
  • GitHub repo DataEngineeringProject

    Example end to end data engineering project.

    Project mention: Can You Recommend Good Data Engineering Projects | reddit.com/r/dataengineering | 2021-02-18

    Here is my project that got me a few interviews so far: https://github.com/damklis/DataEngineeringProject

  • GitHub repo ElasticTMDB

    ElasticTMDB is a Python3 module which sources movie and TV show details from The Movie Database (TMDB) and caches them in an Elasticsearch index to speed up subsequent queries to the same title

    Project mention: How to upload epg files in elasticsearch? | reddit.com/r/elasticsearch | 2021-04-11

    This might include some pointers to get you started: https://github.com/shaunschembri/ElasticTMDB

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2021-04-16.

Index

What are some of the best open-source ElasticSearch projects in Python? This list will help you:

Project Stars
1 TWINT 9,728
2 awesome-aws 9,157
3 dev-setup 5,379
4 sigma 3,478
5 elasticsearch-dsl-py 3,187
6 nyaa 2,786
7 archivy 2,508
8 haystack 1,634
9 RedELK 1,492
10 match 1,123
11 stocksight 1,076
12 nagios-plugins 1,008
13 Eliot 879
14 bertsearch 694
15 FeedHQ 529
16 ck 402
17 wazuh-ruleset 306
18 kiri 155
19 DataEngineeringProject 98
20 ElasticTMDB 2