Need help with best approach to process millions of records

This page summarizes the projects mentioned and recommended in the original post on /r/PHP
Composer Repositories Queue message-queue message-bus Rabbitmq

InfluxDB - Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
featured
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com
featured

  • Enqueue

    Message Queue, Job Queue, Broadcasting, WebSockets packages for PHP, Symfony, Laravel, Magento. DEVELOPMENT REPOSITORY - provided by Forma-Pro

    • Set yourself up with a dedicated instance with rabbitmq, consider a library like this, https://github.com/php-enqueue/enqueue-dev, build a tool to create a chunked set of your unique domains, and start pumping away.

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts

  • plumber VS kaf - a user suggested alternative

    2 projects | 12 Jan 2024

  • Data Pipelines - how do you build data pipelines for sources not available in today’s ELT tools (Fivetran, Talend, Airbyte)? Old fashioned scripts and YOLO?

    1 project | /r/dataengineering | 13 Apr 2023

  • I want to participate in a golang open source projects. Have any suggestions or recommendations?

    4 projects | /r/golang | 19 Feb 2023

  • batchcorp/plumber: A swiss army knife CLI tool for interacting with Kafka, RabbitMQ and other messaging systems.

    1 project | /r/devel | 9 Nov 2022

  • Plumber

    1 project | /r/devopspro | 25 Sep 2022