Data transfer to new Lustre storage overwhelms campus network

This page summarizes the projects mentioned and recommended in the original post on /r/DataHoarder

InfluxDB - Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
featured
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com
featured
  • wdt

    Discontinued Warp speed Data Transfer (WDT) is an embeddedable library (and command line tool) aiming to transfer data between 2 systems as fast as possible over multiple TCP paths. (by facebookarchive)

  • I guess? If you’re building parallel infrastructure (vlans are not enough obviously) just for running Aspera over it might not be the worst thing ever, but that’s an expensive way to live and that’s before you pay for the A$pera licenses. There are free and better behaved platforms out there like https://github.com/facebook/wdt if you don’t want other applications’ TCP sessions to time out while you’re trying to squeeze out the last half percent with Aspera.

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts

  • Global Socket – Connect like there is no firewall. Securely

    1 project | news.ycombinator.com | 30 Apr 2024
  • Zenoh: Zero Overhead Network Protocol

    1 project | news.ycombinator.com | 24 Apr 2024
  • Show HN: OpenZiti (Apache 2.0, P2P, E2E encrypted, full mesh overlay) is now 1.0

    1 project | news.ycombinator.com | 23 Apr 2024
  • 3rd Edition of Programming: Principles and Practice Using C++ by Stroustrup

    6 projects | news.ycombinator.com | 19 Apr 2024
  • Caching RESTful API requests with Heroku’s Redis Add-on

    2 projects | dev.to | 17 Apr 2024