What would you use to get logs in a "pull way"?

• fluent-plugin-http-pull

  1 22 5.0 Ruby

  The input plugin of fluentd to pull log from rest api.

If not and you want some sort shot in the dark solution -- two that come to mind for shipping logs are fluentd has some community REST plugins and Cribl does REST pulls as well.

• prometheus

  384 53,052 9.9 Go

  The Prometheus monitoring system and time series database.

  

This is somewhat similar to how Prometheus scales up scrapes of metrics from 1000s of targets. Probably more complicated than you need, but take a look at the Prometheus scrape package code.

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

Suggest a related project