Data Analytics at Potloc I: Making data integrity your priority with Elementary & Meltano

This page summarizes the projects mentioned and recommended in the original post on dev.to
#Workflow engine #Graph #NodeJS #Airflow #infrastructure-as-code

Our great sponsors
  • InfluxDB - Access the most powerful time series database as a service
  • Sonar - Write Clean Python Code. Always.
  • SaaSHub - Software Alternatives and Reviews
Our great sponsors

  • Airflow

    Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

    Airflow

  • elementary-ext

    - name: elementary namespace: elementary pip_url: elementary-data[platform]==0.6.3 git+https://github.com/potloc/elementary-ext.git executable: elementary_invoker settings: - name: project_dir kind: string value: ${MELTANO_PROJECT_ROOT}/transform/ - name: profiles_dir kind: string value: ${MELTANO_PROJECT_ROOT}/transform/profiles/platform/ - name: file_path kind: string value: ${MELTANO_PROJECT_ROOT}/path/to/report.html - name: skip_pre_invoke env: ELEMENTARY_EXT_SKIP_PRE_INVOKE kind: boolean value: true description: Whether to skip pre-invoke hooks which automatically run dbt clean and deps - name: slack-token kind: password - name: slack-channel-name kind: string value: elementary-notifs config: profiles-dir: ${MELTANO_PROJECT_ROOT}/transform/profiles/platform/ file-path: ${MELTANO_PROJECT_ROOT}/path/to/report.html slack-channel-name: your_channel_name skip_pre_invoke: true commands: initialize: args: initialize executable: elementary_extension describe: args: describe executable: elementary_extension monitor-report: args: monitor-report executable: elementary_extension monitor-send-report: args: monitor-send-report executable: elementary_extension

  • InfluxDB

    Access the most powerful time series database as a service. Ingest, store, & analyze all types of time series data in a fully-managed, purpose-built database. Keep data forever with low-cost storage and superior data compression.

  • terraform

    Terraform enables you to safely and predictably create, change, and improve infrastructure. It is an open source tool that codifies APIs into declarative configuration files that can be shared amongst team members, treated as code, edited, reviewed, and versioned.

    AWS Fargate as our hosting infrastructure, managed through terraform

  • nodejs-bigquery

    Node.js client for Google Cloud BigQuery: A fast, economical and fully-managed enterprise data warehouse for large-scale data analytics.

    Bigquery as our data warehouse

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts