elasticsearch-mapper-attachments VS Ansible

Elasticsearch - www.elastic.co/

The increasing complexity of modern systems led to the rise of AIOps (Artificial Intelligence for IT Operations) and observability practices. AIOps leveraged machine learning algorithms to automate problem detection, analysis, and resolution. Observability focused on gaining insights into system behaviour through metrics, logs, and traces. As a result, tools like Prometheus, Grafana, and ELK stack (Elasticsearch, Logstash, Kibana) gained popularity.

My only experience with NetFlow collection is on my home firewall/router running pfSense Community Edition, which is free to download and can be installed on a wide assortment of X86 hardware. I installed the Softflowd package, which exports NetFlow data to a dedicated Elasticsearch/Logstash/Kibana (ELK) server on my LAN. I believe Security Onion and ElastiFlow also can be NetFlow collectors.

Elasticsearch, Logstash, and Kibana (ELK) Stack: An open source suite of tools for log management and analysis, providing real-time insights into security events.

Recently I have taken an interest in big data. https://neo4j.com/ , https://cassandra.apache.org/ , https://clickhouse.com/, https://www.elastic.co/ - are all databases I have experience with. Neo4j and Cassandra only as a hobby, but Clickhouse I have used in production, and Elasticsearch I have used for some 7 years now.

Buy an enterprise-class, wired router like the Negate 2100 ($349 USD), which runs pfSense, and configure the Deco AXE5400 device(s) to operate in Access Point Mode. Then install the Softflowd package through the pfSense web UI. Softflowd will collect and export NetFlow data to a NefFlow collector, which is the separate computer/VM/container referred to above, running software like Security Onion, ElastiFlow, or Elasticsearch/Logstash/Kibana (ELK).

If you’re unaware, elastic search is some like enterprise level search shit. They just put it in a theme. https://www.elastic.co

systemctl status kibana ● kibana.service - Kibana Loaded: loaded (/etc/systemd/system/kibana.service; enabled; vendor preset: enabled) Active: active (running) since Tue 2023-03-28 09:40:05 UTC; 33min ago Docs: https://www.elastic.co Main PID: 3168 (node) Tasks: 11 (limit: 9432) Memory: 303.3M CPU: 35.190s CGroup: /system.slice/kibana.service └─3168 /usr/share/kibana/bin/../node/bin/node /usr/share/kibana/bin/../src/cli/dist --logging.dest=/var/log/kibana/kibana.log --pid.file=/run/kibana/kibana.pid "--deprecation.skip_deprecated_settings[0]=logging.dest" Mar 28 09:40:05 wazuh systemd[1]: Started Kibana.

Ansible is an open-source IT automation tool that simplifies application deployment, cloud provisioning, and configuration management across diverse environments. It uses a declarative language to describe the desired state of the system, and then takes the necessary actions to achieve that state. Ansible has become incredibly popular due to its simplicity, agentless architecture, and extensive community support. Document: ansible.com, ansible basics

Ansible v2.16

Ansible is a tool used to help manage software automation processes, configuration management across machines, deployment as well as remote execution of commands and scripts. In sports, Ansible operates as the coach of your team by providing strategies (playbooks), and actions, and ensuring the smooth execution of tasks across your infrastructure, just like a coach guides and directs players (Servers)during a game.

They support for-if from python, too: https://jinja.palletsprojects.com/en/3.1.x/templates/#loop-f... but I haven't tried the "recursive" keyword to know if ansible supports that. I say "ansible supports that" because they don't just drop jinja2 into ansible and call it a draw, they have a bunch of custom execution integrations: https://github.com/ansible/ansible/blob/v2.16.3/lib/ansible/...

To manage a VM, you can use something as simple as just manual actions over SSH, or can use tools like Ansible, Hashicorp's Packer and Terraform or other automations. For an app where there is minimal load and security/reliability concern, VMs are still a great option that provide a lot of value for the buck

In this article's context, it is simply a tool that provides a declarative way to automate your machine/OS to configure the development machine as you want (install package, modify the configuration, etc). Examples of these tools are Ansible, Puppet, etc.

Now we're getting more tangential, but for years, Ansible releases were named for Van Halen songs (see old Changelog here: https://github.com/ansible/ansible/blob/v1.8.4/CHANGELOG.md)

In the lab to follow, we'll quickly provision a 3-node kubeadm cluster (1 master, 2 workers) on the cloud provider of your choice using an automation stack comprised of OpenTofu and Ansible, then deploy Rook Ceph using the official Helm charts and confirm that we are now able to successfully create CSI volume snapshots from PVCs by reusing the MinIO example from our last article.