elasticsearch-dump VS elastic-dump-rust

If you're wanting to take data out of Elasticsearch completely and make it readable via other tools you could look at ElasticDump, it supports export to S3 as well as the ability to delete documents as they're exported. https://github.com/elasticsearch-dump/elasticsearch-dump

Hello, in this tutorial I will describe the describe the steps are needed to be performed to migrate Indices from Elasticsearch with the use of Minio S3 Compatible Object Storage and a fantastic cli called elasticsearch-dump.

1. git clone https://github.com/elasticsearch-dump/elasticsearch-dump 2. cd elasticsearch-dump 3. Add any missing packages like curl nano e.t.c ### Start DockerFile FROM node:14-buster-slim LABEL maintainer="[email protected]" ARG ES_DUMP_VER ENV ES_DUMP_VER=${ES_DUMP_VER:-latest} ENV NODE_ENV production RUN npm install elasticdump@${ES_DUMP_VER} -g RUN apt-get update \ && apt-get install -y curl \ && rm -rf /var/lib/{apt,dpkg,cache,log} COPY docker-entrypoint.sh /usr/local/bin/ ENTRYPOINT ["docker-entrypoint.sh"] CMD ["elasticdump"] 4. docker build . -t /: 5. docker login -u 6. docker push /:

To try it - I'd recommend Elasticdump https://github.com/elasticsearch-dump/elasticsearch-dump

A couple of weeks back at work we had to build a server that should run a query on ES (Requires Authentication) and write output to a file and then publish it to S3. We started using elasticdump to achieve this.

Elasticdump would probably work. I dont know how much data youre moving but I use this for backups and it can move a good amount of data. https://github.com/elasticsearch-dump/elasticsearch-dump

I started writing some of the features of the npm package in Rust, which signs the URL and uses scroll API to get the data from ES and write it to a file. You can find the Repo here. I'll be extending features like uploading to S3 and writing the data to another ES.