s3-credentials
A tool for creating credentials for accessing S3 buckets (by simonw)
amazon-s3-find-and-forget
Amazon S3 Find and Forget is a solution to handle data erasure requests from data lakes stored on Amazon S3, for example, pursuant to the European General Data Protection Regulation (GDPR) (by awslabs)
| s3-credentials | amazon-s3-find-and-forget | |
|---|---|---|
| 2 | 3 | |
| 188 | 232 | |
| - | 0.9% | |
| 6.5 | 7.3 | |
| 28 days ago | 8 days ago | |
| Python | Python | |
| Apache License 2.0 | Apache License 2.0 |
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
s3-credentials
Posts with mentions or reviews of s3-credentials.
We have used some of these posts to build our list of alternatives
and similar projects.
amazon-s3-find-and-forget
Posts with mentions or reviews of amazon-s3-find-and-forget.
We have used some of these posts to build our list of alternatives
and similar projects.
-
Deleting particular data from S3 External Tables
Take a look at this: https://github.com/awslabs/amazon-s3-find-and-forget We use it for GDPR compliance; it will open a file, delete a row and pack it back. It will modify the file so watch out if you are using Glue job bookmarks. Because you are using external tables, the manifest file will also have to be updated with a proper lenght for the new, updated file. If you have hundreds of tables and thousands of files, and you need to do this on a regular basis this would be the scalable solution, but if you have few files honestly I would do it manually
-
Update S3 Files
Have a look at S3 Find and Forget
-
How to handle GDPR requests for data stored in S3 ?
S3 Find and Forget is probably worth looking into, even if just to get ideas on how to implement a similar solution for yourself
What are some alternatives?
When comparing s3-credentials and amazon-s3-find-and-forget you can also consider the following projects:
awssert - Declarative assertions for AWS
DataEngineeringProject - Example end to end data engineering project.
ec2-retirement-bulk-fix - A tool that lists EC2 instances scheduled for retirement and optionally stop/start all in bulk
isp-data-pollution - ISP Data Pollution to Protect Private Browsing History with Obfuscation
digital-ocean-spaces - MIT Licensed python client for DigitalOcean spaces
awesome-aws - A curated list of awesome Amazon Web Services (AWS) libraries, open source repos, guides, blogs, and other resources. Featuring the Fiery Meter of AWSome.
data-toolset - Upgrade from avro-tools and parquet-tools jars to a more user-friendly Python package.