minio VS minio-py

Compare minio vs minio-py and see what are their differences.

Our great sponsors
  • WorkOS - The modern identity platform for B2B SaaS
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • SaaSHub - Software Alternatives and Reviews
minio minio-py
99 9
44,220 767
2.2% 3.7%
9.9 8.3
about 21 hours ago about 1 month ago
Go Python
GNU Affero General Public License v3.0 Apache License 2.0
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.

minio

Posts with mentions or reviews of minio. We have used some of these posts to build our list of alternatives and similar projects. The last one was on 2024-02-22.
  • A Distributed File System in Go Cut Average Metadata Memory Usage to 100 Bytes
    2 projects | news.ycombinator.com | 22 Feb 2024
    Looks like minio added this in 2022:

    https://github.com/minio/minio/pull/15433

  • Simulate multi-nodes configuration
    1 project | /r/minio | 6 Dec 2023
    We have this example of docker compose you can adapt to be larger https://github.com/minio/minio/blob/master/docs/orchestration/docker-compose/docker-compose.yaml
  • Ask HN: I have 10 yrs of Exp. Failed 4 takehome projects. What am I doing wrong?
    7 projects | news.ycombinator.com | 11 Jul 2023
    >Again, here you seem to be arguing against a strawman that doesn't know that blocking the IO loop is bad. Try arguing against one that knows ways to work around that. This is why I'm saying this rule isn't true. Extensive computation on single-threaded "scripting" languages is possible (and even if it wasn't, punt it off to a remote pool of workers, which could also be NodeJS!).

    Very rare to find a rule that's absolutely true.. I clearly stated exceptions to the rule (which you repeated) but the generality is still true.

    Threading in nodejs is new and didn't exist since the last time I touched it. It looks like it's not the standard use case as google searches still have websites with titles saying node is single threaded everywhere. The only way I can see this being done is multiple Processes (meaning each with a copy of v8) using OS shared memory as IPC and they're just calling it threads. It will take a shit load of work to make v8 actually multi-threaded.

    Processes are expensive so you can't really follow this model per request. And we stopped following threading per request over a decade ago.

    Again these are exceptions to the rule, from what I'm reading Nodejs is normally still single threaded with a fixed number of worker processes that are called "threads". Under this my general rule is still generally true: backend engineering does no typically involve writing non blocking code and offloading compute to other sources. Again, there are exceptions but as I stated before these exceptions are rare.

    >Here's what I mean -- you can actually solve the ordering problem in O(N) + O(M) time by keeping track of the max you've seen and building a sparse array and running through every single index from max to zero. It's overkill, but it's generally referred to as a counting sort:

    Oh come on. We both know these sorts won't work. These large numbers will throw off memory. Imagine 3 routes. One route gets 352 hits, another route gets 400 hits, and another route gets 600,000 hits. What's Big Oh for memory and sort?

    It's O(600,000) for both memory and runtime. N=3 and it doesn't even matter here. Yeah these types of sorts are almost never used for this reason, they only work for things with smaller ranges. It's also especially not useful for this project. Like this project was designed so "counting sort" fails big time.

    Also we don't need to talk about the O(N) read and write. That's a given it's always there.

    >I don't think these statements make sense -- having docker installed and having redis installed are basically equivalent work. At the end of the day, the outcome is the same -- the developer is capable of running redis locally. Having redis installed on your local machine is absolutely within range for a backend developer.

    Unfortunately these statements do make sense and your characterization seems completely dishonest to me. People like to keep their local environments pure and segregated away from daemons that run in a web server. I'm sure in your universe you are claiming web developers install redis, postgresql and kafka all locally but that just sounds absurd to me. We can agree to disagree but from my perspective I don't think you're being realistic here.

    >Also, remote development is not practiced by many companies -- the only companies I've seen doing thin-clients that are large.

    It's practiced by a large amount and basically every company I've worked at for the past 5 years. Every company has to at least partially do remote dev in order to fully test E2E stuff or integrations.

    >I see it as just spinning up docker, not compose -- you already have access to the app (ex. if it was buildable via a function) so you could spawn redis in a subprocess (or container) on a random port, and then spawn the app.

    Sure. The point is it's hacky to do this without an existing framework. I'll check out that library you linked.

    >I agree that integration testing is harder -- I think there's more value there.

    Of course there's more value. You get more value at higher cost. That's been my entire point.

    >Also, for replicating S3, minio (https://github.com/minio/minio) is a good stand-in. For replicating lambda, localstack (https://docs.localstack.cloud/user-guide/aws/lambda/) is probably reasonable there's also frameworks with some consideration for this (https://www.serverless.com/framework/docs/providers/aws/guid...) built in.

    Good finds. But what about SNS, IOT, Big Query and Redshift? Again my problem isn't about specific services, it's about infra in general.

    >Ah, this is true -- but I think this is what people are testing in interviews. There is a predominant culture/shared values, and the test is literally whether someone can fit into those values.

    No. I think what's going on is people aren't putting much thought into what they're actually interviewing for. They just have some made up bar in their mind whether it's a leetcode algorithm or whether the guy wrote a unit test for the one available pure function for testing.

    >Whether they should or should not be, that's at least partially what interviews are -- does the new team member feel the same way about technical culture currently shared by the team.

    The answer is no. There's always developers who disagree with things and just don't reveal it. Think about the places you worked at. Were you in total agreement? I doubt it. A huge amount of devs are opinionated and think company policies or practices are BS. People adapt.

    >Now in the case of this interview your solution was just fine, even excellent (because you went out of your way to do async io, use newer/easier packaging methodologies, etc), but it's clearly not just that.

    The testing is just a game. I can play the game and suddenly I pass all the interviews. I think this is the flaw with your methodology as I just need to write tests to get in. Google for example in spirit attempted another method which involves testing IQ via algorithms. It's a much higher bar

    The problem with google is that their methodology can also be gamed but it's much harder to game it and often the bar is too high for the actual job the engineer is expected to do.

    I think both methodologies are flawed, but hiring via ignoring raw ability and picking people based off of weirdly specific cultural preferences is the worse of the two hiring methodologies.

    Put it this way. If a company has a strong testing culture, then engineers who don't typically test things will adapt. It's not hard to do, and testing isn't so annoying that they won't do it.

  • Unable to configure a MinIO cluster, pls help
    1 project | /r/selfhosted | 30 Jun 2023
    The answer is here https://github.com/minio/minio/discussions/17543
    1 project | /r/minio | 30 Jun 2023
    You've already helped me here https://github.com/minio/minio/discussions/17543. Thank you very much once more.
  • What's the best AWS S3 protocol alternative?
    3 projects | news.ycombinator.com | 31 May 2023
    You say protocol alternative, but assuming you're more concerned with AWS as the host than S3 as the protocol you might try https://github.com/minio/minio

    If you do feel an aversion to the protocol then the rclone backend list would be a good starting point

    https://rclone.org/overview/

  • proper content delivery (images etc)
    1 project | /r/sysadmin | 25 May 2023
    Seems like you want object storage. S3 would be the goto suggestion here, but you said it needs to run on prem so perhaps MinIO.
  • Reason to use other Build Tool than Make?
    9 projects | /r/golang | 19 May 2023
    You could refer to big OSS project Makefiles to take a look, what could be there, for example: https://github.com/minio/minio/blob/master/Makefile
  • Looking for a Backblaze B2 compatible cloud backup application for Linux that uses standard file level (not block level) ZIP encryption (and with GUI would be nice).
    3 projects | /r/DataHoarder | 16 May 2023
    Backblaze's B2 is compatible with AWS S3 that also implemented in selfhosted minio
  • Why compress-force doesn't compress
    1 project | /r/btrfs | 11 May 2023

minio-py

Posts with mentions or reviews of minio-py. We have used some of these posts to build our list of alternatives and similar projects. The last one was on 2022-12-07.
  • MinIO - Mock S3 in local development
    5 projects | dev.to | 7 Dec 2022
    Python
  • Upload data directly from the web into Minio Bucket
    1 project | /r/minio | 17 Oct 2022
    Alternatively you could look into presigned urls which can also be done in python.
  • Reading JSON files from stream into memory
    1 project | /r/learnpython | 16 Jul 2022
    The program is running a SQL query against a remote file. This example in MinIO docs demonstrates reading from the response stream. So far, so good.
  • Help with making a request to the API?
    1 project | /r/minio | 23 Apr 2022
  • How do you build a bare minimum feature?
    1 project | news.ycombinator.com | 11 Nov 2021
    >So far your instinct as product designer/manager/engineer might be to take what customers say they want at face value.

    That would be the worst way to go about building anything. Taking solutions from customers as opposed to problems to identify jobs-to-be-done and reasons for non consumption isn't something someone who knows how to build products does.

    The question then becomes how does someone who generally knows how to build the right things build the bare minimum feature, the original question and title of the post, given the constraints on time and resources cited in the first paragraphs.

    One can do several things. For example, writing the description and code examples for a yet to be library and shop it around, see if it makes sense, then write the library.

    That's what I did for example with this library: https://pypi.org/project/bmc after shopping it around here: https://github.com/minio/minio-py/issues/829#issuecomment-65...

    I also do that for internal tools, libraries, SDKs. I'll send client code around and see if it makes sense to engineers. I also had non engineers who never coded use libraries to do something useful. I just give them a laptop and docs and see how they use the thing. If they can do it, I know programmers can.

    To get back to the original problem, prioritizing can lead towards minimal features. We have a section called "Instances" in our issue templates. If a feature issue does not have several instances where the problem manifested for several people, we just won't do it. We need concrete examples of a problem being frequent and expensive / high impact (loss of work, or prevents work in the first place).

    For example, I'll look at the analytics for our internal platform and see that my colleague who we built the thing for is not using it. Why, we ask. It turns out the Docker images are too large and take a lot of time. He loses patience. They contain several libraries that he does need.

    We built a minimal image that can get him to start after 30 seconds instead of 2 minutes, and we saw usage increase.

    Yesterday, I saw he was giving a demo using his local environement, not the product. I'll ask why. What's wrong. What sucks. Why did he use his laptop instead of the platform.

    The general sentiment of the article is sound. Always observe, ask questions, look for the underlying problems and how frequently they happen, how expensive they are (as in what are they doing to solve that, is there someone working on it, does it cost opportunities, etc)

  • Ask HN: What point you keep while designing a website for MVP?
    1 project | news.ycombinator.com | 18 Sep 2021
    I used to write libraries and have non-technical people who never programmed follow the documentation and use them. My reference point is that if they could do it, then programmers will have no trouble doing it.

    The next best thing is writing the doc before the actual code, and have others look at it and see if it makes sense, then I'd write the code that makes that possible.[0]

    - [0]: https://github.com/minio/minio-py/issues/829#issuecomment-65...

  • Tell HN: Curb Your Necroposting Aversion
    1 project | news.ycombinator.com | 12 Sep 2021
    Yes. A case of it working well: https://github.com/minio/minio-py/issues/829

    I found this issue for a problem I had with MinIO. I wrote a prototype: docs first and screencast on how the API would be. I then asked the other user if they had implemented it and if the API I was thinking about would suit them as it suited me. MinIO's CTO replied that it was the correct usage, but that it was not on the roadmap, which is understandable.

    I then started implementing the actual commands I needed first (host, ls, etc). Some time later, someone tweeted at us and asked how they could use the library. I then put it on PyPI (Python Package Index). I checked upon that person recently and they told be they've been using it for months without a problem. The other person also had commented and said they've been using it.

    https://pypistats.org/packages/bmc

    It's a really, really, small library for a niche need, also asked people for feedback here: https://news.ycombinator.com/item?id=26540342

    Now I'm getting feature requests on that issue, but I'll soon do it properly in a repo to make it easier for people.

  • Ask HN: How do I improve boring README page?
    7 projects | news.ycombinator.com | 22 Mar 2021
    Thank you for these points. I agree. It's one of those tools that, if you have that problem, you know exactly why you need it.

    This solves this issue: https://github.com/minio/minio-py/issues/829. MinIO is an S3 compatible object storage software. https://min.io/

    Thanks again.

  • The Trouble with Cassandra: Why It's a Poor Choice for Object Store Metadata DB
    3 projects | news.ycombinator.com | 24 Feb 2021
    We use it in anger at Splitgraph [0] to let people upload/download datasets and Postgres table snapshots (instead of storing them directly in S3).

    Pros:

    * Less platform-dependent. By self-managing it, we can also run deploys to GCP / Azure / Scaleway / other providers without writing a separate adapter for e.g. Azure Blob Storage.

    * Python API [1] much more pleasant to use than boto3 (and can speak to normal S3). It doesn't do everything that boto3 does, but it supports everything we need (e.g. pre-signed URLs).

    * minio server itself supports a large chunk of S3's functionality (e.g. SELECT API / AssumeRole / bucket versioning)

    * Don't pay per request and for egress: this was a big deal since people might want to download large amounts of data from us (or make a bunch of small requests to download/upload a subset of data).

    Cons:

    * Have to manage own infrastructure. We run it on managed VMs so it's semi-managed, but we still have to provision block storage, set up backup policies etc.

    * In a similar vein, scaling and availability all have to be DIY [2]. We haven't run into situations yet where Minio would be the bottleneck, but it might be something to keep in mind.

    * Obviously not as seamless: you don't get things like Glacier or integration with other IAM.

    [0] https://www.splitgraph.com/

    [1] https://github.com/minio/minio-py

    [2] https://docs.min.io/docs/distributed-minio-quickstart-guide....

What are some alternatives?

When comparing minio and minio-py you can also consider the following projects:

Nextcloud - ☁️ Nextcloud server, a safe home for all your data

Seaweed File System - SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, cross-DC active-active replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding. [Moved to: https://github.com/seaweedfs/seaweedfs]

s3www - Serve static files from any S3 compatible object storage services (Let's Encrypt ready)

GlusterFS - Gluster Filesystem : Build your distributed storage in minutes

msgraph-sdk-python-core - Microsoft Graph client library for Python

Samba - https://gitlab.com/samba-team/samba is the Official GitLab mirror of https://git.samba.org/samba.git -- Merge requests should be made on GitLab (not on GitHub)

seaweedfs - SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, cross-DC active-active replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding.

awesome-readme - A curated list of awesome READMEs

Swift - OpenStack Storage (Swift). Mirror of code maintained at opendev.org.

minio-java - MinIO Client SDK for Java