minio
Mastodon
Our great sponsors
minio | Mastodon | |
---|---|---|
99 | 1,224 | |
44,094 | 45,874 | |
1.7% | 0.7% | |
9.9 | 10.0 | |
about 3 hours ago | about 14 hours ago | |
Go | Ruby | |
GNU Affero General Public License v3.0 | GNU Affero General Public License v3.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
minio
-
A Distributed File System in Go Cut Average Metadata Memory Usage to 100 Bytes
Looks like minio added this in 2022:
-
Simulate multi-nodes configuration
We have this example of docker compose you can adapt to be larger https://github.com/minio/minio/blob/master/docs/orchestration/docker-compose/docker-compose.yaml
-
Ask HN: I have 10 yrs of Exp. Failed 4 takehome projects. What am I doing wrong?
>Again, here you seem to be arguing against a strawman that doesn't know that blocking the IO loop is bad. Try arguing against one that knows ways to work around that. This is why I'm saying this rule isn't true. Extensive computation on single-threaded "scripting" languages is possible (and even if it wasn't, punt it off to a remote pool of workers, which could also be NodeJS!).
Very rare to find a rule that's absolutely true.. I clearly stated exceptions to the rule (which you repeated) but the generality is still true.
Threading in nodejs is new and didn't exist since the last time I touched it. It looks like it's not the standard use case as google searches still have websites with titles saying node is single threaded everywhere. The only way I can see this being done is multiple Processes (meaning each with a copy of v8) using OS shared memory as IPC and they're just calling it threads. It will take a shit load of work to make v8 actually multi-threaded.
Processes are expensive so you can't really follow this model per request. And we stopped following threading per request over a decade ago.
Again these are exceptions to the rule, from what I'm reading Nodejs is normally still single threaded with a fixed number of worker processes that are called "threads". Under this my general rule is still generally true: backend engineering does no typically involve writing non blocking code and offloading compute to other sources. Again, there are exceptions but as I stated before these exceptions are rare.
>Here's what I mean -- you can actually solve the ordering problem in O(N) + O(M) time by keeping track of the max you've seen and building a sparse array and running through every single index from max to zero. It's overkill, but it's generally referred to as a counting sort:
Oh come on. We both know these sorts won't work. These large numbers will throw off memory. Imagine 3 routes. One route gets 352 hits, another route gets 400 hits, and another route gets 600,000 hits. What's Big Oh for memory and sort?
It's O(600,000) for both memory and runtime. N=3 and it doesn't even matter here. Yeah these types of sorts are almost never used for this reason, they only work for things with smaller ranges. It's also especially not useful for this project. Like this project was designed so "counting sort" fails big time.
Also we don't need to talk about the O(N) read and write. That's a given it's always there.
>I don't think these statements make sense -- having docker installed and having redis installed are basically equivalent work. At the end of the day, the outcome is the same -- the developer is capable of running redis locally. Having redis installed on your local machine is absolutely within range for a backend developer.
Unfortunately these statements do make sense and your characterization seems completely dishonest to me. People like to keep their local environments pure and segregated away from daemons that run in a web server. I'm sure in your universe you are claiming web developers install redis, postgresql and kafka all locally but that just sounds absurd to me. We can agree to disagree but from my perspective I don't think you're being realistic here.
>Also, remote development is not practiced by many companies -- the only companies I've seen doing thin-clients that are large.
It's practiced by a large amount and basically every company I've worked at for the past 5 years. Every company has to at least partially do remote dev in order to fully test E2E stuff or integrations.
>I see it as just spinning up docker, not compose -- you already have access to the app (ex. if it was buildable via a function) so you could spawn redis in a subprocess (or container) on a random port, and then spawn the app.
Sure. The point is it's hacky to do this without an existing framework. I'll check out that library you linked.
>I agree that integration testing is harder -- I think there's more value there.
Of course there's more value. You get more value at higher cost. That's been my entire point.
>Also, for replicating S3, minio (https://github.com/minio/minio) is a good stand-in. For replicating lambda, localstack (https://docs.localstack.cloud/user-guide/aws/lambda/) is probably reasonable there's also frameworks with some consideration for this (https://www.serverless.com/framework/docs/providers/aws/guid...) built in.
Good finds. But what about SNS, IOT, Big Query and Redshift? Again my problem isn't about specific services, it's about infra in general.
>Ah, this is true -- but I think this is what people are testing in interviews. There is a predominant culture/shared values, and the test is literally whether someone can fit into those values.
No. I think what's going on is people aren't putting much thought into what they're actually interviewing for. They just have some made up bar in their mind whether it's a leetcode algorithm or whether the guy wrote a unit test for the one available pure function for testing.
>Whether they should or should not be, that's at least partially what interviews are -- does the new team member feel the same way about technical culture currently shared by the team.
The answer is no. There's always developers who disagree with things and just don't reveal it. Think about the places you worked at. Were you in total agreement? I doubt it. A huge amount of devs are opinionated and think company policies or practices are BS. People adapt.
>Now in the case of this interview your solution was just fine, even excellent (because you went out of your way to do async io, use newer/easier packaging methodologies, etc), but it's clearly not just that.
The testing is just a game. I can play the game and suddenly I pass all the interviews. I think this is the flaw with your methodology as I just need to write tests to get in. Google for example in spirit attempted another method which involves testing IQ via algorithms. It's a much higher bar
The problem with google is that their methodology can also be gamed but it's much harder to game it and often the bar is too high for the actual job the engineer is expected to do.
I think both methodologies are flawed, but hiring via ignoring raw ability and picking people based off of weirdly specific cultural preferences is the worse of the two hiring methodologies.
Put it this way. If a company has a strong testing culture, then engineers who don't typically test things will adapt. It's not hard to do, and testing isn't so annoying that they won't do it.
-
Unable to configure a MinIO cluster, pls help
The answer is here https://github.com/minio/minio/discussions/17543
You've already helped me here https://github.com/minio/minio/discussions/17543. Thank you very much once more.
-
What's the best AWS S3 protocol alternative?
You say protocol alternative, but assuming you're more concerned with AWS as the host than S3 as the protocol you might try https://github.com/minio/minio
If you do feel an aversion to the protocol then the rclone backend list would be a good starting point
-
proper content delivery (images etc)
Seems like you want object storage. S3 would be the goto suggestion here, but you said it needs to run on prem so perhaps MinIO.
-
Reason to use other Build Tool than Make?
You could refer to big OSS project Makefiles to take a look, what could be there, for example: https://github.com/minio/minio/blob/master/Makefile
-
Looking for a Backblaze B2 compatible cloud backup application for Linux that uses standard file level (not block level) ZIP encryption (and with GUI would be nice).
Backblaze's B2 is compatible with AWS S3 that also implemented in selfhosted minio
- Why compress-force doesn't compress
Mastodon
-
Open source at Fastly is getting opener
Through the Fast Forward program, we give free services and support to open source projects and the nonprofits that support them. We support many of the world’s top programming languages (like Python, Rust, Ruby, and the wonderful Scratch), foundational technologies (cURL, the Linux kernel, Kubernetes, OpenStreetMap), and projects that make the internet better and more fun for everyone (Inkscape, Mastodon, Electronic Frontier Foundation, Terms of Service; Didn’t Read).
-
Bluesky announces data federation for self hosters
Mastodon DMs have absolutely no privacy: https://github.com/mastodon/mastodon/issues/18079
For a decentralized protocol doing things right is much more important than doing things fast, it is very difficult (and in a lot of cases impossible) to break backwards compatibility.
- External OpenID Connect Account Takeover by Email Change
-
Ask HN: Best practice for posting links to large Mastodon threads?
Postmortem on what happened here: https://news.ycombinator.com/edit?id=39305884
The v1 API of Mastodon limits the size of the tree that it will expand for users who are not logged into the server: https://github.com/mastodon/mastodon/blob/main/app/controllers/api/v1/statuses_controller.rb . I am guessing that this or some similar limit applies to threads being returned to unauthenticated users of the web UI. It just arbitrarily stops expanding the replies at some point, including the main thread from the OP.
If a thread is truncated, users expect it to expand automatically and autoscroll when you hit the bottom. In my desktop browser, that does not occur, and there is no indication that there is more to see. This is the situation of the web interface as of Mastodon version 4.2.5.
The issue is very sensitive to observer conditions. If you are logged into the server, the behavior is different. If you use a Mastodon app instead of the web, the behavior might be different. As the tree expands, the cutoffs become different. If you look at the thread on a different Mastodon server, the tree is different because every server has its own view of the Fediverse.
HN needs a best practice for linking to Mastodon threads in a way that provides a consistent experience to HN readers. The average Mastodon server would be crushed by hundreds of HN readers grabbing the entirety of a huge thread all at once, so this might involve some thread-unroll-and-cache service. I tried https://mastoreader.io/ but it did not solve the problem.
Alternately, we push changes into the Mastodon web UI to warn users when they need to click to see more and assume that people will get used to the navigation.
Suggestions?
-
CVE-2024-23832 Mastodon Vulnerability: Remote user impersonation and takeover
Fixed in Mastodon v4.2.5 https://github.com/mastodon/mastodon/releases/tag/v4.2.5
-
Unity's Open-Source Double Standard: The Ban of VLC
>You can defeat the Affero clause by putting the software behind a proxy, for example
Could someone elaborate on this? This is NOT my understanding of the license, and it seems absurd considering e.g. Mastodon is AGPL but the standard install requires a reverse proxy[1]. If using a proxy defeats Affero, why would the Mastodon team do this? Are they stupid?
[1] https://github.com/mastodon/mastodon/blob/main/dist/nginx.co...
-
You Can't Follow Me
Mastodon is free and open-source. Go ahead and add the flag:
https://github.com/mastodon/mastodon/blob/main/CONTRIBUTING....
- Change Referer value to something generic such as "urn:activitypub:Mastodon"
-
Welcome to the public domain, Steamboat Willie
Didn't say anything about freedom of speech. And again: I'm not the one to talk to. I don't have any strong feelings on the topic, but if you do, you should take it somewhere that people who can do something about it will see.
I tried to find an existing discussion to help get you started, but couldn't. You can start one here: https://github.com/mastodon/mastodon/issues
It's easy to sit here on Hacker News and say "they should just..."
Coming up with a standard for an international project will be a long, noisy discussion. You'll tread on internecine conflicts you had no idea about. Old wounds from past related discussions will come out. People will soapbox.
This is why I have no interest in discussing it. It probably won't go anywhere in a place where it actually could. It definitely won't here.
-
Mastodon with Docker rootless, compose, and Nginx reverse proxy
I've written down how I set up my Mastodon server here. This includes some topics that seem not well covered currently:
- use nginx reverse proxy with the official nginx.conf [1], but with some changes needed for compatibility with docker
- use rootless docker, for security, together with bind mounts, for maintainability
- use compose, with some modifications to the official docker-compose.yml [2] that make life easier and are compatible with the reverse proxy
[1]: https://github.com/mastodon/mastodon/blob/main/dist/nginx.co...
[2]: https://github.com/mastodon/mastodon/blob/main/docker-compos...
What are some alternatives?
Nextcloud - ☁️ Nextcloud server, a safe home for all your data
diaspora* - A privacy-aware, distributed, open source social network.
Seaweed File System - SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, cross-DC active-active replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding. [Moved to: https://github.com/seaweedfs/seaweedfs]
Misskey - 🌎 An interplanetary microblogging platform 🚀
GlusterFS - Gluster Filesystem : Build your distributed storage in minutes
Lemmy - 🐀 A link aggregator and forum for the fediverse
Samba - https://gitlab.com/samba-team/samba is the Official GitLab mirror of https://git.samba.org/samba.git -- Merge requests should be made on GitLab (not on GitHub)
Friendica - Friendica Communications Platform
seaweedfs - SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, cross-DC active-active replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding.
GNU social - GNU social is social communication software for both public and private communications.
Swift - OpenStack Storage (Swift). Mirror of code maintained at opendev.org.
nostr - a truly censorship-resistant alternative to Twitter that has a chance of working