Go IPFS
DISCONTINUED
Apache Hadoop
Our great sponsors
Go IPFS | Apache Hadoop | |
---|---|---|
63 | 26 | |
13,905 | 14,229 | |
- | 0.8% | |
9.6 | 9.9 | |
over 1 year ago | 7 days ago | |
Go | Java | |
GNU General Public License v3.0 or later | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Go IPFS
-
improving download infra
For me, https://github.com/ipfs/go-ipfs/issues/9044 is the main blocker atm and https://github.com/ipfs/go-ipfs/issues/2167 is still around and annoying.
-
Cheap, reliable way to host free archive of films of solidarity and struggle
I'm using the IPFS fuse mount to load mine into Plex/Jellyfin. It's nice that I can load a movie into a virtual directory on IPFS and my home and remote servers get updated automatically. (when I update my IPNS) So you could run an official solidaritycinema IPNS address that people load into their Plex as a library.
-
Remote Plex server and local Plex Server Sync
Maybe tangentially related, I've been interested in IPFS as a network medium. ( Using the IPFS fuse mount ) Rather than syncing the entire file it syncs the Library list. When the Plex server makes the request for the file, IPFS negotiates the download. It makes it more like Netflix.
-
We Put IPFS in Brave
"Implement bandwidth limiting" https://github.com/ipfs/go-ipfs/issues/3065
Going on six years now. You can use external tools (like "trickle") or your OS knobs.
- Can IPFS be used to share large files with others?
-
Does anyone else believe FIL will make them lot of money
The IPFS project in Github which requires a Github login in order to star a repo has over 20k unique people come by and star the project (https://github.com/ipfs) with language specific bindings for JS (https://github.com/ipfs/js-ipfs), go-ipfs (https://github.com/ipfs/go-ipfs) each of which ALSO have thousands of stars. It's fake!
-
A mostly complete guide to hosting a public IPFS gateway
sh apt install make pkg-config libssl-dev libcrypto++-dev mkdir -p ~/Applications git clone https://github.com/ipfs/go-ipfs.git ~/Applications/ipfs cd ~/Applications/ipfs go get github.com/lucas-clemente/quic-go@go118 GOTAGS=openssl make install
-
Is IPFS Dead?
The blog and go implementation and js implementation are still actively updated. So I wouldn't say it's dead. There are still pain points to be sure, but I'm still excited about this tech.
-
Webamp IPFS media player; sample hash included;)
> IPFS can use any transport protocol (see section 3.2 in the whitepaper [1]),
In theory. In practice, the network (I checked my local node with ~2500 nodes connected to it) is mostly using quic over tcp/udp, more or less 50%/50% split between tcp/udp.
> This busyness is acknowledged by the developers and should be addressed somewhere down the line.
IPFS has been killing routers[https://github.com/ipfs/go-ipfs/issues/3320] and sending/receiving lots of network traffic[https://github.com/ipfs/go-ipfs/issues/2917] since 2016 and there hasn't been any notable improvements on that front yet. When is "down the line" in reality?
-
A few notes on IPNS-Link-Gateways and www.ipns.live
Due to a full go-ipfs node at its core, long running Gateways at small VM (virtual machine)s (with 1GB RAM) might suffer from periodic OOM (Out-Of-Memory) outages. Periodic restarts are enough to get around this. ipns.live currently restarts hourly causing just a few seconds downtime every hour. This memory leak issue will be fixed with future go-ipfs releases or in future implementations of IPNS-Link-Gateway.
Apache Hadoop
- Unveiling the Analytics Industry in Bangalore
-
5 Best Practices For Data Integration To Boost ROI And Efficiency
There are different ways to implement parallel dataflows, such as using parallel data processing frameworks like Apache Hadoop, Apache Spark, and Apache Flink, or using cloud-based services like Amazon EMR and Google Cloud Dataflow. It is also possible to use parallel dataflow frameworks to handle big data and distributed computing, like Apache Nifi and Apache Kafka.
-
Data Engineering and DataOps: A Beginner's Guide to Building Data Solutions and Solving Real-World Challenges
There are several frameworks available for batch processing, such as Hadoop, Apache Storm, and DataTorrent RTS.
-
In One Minute : Hadoop
GitHub
The Apache™ Hadoop™ project develops open-source software for reliable, scalable, distributed computing.
-
Elon Musk dissolves Twitter's board of directors
So, clearly with your AP CS class and PLC logic knowledge, if you were dumped into a codebase like Hadoop, QT, or TensorFlow you'd be able to quickly and competently analyze what is going on with that code, understand all the libraries used, know the reasons why certain compromises were made, and be able to make suggestions on how to restructure the code in a different way? Because I've been programming for coming up on two decades and unless a system is within the domains that I have experience in, I would not be able to provide any useful information without a massive onboarding timeline, and definitely wouldn't be able to help redesign anything until actually coding within the system for a significant amount of time.
-
A peek into Location Data Science at Ola
This requires the use of distributed computation tools such as Spark and Hadoop, Flink and Kafka are used. But for occasional experimentation, Pandas, Geopandas and Dask are some of the commonly used tools.
-
How-to-Guide: Contributing to Open Source
Apache Hadoop
-
Python vs. Java: Comparing the Pros, Cons, and Use Cases
Hadoop (a Big Data tool).
-
Big Data Processing, EMR with Spark and Hadoop | Python, PySpark
Apache Hadoop is an open source framework that is used to efficiently store and process large datasets ranging in size from gigabytes to petabytes of data.Wanna dig more dipper?
What are some alternatives?
Ceph - Ceph is a distributed object, block, and file storage platform
Tahoe-LAFS - The Tahoe-LAFS decentralized secure filesystem.
minio - The Object Store for AI Data Infrastructure
syncthing - Open Source Continuous File Synchronization
Seaweed File System - SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, cross-DC active-active replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding. [Moved to: https://github.com/seaweedfs/seaweedfs]
Weka
MooseFS - MooseFS – Open Source, Petabyte, Fault-Tolerant, Highly Performing, Scalable Network Distributed File System (Software-Defined Storage)
GlusterFS - Web Content for gluster.org -- Deprecated as of September 2017
GlusterFS - Gluster Filesystem : Build your distributed storage in minutes
Airflow - Apache Airflow - A platform to programmatically author, schedule, and monitor workflows