fpart
www.submarinecablemap.com
fpart | www.submarinecablemap.com | |
---|---|---|
5 | 206 | |
216 | 1,064 | |
- | - | |
7.9 | 6.3 | |
3 months ago | over 1 year ago | |
C | JavaScript | |
BSD 2-clause "Simplified" License | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
fpart
-
Rsync extremely slow on two ZFS local pools
Native rsync is terrible for lots of small file as it copies each file one by one sequentially. If you have lots of cores to work with, use the fpsync utility that comes with the fpart command to run parallel rsync's. You can easily saturate a 10Gb link with multiple rsync processes in parallel
-
Am I crazy to expect 100gbps across the pacific ocean?
You should probably use something like fpsync and multiple rsync jobs to get the most bandwidth.
-
Advice on 100gbps WAN?
My favorite free solution is fpsync/fpart from https://github.com/martymac/fpart -- basically that is a highly optimized filesystem crawler and indexer that can spit out balanced lists of files to transfer to a waiting army of parallel rsync workers. Tools are provided to manage the rsync fleet. Combining fpsync/fpart with an army of parallel rsync workers is a great design pattern especially for HPC as you can farm the rsync workers out to compute nodes
-
zfs replication vs multithreaded rsync
I've migrated data from our Isilon to zfs hostA using the fpsync tool that comes with the fpart utility. I get reasonably good throughput from this. 15TB in 5 and 1/2 hours
- How to back up 100TB NAS to USB HDDs??
www.submarinecablemap.com
-
Hetzner continues its growth in the US with a new location
Hillsboro, Oregon's network connections have a lot of advantages. It's worthwhile checking it out here: https://www.submarinecablemap.com/ --Katie
-
What is the internet?
Now you can say that you've 'seen' the internet. You can see the map here
-
Fiber carriers to Bermuda
Looking at this site there are 3 companies that own/operate undersea cables to the US
-
Sudden ping increase playing from South America, anyone else?
This unfortunately won't get you the whole picture, but using some tools you can determine how your traffic is routed in one direction, and possibly the geographic path it takes as well. For example, in mine, I can see based on the names that I know my traffic is going through Equinix San Jose (equinix-sj), then likely to Palo Alto (pao1, palo), then to Los Angeles (lax). By looking up who owns what IP addresses, I can also see that my traffic goes from my local ISP (Sonic) to Telia, then to Amazon. While concerning, you can effectively ignore all of the hops that say "Request timed out." as those just mean the hop wasn't responding to pings (or in the case of the very end, the game server itself likely doesn't respond to pings). Unfortunately though, this is only half of the picture, as this doesn't let me see the path from anets servers to me. For that, I would need an AWS instance with similar routing rules to anet's servers. Still, this may be useful, as I'm guessing your traffic is using a submarine cable to get to anet's servers in the US. These unfortunately often have issues or maintenance that can cause measurable connectivity impacts - I'm in the US and we have a node on the NLNOG Ring, and we get alerts of connectivity issues with Europe on a regular basis.
- Data Centers
- Dota in EU is far away from dead
- Why American Power Endures
-
Zero Point Leet Seconds
Well. Significantly more than that due to latency from switches etc and also because of the fact that there's so little land along the equator, meaning there's only one cable that travels roughly equatorially. It's from Fortaleza, Brazil to Kribi, Cameroon: https://www.submarinecablemap.com/
If you set up a bunch of good first-surface mirrors, I'm pretty sure you could get to pretty much the speed of light. You'd have to put them pretty high up in the air to avoid hitting things (a problem for cables as well, obviously) but putting the beam 2km in the air would still only lengthen the path by 4pi km, or .03%.
I have always found it very neat that the propagation speed of a light wave in glass is roughly the same as electrical waves in a coaxial cable. Both are shockingly slow compared to air/vacuum, but for completely different reasons. In both cases the advantages in signal integrity are immense.
- Casual Friday - Rave edition
- Could the internet literally be broken?
What are some alternatives?
TDengine - TDengine is an open source, high-performance, cloud native time-series database optimized for Internet of Things (IoT), Connected Cars, Industrial IoT and DevOps.
rayrender - A pathtracer for R. Build and render complex scenes and 3D data visualizations directly from R
pgBackRest - Reliable PostgreSQL Backup & Restore
mapgen4 - Mapgen4 procedural wilderness map generator
libarchive - Multi-format archive and compression library
globe.gl - UI component for Globe Data Visualization using ThreeJS/WebGL
criu - Checkpoint/Restore tool
Fantasy-Map-Generator - Web application generating interactive and highly customizable maps
sanoid - These are policy-driven snapshot management and replication tools which use OpenZFS for underlying next-gen storage. (Btrfs support plans are shelved unless and until btrfs becomes reliable.)
what-happens-when - An attempt to answer the age old interview question "What happens when you type google.com into your browser and press enter?"
stm32-usart-uart-dma-rx-tx - STM32 examples for USART using DMA for efficient RX and TX transmission
duckduckgo-locales - Translation files for <a href="https://duckduckgo.com"> </a>