gnu-parallel
xidel
| gnu-parallel | xidel | |
|---|---|---|
| 22 | 18 | |
| 25 | 652 | |
| - | - | |
| 10.0 | 5.6 | |
| about 9 years ago | 11 days ago | |
| Perl | Pascal | |
| GNU General Public License v3.0 only | GNU General Public License v3.0 only |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
gnu-parallel
-
SQL query execution idea
You can use GNU Parallel (https://www.gnu.org/software/parallel/) to run command-line clients with all of those queries. You can set up the upper limit of simultaneous clients run, and this will automatically handle all possible parallelism.
- Parallel – shell tool for executing jobs in parallel using one or more computers
-
Distcc: A fast, free distributed C/C++ compiler
Some other multi machine options that have worked well for me, well beyond just compilation of C/C++ on multiple machines with multiple cores.
1) set up passwordless, ssh.
and
2) use the gnu parallel. https://www.gnu.org/software/parallel/
gnu parallel is super flexible, very useful.
-
Peplum: F/OSS distributed parallel computing and supercomputing at Home with Ruby infrastructure
How does this stack up againg GNU parallel? If you just wanna parallelize CLI work-loads (like nmap), parallel should be easier, I guess.
-
Search in your Jupyter notebooks from the CLI, fast.
It requires jq for JSON processing and GNU parallel for concurrent searches in the notebooks.
- Is there a way to use all CPU cores while using RIBlast?
-
Can cuda help me here?
Since you've got lots of images, you could use GNU Parallel to spread the job across multiple CPUs.
-
5 great Perl scripts to keep in your sysadmin toolbox
Gnu parallel
- Is there an .deb package for installing GNU parallel?
-
Modern SPAs without bundlers, CDNs, or Node.js
You could easily use something like GNU Parallel:
https://www.gnu.org/software/parallel/
xidel
-
Move over jq I found something easier: fx
You could try Xidel[1]. It supports JSON, XML and HTML using XPath/XQuery 3.1
It has some extensions to the standard that are pretty nice (JSONiq, CSS selectors, html “template” matching), but you can limit it to just standard XPath/XQuery if you like.
I recommend getting the nightly v .99 build if you give it a try, the stable .98 version is pretty old and I’ve had no issues with .99
1. https://www.videlibri.de/xidel.html
-
Batch Win Installer - from a defined list of software, BWI will install software on 64 bit Windows 10/11 x64 machine without prompts ; check what software is installed and offer to install and/or upgrade software and scan program's websites to determine the latest version of the software available
Windows binary of Xidel (https://github.com/benibela/xidel) a commandline tool to download and extract data from HTML pages
- pup: Parsing HTML at the Command Line
- Remove white spaces from last column of CSV
-
What's the best tool to build pipelines from REST APIs?
Xidel for extraction and pagination
-
What are your coolest tools for one-liners ?
Download an entire subreddit to JSON Lines with Xidel:
-
Fetch data from XML
I believe that you can't read from a url with batch alone. you'll need a helper app like xidel https://www.videlibri.de/xidel.html
-
Tutorial: Rapid Script Development with Bash, JC, and JQ (no grep/sed/awk)
I have not played with this, but it looks like xidel might allow you to do this in Bash. jc also has a URL string parser that could be used in such a script.
- Xidel. A tool to query data from anything on the web and extract what you want
-
How to make http request with curl on certain page after being authenticated?
I built Xidel for such authenticated requests:
What are some alternatives?
Parallel
jq - Command-line JSON processor [Moved to: https://github.com/jqlang/jq]
bazel-buildfarm - Bazel remote caching and execution service
tools - all-in collection of productivity scripts, CLI tools, utility libraries, fuse filesystems, and also some stuff
lolcate-rs - Lolcate -- A comically fast way of indexing and querying your filesystem. Replaces locate / mlocate / updatedb. Written in Rust.
pup - Parsing HTML at the command line
jc - CLI tool and python library that converts the output of popular command-line tools, file-types, and common strings to JSON, YAML, or Dictionaries. This allows piping of output to tools like jq and simplifying automation scripts.
gron - Make JSON greppable!
ripgrep - ripgrep recursively searches directories for a regex pattern while respecting your gitignore
yq - Command-line YAML, XML, TOML processor - jq wrapper for YAML/XML/TOML documents
parallel - xargs for concurrent, distributed execution of shell commands
xmltodict - Python module that makes working with XML feel like you are working with JSON