kamu-cli VS clvm_rs

Hi Rust community, This is a quick update on the kamu project - a tool that lets you build decentralized data pipelines and collaborate on data in a similar way to software.

Hi, I'm a developer of the kamu project and wanted to share its interesting use-case (outside of the conventional enterprise pipelines) with the Spark community.

Kamu Data. Tool for decentralized exchange and transformation of semi-structured data.

kamu is a new data management tool for rapid exchange and global collaboration on structured data.

We've been developing kamu for 2.5 years, and recently re-wrote it from Scala to Rust - best decision ever :)

#!/bin/sh PYTHON=python3.8 #***************************************************************** # Dependencies adapted from # https://github.com/Chia-Network/chia-blockchain/wiki/FreeBSD-Install # # Either install these from packages or ports # # lang/gcc9 lang/gcc devel/gmake devel/cmake lang/rust # shells/bash www/node www/npm devel/git security/openssl # devel/py-maturin devel/py-virtualenv devel/py-pip # devel/py-setuptools devel/py-wheel databases/py-sqlite3 devel/py-cffi # # Usage: # ./build_chia $version # so for version 1.3.1 you would run "./build_chia 1.3.1" # when finished you should have chia a 1.3.1 folder in the chia_builds # directory. Activate the virtual environment with # ". chia_builds/$version/venv/bin/activate" and you are good to go. # #***************************************************************** BASEDIR=`dirname "$0"` GET_CHIA_RS_PYTHON=" from setup import dependencies chia_rs = [x for x in dependencies if x.startswith('chia_rs==') or x.startswith('clvm_rs==')] print(chia_rs[0]) " GET_CLVM_TOOLS_RS_PYTHON=" from setup import dependencies clvm_tools_rs = [x for x in dependencies if x.startswith('clvm-tools-rs==')] print(clvm_tools_rs[0]) " mkdir $BASEDIR/chia_builds/$1 cd $BASEDIR/chia_builds/$1 git clone -b main https://github.com/Chia-Network/chia-blockchain.git $PYTHON -m venv $BASEDIR/chia_builds/$1/venv . $BASEDIR/chia_builds/$1/venv/bin/activate pip install --upgrade pip cd $BASEDIR/chia_builds/$1/chia-blockchain git checkout tags/$1 CHIA_RS_STRING=`echo "$GET_CHIA_RS_PYTHON" | python` CHIA_RS_LIBRARY=${CHIA_RS_STRING%%==*} CHIA_RS_VERSION=${CHIA_RS_STRING##*==} CLVM_TOOLS_RS=`echo "$GET_CLVM_TOOLS_RS_PYTHON" | python` echo Detected CHIA_RS type: $CHIA_RS_LIBRARY version $CHIA_RS_VERSION if [ "$CHIA_RS_LIBRARY" = "clvm_rs" ]; then CHIA_RS_REPO=https://github.com/Chia-Network/clvm_rs.git elif [ "$CHIA_RS_LIBRARY" = "chia_rs" ]; then CHIA_RS_REPO=https://github.com/Chia-Network/chia_rs.git fi cd $BASEDIR/chia_builds/$1 git clone --branch $CHIA_RS_VERSION $CHIA_RS_REPO cd $BASEDIR/chia_builds/$1/$CHIA_RS_LIBRARY git checkout tags/$CHIA_RS_VERSION cd $BASEDIR/chia_builds/$1/$CHIA_RS_LIBRARY/wheel maturin develop --release pip install git+https://github.com/Chia-Network/clvm@use_clvm_rs cd $BASEDIR/chia_builds/$1 if [ ! -z "$CLVM_TOOLS_RS" ]; then git clone -b base https://github.com/Chia-Network/clvm_tools_rs.git cd $BASEDIR/chia_builds/$1/clvm_tools_rs git checkout tags/${CLVM_TOOLS_RS##*==} maturin develop --release fi cd $BASEDIR/chia_builds/$1/chia-blockchain sh ./install.sh

Not a Chialisp developer, but are you thinking of more direct integration with rust implementation of the CLVM?

clvm-rs. Rust implementation of clvm.