hal9ai VS orchest

At https://hal9.com, we built components for data science com native JavaScript to avoid the waiting times and download overhead if Pyodide. We found out the best tools for doing data science in the browser are a combination of Arquero and D3 and TensorFlow.js. At least for now.

We wrote our findings of this and many other libraries here: https://news.hal9.com/posts/data-science-with-javascript

We are not using libfortran not gdpr, we are basically using whatever libraries are available for the web. Since most data scientists don't want to use JS per se, you can build the apps as blocks in the Hal9 site or using a soon-to-be-released Python/R package, see https://notebooks.hal9.com

Feel free to check out our repo as well, all the "primitives" / blocks code is in the scripts folder: https://github.com/hal9ai/hal9ai

https://hal9.com helps data scientists build faster web applications.

It uses WebGL and WebAssembly to process larger datasets, perform inference in the browser with TensorFlow.js, and enables running Python code with Pyodide.

If you want to build a web application on top of your ML project, give https://hal9.com a shot. We designed Hal9 with ease of use for deployment and maximum compatibility with web technologies that enable you to build ML apps with React, Vue, etc. We launched a couple months ago but could use some early feedback and users. Thank you!

BTW. If you are ever interested in helping us out, you can send a PR's to our GitHub repo. For instance, the summarize and convert blocks are here: https://github.com/hal9ai/hal9ai/blob/main/scripts/transforms/summarize.txt.js and https://github.com/hal9ai/hal9ai/blob/main/scripts/transforms/convert.txt.js

In addition, you can also use Hal9 as a standalone JS library for data analysis and skip the UX, see https://github.com/hal9ai/hal9ai

Thanks! Is actually all Vue. I like Vue better over React, just personal preference, React is great and probably better for large applications. The trickiest path is the code that executes the pipelines since it has to run dynamic JS code with dynamic parameter; made that open source to make sure people are not stuck with the product if they ever have to leave or want to scar outside the product: https://github.com/hal9ai/hal9ai

You can find more about this project at https://hal9.com — We allow you to edit any block with JavaScript and to export the analysis as as embeddable HTML. You can also use Python or NodeJS if you need more advanced functionality.

We are working in https://hal9.com which is language agnostic and allows you to compose different programming languages; however, we are focused at the moment at 1D-graphs but have plans to support 2D-graphs in the coming weeks.

If you want a demo or just time to chat, I'm available at javier at hal9.ai.

We would love to help out since this is a perfect use case for https://hal9.ai; we are about to release our beta version that makes this as easy as copy-pasting code. You can find me at javier at hal9.ai to find some time to chat and give you a walkthrough of our code-to-api functionality.

The code example is part of our ELT/data pipeline tool called Orchest: https://github.com/orchest/orchest/

First want to say congrats to the Patterns team for creating a gorgeous looking tool. Very minimal and approachable. Massive kudos!

Disclaimer: we're building something very similar and I'm curious about a couple of things.

One of the questions our users have asked us often is how to minimize the dependence on "product specific" components/nodes/steps. For example, if you write CI for GitHub Actions you may use a bunch of GitHub Action references.

Looking at the `graph.yml` in some of the examples you shared you use a similar approach (e.g. patterns/openai-completion@v4). That means that whenever you depend on such components your automation/data pipeline becomes more tied to the specific tool (GitHub Actions/Patterns), effectively locking in users.

How are you helping users feel comfortable with that problem (I don't want to invest in something that's not portable)? It's something we've struggled with ourselves as we're expanding the "out of the box" capabilities you get.

Furthermore, would have loved to see this as an open source project. But I guess the second best thing to open source is some open source contributions and `dcp` and `common-model` look quite interesting!

For those who are curious, I'm one of the authors of https://github.com/orchest/orchest

Haven't tried it. In its favor, Argo is vendor neutral and is really easy to set up in a local k8s environment like docker for desktop or minikube. If you already use k8s for configuration, service discovery, secret management, etc, it's dead simple to set up and use (avoiding configuration having to learn a whole new workflow configuration language in addition to k8s). The big downside is that it doesn't have a visual DAG editor (although that might be a positive for engineers having to fix workflows written by non-programmers), but the relatively bare-metal nature of Argo means that it's fairly easy to use it as an underlying engine for a more opinionated or lower-code framework (orchest is a notable one out now).

We automatically provide container level parallelism in Orchest: https://github.com/orchest/orchest

If you’re looking for something with a great UI experience you can check out our open source project called Orchest. It might be what you seek from a simplicity perspective. https://github.com/orchest/orchest

Argo is pretty amazing if you want to take advantage of the work Kubernetes has done to scale resource efficiently across a cluster of compute nodes.

If you’re looking for something that’s a bit more high level and friendly to expose directly to your data team (data scientists/data engineers/data analysts) you can check out https://github.com/orchest/orchest

You can think of it as a browser UI/workbench for Argo scheduled pipelines. Disclaimer: author of the project

+1 on a lightweight version of GE to more easily make part of an existing pipeline. Would like it for internal use (our data pipelines), but also for our open source users (https://github.com/orchest/orchest).

You might want to add https://github.com/orchest/orchest/ to the Pipeline orchestration category (disclaimer: I work at the company making it)