Launch HN: Enso (YC S21) – Visual programming and workflow tool for data science

This page summarizes the projects mentioned and recommended in the original post on news.ycombinator.com

Our great sponsors
  • SonarLint - Deliver Cleaner and Safer Code - Right in Your IDE of Choice!
  • Scout APM - Less time debugging, more time building
  • SaaSHub - Software Alternatives and Reviews
  • enso

    Hybrid visual and textual functional programming.

    docs (https://enso.org/docs/syntax). We've added many new libraries, so you can do many more things with it now. Oh, and we changed the name to Enso and got accepted to YC! :)

    The problem we address is that data analysts still waste up to half of their time on repetitive manual work that can be automated [6]. To give one example, a company we're working with hires business users who use Excel to define data quality rules. These get manually translated to SQL, then manually translated to Python. This is not only error prone, it’s so slow that it takes them 90 days to introduce a single new rule. There’s 60 days’ worth of overhead in this process—it’s insane!

    Years ago I (Wojciech) led the in-house development of visual effects (VFX) tools at a motion picture studio. We made tools like cloud renderers and smoke simulation engines. The artists using these tools did not have any programming background, yet they were designing complex algorithms for forces between particles, light subsurface scattering, things like that. Earlier generations of these tools had hundreds of config options, buttons, etc., for masses of different use cases, but this approach got way too complex and people eventually realized that it falls short when you need to do anything that the vendor did not think of. Nowadays they use node-based software (like the Houdini FX) which lets users draw algorithms as a sequence of data processing steps (these steps are often referred to as “nodes”). Later, when I was working in other industries and encountered the same rats’ nests of complex GUIs for solving data processing problems, I realized that the data analytics/science space was in need of the same breakthrough that we had already gone through in the VFX space.

    Most visual programming languages / workflow-builders do not scale well because they don't let users express abstractions. Try to build a complex pipeline and you'll end with an unreadable spaghetti of connections—it's like coding a web app in the assembler. Enso is different because we allow you to build abstractions to manage the complexity. As a result, you never have more than 10-20 nodes on the stage in Enso (nodes are hierarchical). You can create custom data types, custom components (functions), catch errors, etc. All this works because under the hood, Enso is a real programming language. However, naive implementations of such systems are super slow. Each component may be built of hundreds, sometimes thousands of lower-level ones. The real trick is making these hierarchical components run fast. For that you need a dedicated compiler and a runtime system, and this is a hard technical space. Our system involves a dedicated JIT compiler based on GraalVM. For details, see https://enso.org/language#compiler. In case this is interesting for you, here is our podcast about how the compiler works under the hood: https://www.youtube.com/watch?v=BibjcUjdkO4.

    Enso is interactive, meaning that we recompute the relevant parts of graphs as parameters change, which shortens feedback loops dramatically. Like a lot of people on HN, we were inspired by Bret Victor's classic talk on instant feedback: https://www.youtube.com/watch?v=8QiPFmIMxFc. We’ve also put a lot of effort into extensibility. You can add Java, JavaScript, R, and Python (soon also Ruby, Scala, Kotlin, Rust, and C) directly into Enso nodes without the need to write any wrappers and with a close-to-zero performance overhead.

    Enso is open source. Our compiler code is at https://github.com/enso-org/enso and our GUI code at https://github.com/enso-org/ide. Our business model is based on selling domain specific libraries, on-premise installations with enhanced user permission management, and coming soon, a hosted solution called Enso Cloud, which will be our only non-open-source codebase. Since this is Hacker News, I should add that all our alpha releases collect anonymous usage statistics which we use to improve Enso and prepare it for a stable release. Full details about that are always in our release notes (https://github.com/enso-org/ide/releases/latest).

    Dear HN Family, we are super excited to show Enso to you. Please, share with us your thoughts, experiences, ideas and feedback. It is insanely important to us, as our dream is to make Enso the most useful data processing platform in your toolbox! Also, in case you’d like to build your projects on top of Enso, we would love to help you do it – describe what you have in mind here, and we will reach out to you: https://airtable.com/shrsnx2mJuRn0MxIS :)

    === Links ===

    [1] Luna: Visual and textual functional programming language* - https://news.ycombinator.com/item?id=11144828 - Feb 2016 (100 comments)

  • ide

    Enso – a visual and textual functional programming language.

    docs (https://enso.org/docs/syntax). We've added many new libraries, so you can do many more things with it now. Oh, and we changed the name to Enso and got accepted to YC! :)

    The problem we address is that data analysts still waste up to half of their time on repetitive manual work that can be automated [6]. To give one example, a company we're working with hires business users who use Excel to define data quality rules. These get manually translated to SQL, then manually translated to Python. This is not only error prone, it’s so slow that it takes them 90 days to introduce a single new rule. There’s 60 days’ worth of overhead in this process—it’s insane!

    Years ago I (Wojciech) led the in-house development of visual effects (VFX) tools at a motion picture studio. We made tools like cloud renderers and smoke simulation engines. The artists using these tools did not have any programming background, yet they were designing complex algorithms for forces between particles, light subsurface scattering, things like that. Earlier generations of these tools had hundreds of config options, buttons, etc., for masses of different use cases, but this approach got way too complex and people eventually realized that it falls short when you need to do anything that the vendor did not think of. Nowadays they use node-based software (like the Houdini FX) which lets users draw algorithms as a sequence of data processing steps (these steps are often referred to as “nodes”). Later, when I was working in other industries and encountered the same rats’ nests of complex GUIs for solving data processing problems, I realized that the data analytics/science space was in need of the same breakthrough that we had already gone through in the VFX space.

    Most visual programming languages / workflow-builders do not scale well because they don't let users express abstractions. Try to build a complex pipeline and you'll end with an unreadable spaghetti of connections—it's like coding a web app in the assembler. Enso is different because we allow you to build abstractions to manage the complexity. As a result, you never have more than 10-20 nodes on the stage in Enso (nodes are hierarchical). You can create custom data types, custom components (functions), catch errors, etc. All this works because under the hood, Enso is a real programming language. However, naive implementations of such systems are super slow. Each component may be built of hundreds, sometimes thousands of lower-level ones. The real trick is making these hierarchical components run fast. For that you need a dedicated compiler and a runtime system, and this is a hard technical space. Our system involves a dedicated JIT compiler based on GraalVM. For details, see https://enso.org/language#compiler. In case this is interesting for you, here is our podcast about how the compiler works under the hood: https://www.youtube.com/watch?v=BibjcUjdkO4.

    Enso is interactive, meaning that we recompute the relevant parts of graphs as parameters change, which shortens feedback loops dramatically. Like a lot of people on HN, we were inspired by Bret Victor's classic talk on instant feedback: https://www.youtube.com/watch?v=8QiPFmIMxFc. We’ve also put a lot of effort into extensibility. You can add Java, JavaScript, R, and Python (soon also Ruby, Scala, Kotlin, Rust, and C) directly into Enso nodes without the need to write any wrappers and with a close-to-zero performance overhead.

    Enso is open source. Our compiler code is at https://github.com/enso-org/enso and our GUI code at https://github.com/enso-org/ide. Our business model is based on selling domain specific libraries, on-premise installations with enhanced user permission management, and coming soon, a hosted solution called Enso Cloud, which will be our only non-open-source codebase. Since this is Hacker News, I should add that all our alpha releases collect anonymous usage statistics which we use to improve Enso and prepare it for a stable release. Full details about that are always in our release notes (https://github.com/enso-org/ide/releases/latest).

    Dear HN Family, we are super excited to show Enso to you. Please, share with us your thoughts, experiences, ideas and feedback. It is insanely important to us, as our dream is to make Enso the most useful data processing platform in your toolbox! Also, in case you’d like to build your projects on top of Enso, we would love to help you do it – describe what you have in mind here, and we will reach out to you: https://airtable.com/shrsnx2mJuRn0MxIS :)

    === Links ===

    [1] Luna: Visual and textual functional programming language* - https://news.ycombinator.com/item?id=11144828 - Feb 2016 (100 comments)

  • SonarLint

    Deliver Cleaner and Safer Code - Right in Your IDE of Choice!. SonarLint is a free and open source IDE extension that identifies and catches bugs and vulnerabilities as you code, directly in the IDE. Install from your favorite IDE marketplace today.

  • enso

    Is this related to William Cook's Enso? https://github.com/enso-lang/enso

  • benchmarks

    Luna was our early prototype and it indeed had significant performance problems. Our new JIT compiler is able to execute the graphs really fast (in fact, in many cases we are up to 80x faster than Python nowadays: https://github.com/enso-org/benchmarks).

    However, sometimes we see that the WebGL is super-slow on some machines. May I ask you for more details, please? The important bits would be whether you experience the slowdown when using the IDE (like dragging, zooming) or when waiting for data to be computed? In both cases, we would be more than thankful for an issue containing machine spec: https://github.com/enso-org/enso/issues .

    Enso should work for you with the same visual performance as in this video (this was recorded on a few years old MacBook Pro): https://youtu.be/fQvWMoOjmQk . If its not this smooth, we need to investigate it, because this is something system / machine specific.

    There is unfortunately no other way for us to improve and progress from alpha to beta, so we will be very thankful for help here! <3

  • graalvm-ce-builds

    GraalVM CE binaires built by the GraalVM community

    4. The rest (approx 200Mb) are core Enso libraries and tooling.

    We will be able to cut it down in the future as soon as we release Enso Marketplace which would allow you to install these things on-demand.

    [1] https://github.com/graalvm/graalvm-ce-builds/releases/tag/vm...

  • graalpython

    A Python 3 implementation built on GraalVM

  • fastr

    A high-performance implementation of the R programming language, built on GraalVM.

  • Scout APM

    Less time debugging, more time building. Scout APM allows you to find and fix performance issues with no hassle. Now with error monitoring and external services monitoring, Scout is a developer's best friend when it comes to application development.

  • graaljs

    A ECMAScript 2021 compliant JavaScript implementation built on GraalVM. With polyglot language interoperability support. Running Node.js applications!

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts