Ask HN: Visualizing software designs, especially of large systems (if at all)?

This page summarizes the projects mentioned and recommended in the original post on news.ycombinator.com

SurveyJS - Open-Source JSON Form Builder to Create Dynamic Forms Right in Your App
With SurveyJS form UI libraries, you can build and style forms in a fully-integrated drag & drop form builder, render them in your JS app, and store form submission data in any backend, inc. PHP, ASP.NET Core, and Node.js.
surveyjs.io
featured
InfluxDB - Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
featured
  • c4-notation

    Technical resources for using the C4 model for visualizing software architecture.

  • If you're interested in diagramming system architecture in a top-down fashion, I'll pile on another recommendation to check out the C4 model[0].

    As others have mentioned, to effectively communicate a system you must limit the context to particular layers of abstraction, and C4 is a good approach to doing just that. There's also a C4 plugin for PlantUML[1].

    But don't forget that, as with all visualizations, audience and purpose are key.

    Consider whether you are addressing short-term needs (eg identifying inefficiencies, modeling for a client pitch) or long-term needs (eg knowledge retention, managing complexity). If your audience's needs are short-term, you can certainly get by with much simpler tools (eg Inkscape, excalidraw/draw.io/etc, picture of a whiteboard, doodles on a napkin).

    Also consider whether or not you actually have a problem better served by bottom-up (ie generated) visualizations (eg ERDs for database schema refactoring, heatmaps for profiling, GraphViz for debugging DAGs).

    [0] https://c4model.com/

    [1] https://github.com/plantuml-stdlib/C4-PlantUML

  • mermaid

    Generation of diagrams like flowcharts or sequence diagrams from text in a similar manner as markdown

  • I like using https://c4model.com/ specifically - the Level 2 diagram is particularly useful.

    I use https://mermaid-js.github.io/mermaid/#/ for the diagram itself because Github natively supports it in markdown files. I managed to get reasonably close to the C4 diagrams minus a few features that mermaid does not support.

  • SurveyJS

    Open-Source JSON Form Builder to Create Dynamic Forms Right in Your App. With SurveyJS form UI libraries, you can build and style forms in a fully-integrated drag & drop form builder, render them in your JS app, and store form submission data in any backend, inc. PHP, ASP.NET Core, and Node.js.

    SurveyJS logo
  • plantuml

    Generate diagrams from textual description

  • This tool is good for making simple UML diagrams and even lets you do it with simplified syntax

    https://plantuml.com/

    I'd say the big problem in visualizing big systems is that you can't usefully do it in one graph. For instance I worked on a system that had 2000+ database tables if you were going to make a diagram of that which shows everything it is going to take up a long wall. (This can be useful, but it is a big commitment)

    A useful tool is going to let you make meaningful diagrams that show the subset of entities that are part of a story. I went to an art show of Mark Lombardi's works

    https://en.wikipedia.org/wiki/Mark_Lombardi

    who (before he was murdered) drew elaborate diagrams of conspiracies. One thing they showed was drafts that he made in the progress of creating his visualizations and he would sometimes make 40 or more of them. He would start out with a "hairball" that was disorganized and gradually figure out how to lay the diagram out in a way that made the meaning obvious.

  • TypeScript-Call-Graph

    CLI to generate an interactive graph of functions and calls from your TypeScript files

  • Specifically for TypeScript I created a CLI to visualize the call graph

    https://github.com/whyboris/TypeScript-Call-Graph

    Works for _functions_ not classes. I'm unsure how useful this tool is, but I suspect it might be helpful in some codebases.

  • plurid

    Explore Information as a 3D Structure

  • I gave this a bit of thought earlier this year (https://alexanderell.is/posts/visualizing-code/) and, with the help of HN commenters, collected a small list of ways people are working to help with code visualization. I don't think most of them are production ready (some are just research papers), but you may find them interesting all the same.

    SoftVis3D (https://softvis3d.com/): where a "‘code city’ view provides a visualization for the hierarchical structure of the project".

    Code Park: A New 3D Code Visualization Tool (2017) (https://arxiv.org/pdf/1708.02174.pdf), a “novel tool for visualizing codebases in a 3D game-like environment” with code represented as “code rooms” with code on the walls.

    Code Structure Visualization Using 3D-Flythrough (2016) (https://opus-htw-aalen.bsz-bw.de/frontdoor/deliver/index/doc...), with spatial metaphors and first-person exploration of code.

    Primitive (https://primitive.io/), a VR collaboration startup with a Matrix-looking “Immersive Development Environment” with “new tools for visually analyzing software in 3D”.

    AppMap (https://appland.com/docs/how-to-use-appmap-diagrams.html), an automated code analysis tool that includes dependency maps and trace views.

    plurid (https://github.com/plurid/plurid), a framework for visualizing and debugging code in a 3D explorable structure.

    fsn (file manager) (https://en.wikipedia.org/wiki/Fsn_(file_manager)), an experimental application to view a file system in 3D (featured in Jurassic Park).

  • LookAtThat

    Render source code in 3D, for macOS and iOS.

  • If I may be so bold as include something I’m working on myself, I’d love to chime in! I’m not functionally complete, but feel like I’m writing an amalgamation of all of the above tools.

    https://github.com/tikimcfee/LookAtThat

  • spekt8

    Visualize your Kubernetes cluster in real time

  • In my experience it tends not to be done because there's an inevitable drift between the system as it actually operates and the relevant visualizations and diagrams. I have seen them used as a starting off point at times, although that's also been infrequent.

    There is however a growing movement of being able to visualize how a system is functioning at various levels. XState/Statecharts are a good example (https://xstate.js.org/viz/). Another example in the Ops space would be https://github.com/spekt8/spekt8 for K8S. I work at Grafana and we're more or less trying to expose these things in ways that make sense. Our bread and butter is timeseries data but we're adding more in that regard (it's possible to build node graphs from running systems).

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
  • codebase-visualizer-action

    Visualize your codebase during CI.

  • My friend and I have been working on https://www.codeatlas.dev in our spare time, which is a tool that creates pretty (2D!) visualisations of codebases, while providing additional insights via overlays (e.g. commit density, programming language). For example here's the Kubernetes codebase visualised using codeatlas: https://www.codeatlas.dev/repo/kubernetes/kubernetes.

    At the moment, codeatlas is only a static gallery, but we're currently about 1-2 weekends away from releasing a Github action that deploys this diagram on github pages for your own repos - if you're interested, feel free to watch this repo: https://github.com/codeatlasHQ/codebase-visualizer-action

  • backstage

    Backstage is an open platform for building developer portals

  • Not a large systems, but we started using https://backstage.io to help visualize our software. Additionally, we're hoping to leverage backstage plugins to provide tooling around ownership for our components.

  • Sourcetrail

    Discontinued Sourcetrail - free and open-source interactive source explorer

  • > So, is it a good idea to try documenting the code design through some sort of visualization?

    Yes, if it helps you understand how it works and how the pieces fit together.

    No, if the previous is not all that useful for you (different types of learners), or you need to spend significant amounts of time doing it manually, especially given that code could change.

    If you can, look into any tool that might allow you to get visualizations in an automated manner.

    For example, JetBrains IDEs have a few different graph visualizations for dependencies and inheritance etc.: https://www.jetbrains.com/help/idea/2022.1/tests-in-ide.html...

    There also used to be SourceTrail, though sadly the project is now retired: https://github.com/CoatiSoftware/Sourcetrail

    For databases, you can also use external tools like DbVis: https://www.dbvis.com/features/

    There are also a few tools here and there for visualizing networks or how container deployments look, but those are pretty situational/specific for each platform/setup.

  • depict

    depict – share mental models better (by mstone)

  • You might find it helpful to distinguish between visualizing the design of the system being implemented by your software, visualizing protocols being implemented by your software, visualizing the design of the your software itself, and visualizing important implementation details at runtime, e.g. for debugging, profiling, and operations.

    For visualizing system designs, you should take a look at STAMP, e.g., via “Engineering A Safer World” + the resources at mit.edu/psas + on YouTube.

    (Multiple tools, both commercial and libre, exist and are being developed to make these diagrams, although for what it’s worth, I mostly hear about people making them using draw.io, Google Drawings, on physical paper/whiteboards, or occasionally with specialized tooling.

    I have also recently published a project in this area, https://github.com/mstone/depict, which I believe is well on its way toward addressing some unmet needs here.)

    For visualizing protocols, things like sequence diagrams, data flow diagrams, DRAKON flow charts, value stream maps, and occasional more specialized objects like CPSA “cryptographic protocol shapes” / strand space skeletons are where I start depending on the flavor of what’s needed.

    For visualizing the design of implementations themselves, I have not yet seen anything that I feel obliged to recommend; rather, here, I suggest investing in adding illustrations to your existing documentation in whatever way is easiest for you to use to clarify whatever subtleties you need to clarify for your audience.

    (Here I tend to look at things like ASCII-art, SQLite’s railroad diagrams (now made with pikchr, AIUI), and sequence diagrams, as mentioned by other commenters, as helpful examples to start with.)

    Finally, for implementing debugging/profiling/operational illustrations, there is a such a rich set of examples to turn to — whether from the very specialized (custom process model video rendering pipelines in robotics) to TensorBoard for TensorFlow to general-purpose tools like browser performance debugging suites, flame charts, or Go’s built-in profile graphing tools - that rather than learn any particular such tools, I’d instead suggest trying to get comfortable with the building blocks underlying these systems, which include contemporary GUI/web apps, custom drawing and animation tools like SVG, pretty printers, and Grammar-of-Graphics systems like vega-lite.

    (Note: although it may seem superficially extraneous to your question, the reason I also suggest thinking about debugging visualizations in this context is because IMO, to work, they ~necessarily encode a visual model of the design of your implementation since it is the design of the implementation that provides the vocabulary and relationships that have to be understood and navigated in order to successfully debug/optimize/monitor any given running instance of whatever system you are building.)

  • C4-PlantUML

    C4-PlantUML combines the benefits of PlantUML and the C4 model for providing a simple way of describing and communicate software architectures

  • If you're interested in diagramming system architecture in a top-down fashion, I'll pile on another recommendation to check out the C4 model[0].

    As others have mentioned, to effectively communicate a system you must limit the context to particular layers of abstraction, and C4 is a good approach to doing just that. There's also a C4 plugin for PlantUML[1].

    But don't forget that, as with all visualizations, audience and purpose are key.

    Consider whether you are addressing short-term needs (eg identifying inefficiencies, modeling for a client pitch) or long-term needs (eg knowledge retention, managing complexity). If your audience's needs are short-term, you can certainly get by with much simpler tools (eg Inkscape, excalidraw/draw.io/etc, picture of a whiteboard, doodles on a napkin).

    Also consider whether or not you actually have a problem better served by bottom-up (ie generated) visualizations (eg ERDs for database schema refactoring, heatmaps for profiling, GraphViz for debugging DAGs).

    [0] https://c4model.com/

    [1] https://github.com/plantuml-stdlib/C4-PlantUML

  • saddle-data-graph

    where does it come from, where does it go?

  • I took a stab at writing my own UML-type diagram in Python using networkx and rendered with dot. Since it's a real network instead of just pictures, I can slice it however I want it (eg "show me all dependencies of page X" or "show me all nodes of type 'state'"

    https://github.com/hammeiam/saddle-data-graph/blob/master/Sa... (scroll down for images)

  • Pythonocc-nodes-for-Ryven

    Pythonocc nodes for Ryven

  • "What does an algorithm look like?"

    I'm an intensely visual person, but have never found a visual programming system which scales well --- the problem is, past a certain level of complexity one has to use modules, which then devolves the visual representation down to just a bunch named blocks.

    That said, I'm using BlockSCAD:

    https://www.blockscad3d.com/community/projects/1421975

    to work up designs which I'm then putting into other tools.

    Looking at GraphSCAD:

    http://graphscad.blogspot.com

    and there's also Ryven and pythonocc which I managed to get installed:

    https://ryven.org

    https://github.com/Tanneguydv/Pythonocc-nodes-for-Ryven

    but I'd really like to see a tool for this sort of thing which made G-code.

  • Ryven

    Flow-based visual scripting for Python

  • "What does an algorithm look like?"

    I'm an intensely visual person, but have never found a visual programming system which scales well --- the problem is, past a certain level of complexity one has to use modules, which then devolves the visual representation down to just a bunch named blocks.

    That said, I'm using BlockSCAD:

    https://www.blockscad3d.com/community/projects/1421975

    to work up designs which I'm then putting into other tools.

    Looking at GraphSCAD:

    http://graphscad.blogspot.com

    and there's also Ryven and pythonocc which I managed to get installed:

    https://ryven.org

    https://github.com/Tanneguydv/Pythonocc-nodes-for-Ryven

    but I'd really like to see a tool for this sort of thing which made G-code.

  • shotglass

    Tools to visualize large code bases in different ways.

  • Very interesting! I'm convinced humans looking at code plots can see things that computers can't. An extension of your idea is to show how code changes over time. Sections that don't change much = "backbone" of system, probably bug-free. New code, or code that changes a lot = "sketchy", might have bugs. Alternatively, show code colored by "quality" i.e. complexity.

    Here's my take: https://github.com/johntellsall/shotglass#demo-flask-a-small...

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts

  • Using Rust to speed up 3D rendering in the browser

    3 projects | /r/rust | 16 Mar 2022
  • This Week In Python

    5 projects | dev.to | 10 May 2024
  • Google lays off its Python team

    4 projects | news.ycombinator.com | 27 Apr 2024
  • Yes, Python and Matplotlib can make pretty charts

    3 projects | news.ycombinator.com | 16 Apr 2024
  • Graph Data Fits in Memory

    1 project | news.ycombinator.com | 15 Apr 2024