single-spa VS Pandas

At JupiterOne, we've made a significant transition from a monolithic frontend to a distributed frontend architecture using microfrontends, powered by (single-spa-react)[https://single-spa.js.org/]. This architectural shift has brought many benefits, such as lazy loading sections of the app and accelerating our development cycle. However, like any significant change, it has also come with its set of challenges.

Single-SPA Framework - One of the most popular choices. It has a large community behind it and offers robust routing and lifecycle management for microapps. Ref - https://single-spa.js.org/

Joel Denning is the visionary behind single-spa, boasting profound insights into the true mechanics of the web and is, in my view, a pioneer in micro frontend architecture. I highly recommend checking the single-spa website and his YouTube channel. While the videos might appear dated at first glance, rest assured, Joel is ahead of his time, and the content remains incredibly relevant today.

You got me curious and I found that single spa is something new to me - if I read https://github.com/single-spa/single-spa then I reckon you can use Svelte(Kit) in such framework - like you can use angular etc

I’ve actually done that “web components as the interoperability layer”!

It was this old angular 1.8 app with new features being written in angular-hybrid-ized angular 8. Ripping out angular-hybrid and separating the angular 1.8 routes from the modern angular routes was difficult, but now they were totally separate.

The only thing linking them together was an object with some RXJS streams in it for state, and a little in-house wrapper “app” who just loaded one component or another depending on the URL and a hash of routes for config. (I’d probably use SingleSPA [1] now. Same thing really.)

We could deploy them separately since the build just ends up being another JS file somewhere that just gets included with a script tag at runtime. No version bumping! No big mega build!

We started replacing the remaining “old” routes 1 by 1 with a “new” counter part. That was the easiest part, and went at a pace devs were comfortable with (fast enough) and business folk could tolerate. (modular enough to not HAVE to be done all at once)

Last I checked, the angular 1.8 stuff is gone years ago. :)

[1] https://single-spa.js.org/

Module Federation is not the unique solution, for example single-spa

You may want to look into using single-spa to run Vue 2 and Vue 3 side by side while you're migrating. Once your app is fully converted, you can stop using single-spa

Single-SPA and module federation are potent tools for building scalable micro-frontends. Single-spa provides a framework for building a modular front-end application. It allows lazy loading of micro-frontends, which helps improve the application's performance. In contrast, module federation enables communication and dependency sharing between different micro-frontends. It reduces code duplication, which can help with scaling an application.

Python is a natural fit for serverless development. It boasts a vast array of libraries, including Powertools for AWS and robust libraries for data engineers. Its versatility and excellent developer experience make it a top choice for serverless projects, offering a seamless and enjoyable development experience.

In data analysis, managing the structure and layout of data before analyzing them is crucial. Python offers versatile tools to manipulate data, including the often-used Pandas reset_index() method.

Dash is a Python framework that enables you to build interactive frontend applications without writing a single line of Javascript. Internally and in projects we like to use it in order to build a quick proof of concept for data driven applications because of the nice integration with Plotly and pandas. For this post, I'm going to assume that you're already familiar with Dash and won't explain that part in detail. Instead, we'll focus on what's necessary to make it run serverless.

there is pull request to integrate in both pydantic extra types and into pandas cose [1]

[1]: https://github.com/pandas-dev/pandas/issues/53999

Last year I worked through the challenges using VisiData, Datasette, and Pandas. I walked through my thought process and solutions in a series of posts.

pandas: A library for data analysis in Python

Data analysis involves scrutinizing datasets for class imbalances or protected features and understanding their correlations and representations. A classical tool like pandas would be my obvious choice for most of the analysis, and I would use OpenCV or Scikit-Image for image-related tasks.

Pandas, a powerful data manipulation library in Python, has become an essential tool for data scientists and analysts. One of its key functions is read_csv(), which allows users to read data from CSV (Comma-Separated Values) files into a Pandas DataFrame. In this tutorial, brought to you by CodesWithPankaj.com, we will explore the intricacies of read_csv() with clear examples to help you harness its full potential.

So, what I'd like to do is write a documentation package in Python to recreate what I've lost. I plan to build upon the fantastic python-docx and docxtpl packages, and I'll probably rely on pandas from much of the tabular stuff. Here are the features I intend to include:

Weirdly most of my time spent with data analysis was in the C layers in pandas.