chispa
tera
Our great sponsors
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
chispa
-
Testing spark applications
Unit and e2e tests using a combination of pytest and chispa (https://github.com/MrPowers/chispa). Custom library to create random test data that fits schema with optional hardcoded overrides for relevant fields to test business logic.
-
Spark open source community is awesome
here's a little README fix a user pushed to chispa
-
Invitation to collaborate on open source PySpark projects
chispa is a library of PySpark testing functions.
-
installing pyspark on my m1 mac, getting an env error
The other approach I've used is Poetry, see the chispa project as an example. Poetry is especially nice for projects that you'd like to publish to PyPi because those commands are built-in.
-
Spark: local dev environment
- All Spark transformations are tested with pytest + chispa (https://github.com/MrPowers/chispa)
-
Pyspark now provides a native Pandas API
Pandas syntax is far inferior to regular PySpark in my opinion. Goes to show how much data analysts value a syntax that they're already familiar with. Pandas syntax makes it harder to reason about queries, abstract DataFrame transformations, etc. I've authored some popular PySpark libraries like quinn and chispa and am not excited to add Pandas syntax support, haha.
-
Show dataengineering: beavis, a library for unit testing Pandas/Dask code
I am the author of spark-fast-tests and chispa, libraries for unit testing Scala Spark / PySpark code.
-
Tips for building popular open source data engineering projects
Blogging has been the main way I've been able to attract users. Someone searches "testing PySpark", they see this blog, and then they're motivated to try chispa.
-
Ask HN: What are some tools / libraries you built yourself?
I built daria (https://github.com/MrPowers/spark-daria) to make it easier to write Spark and spark-fast-tests (https://github.com/MrPowers/spark-fast-tests) to provide a good testing workflow.
quinn (https://github.com/MrPowers/quinn) and chispa (https://github.com/MrPowers/chispa) are the PySpark equivalents.
Built bebe (https://github.com/MrPowers/bebe) to expose the Spark Catalyst expressions that aren't exposed to the Scala / Python APIs.
Also build spark-sbt.g8 to create a Spark project with a single command: https://github.com/MrPowers/spark-sbt.g8
-
Open source contributions for a Data Engineer?
I've built popular PySpark (quinn, chispa) and Scala Spark (spark-daria, spark-fast-tests) libraries.
tera
-
Getting Started with Axum - Rust's Most Popular Framework
You can also use HTML templating with crates like askama, tera and maud! This can be combined with the power of lightweight JavaScript libraries like htmx to speed up time to production. You can read more about this on our other article about using HTMX with Rust which you can find here.. We also collaborated with Stefan Baumgartner on an article for serving HTML with Askama!
- What is the current ideal choice for server-side rendered web frameworks?
-
Server-side rendering in Rust - a Dall.E use-case
Tera, based on Jinja, as the next two
-
Full-Stack-Rust: Which approach in Frontend?
Tera
-
Full-stack authentication system using rust (actix-web) and sveltekit
An authentication system is an integral part of modern applications. It's so important that almost all modern applications have some sort of it. Because of their critical nature, such systems should be secure and should follow OWAP®'s recommendations on web security and password hashing as well as storage to prevent attacks such as Preimage and Dictionary attacks (common to SHA algorithms). To demonstrate some of the recommendations, we'll be building a robust session-based authentication system in Rust and a complementary frontend application. For this article series, we'll be using Rust's actix-web and some awesome crates for the backend service. SvelteKit will be used for the frontend. It should be noted however that what we'll be building is largely framework agnostic. As a result, you can decide to opt for axum, rocket, warp or any other rust's web framework for the backend and react, vue or any other javascript framework for the frontend. You can even use rust's yew, seed or some templating engines such as MiniJinja or tera at the frontend. It's entirely up to you. Our focus will be more on the concepts.
-
Show HN: Robyn – the fastest Python web framework written in Rust
Or Flask!
My guess is that "fastest" refers to the request-response loop.
I'd be interested in knowing how fast it is once you tack your favourite template rendering engine on top.
It would be nice if it supported Tera, the Rust template engine that is inspired by Jinja2:
https://github.com/Keats/tera
-
I made a status bar generator for xmobar (and other text based bars)
supports sophisticated templating using Tera,
-
Help with warp routes
As you might've noticed I have a static www folder with all my files. If I go to /, /login, /register I want to respond with my templated HTML. If the browser asks for another file, such as index.js or something.png I want to serve it from the static folder. I someone wants to access the raw template HTML, such as index.html I want to response with a 404 message.
-
Rust for web development: 3 years later
tera for email templates.
-
a crate for rendering HTML to an image buffer?
I've been using Tera and Chromium Oxide to generate and render reports to PDF and its been very needs suiting. It can also render to a PNG file.
What are some alternatives?
spark-fast-tests - Apache Spark testing helpers (dependency free & works with Scalatest, uTest, and MUnit)
askama - Type-safe, compiled Jinja-like templates for Rust
spark-daria - Essential Spark extensions and helper methods ✨😲
handlebars-rust - Rust templating with Handlebars
quinn - pyspark methods to enhance developer productivity 📣 👯 🎉
actix-web - Actix Web is a powerful, pragmatic, and extremely fast web framework for Rust.
lowdefy - The config web stack for business apps - build internal tools, client portals, web apps, admin panels, dashboards, web sites, and CRUD apps with YAML or JSON.
minijinja - MiniJinja is a powerful but minimal dependency template engine for Rust compatible with Jinja/Jinja2
null - Nullable Go types that can be marshalled/unmarshalled to/from JSON.
maud - :pencil: Compile-time HTML templates for Rust
dagster - An orchestration platform for the development, production, and observation of data assets.
horrorshow-rs - A macro-based html builder for rust