#Microservices

Open-source projects categorized as Microservices | Edit details

Top 23 Microservice Open-Source Projects

  • GitHub repo nodebestpractices

    :white_check_mark: The Node.js best practices list (April 2021)

    Project mention: Any resources on best structuring practices for Express projects? | reddit.com/r/node | 2021-04-29

    node best practices

  • GitHub repo Nest

    A progressive Node.js framework for building efficient, scalable, and enterprise-grade server-side applications on top of TypeScript & JavaScript (ES6, ES7, ES8) 🚀

    Project mention: Best React + NodeJS tech stacks in 2021? | reddit.com/r/webdev | 2021-05-06

    NestJS -> Opinionated Node.js web framework. Build on top of ExpressJS, can be configured to use Fastify

  • GitHub repo Kong

    🦍 The Cloud-Native API Gateway

    Project mention: Can someone please help on this query? | reddit.com/r/kong_gateway | 2021-04-30
  • GitHub repo istio

    Connect, secure, control, and observe services.

    Project mention: How to avoid cascading failures using Circuit Breaker? | dev.to | 2021-05-02

    Istio Service Mesh

  • GitHub repo go-kit

    A standard library for microservices.

    Project mention: Show HN: Encore – Go framework for distributed systems | news.ycombinator.com | 2021-04-14
  • GitHub repo Nacos

    an easy-to-use dynamic service discovery, configuration and service management platform for building cloud native applications.

  • GitHub repo eShopOnContainers

    Cross-platform .NET sample microservices and container based application that runs on Linux Windows and macOS. Powered by .NET 5, Docker Containers and Azure Kubernetes Services. Supports Visual Studio, VS for Mac and CLI based environments with Docker CLI, dotnet CLI, VS Code or any other code editor.

    Project mention: Hey Rustaceans! Got an easy question? Ask here (18/2021)! | reddit.com/r/rust | 2021-05-03

    Please create a repo like eShopOnContainers to explain how to use rust to create a highly available and highly concurrent back-end service. It's just my naive suggestion, because I am a front-end developer and I don't know how to build a production-level, highly available and highly scalable back-end server. If there is a similar repo, it will be very useful

  • GitHub repo go-micro

    Go Micro is a framework for distributed systems development

    Project mention: Why and how we built Distributed with Next.js and Micro | dev.to | 2021-03-26

    Distributed is a live social chat app built as a Jamstack demo using Next.js and Micro. We built it to demonstrate the value proposition of M3O - a cloud platform for API development. This post explains what went into building Distributed in just a few weeks and how M3O helped rapidly build our MVP. You can find the source code for distributed on Github. If you want to build and host your own version signup to M3O and start running the same services from our open source repository micro/services. Why we built Distributed Distributed was built as a jamstack demo to show how you could leverage M3O as an API backend for rapid MVP development. M3O itself is a cloud platform for API development, built on the popular open source project Micro. Micro enables you to quickly build APIs in Go on the backend and M3O provides simple free hosting of those services. We wanted to show the Jamstack audience how you could quickly leverage those APIs to build something on the frontend. Not only that, we really wanted to understand and experience the frontend developers perspective through dogfooding of our own APIs rather than just throwing stuff over the wall and hoping it works. Hopefully in that we we've done is demonstrate the value of our platform and how others can also make use of it with a real world app like Distributed to learn from. Let's talk more about Jamstack now. Jamstack Development Jamstack is a new architecture pattern for frontend which offloads dynamic aspects of the stack to javascript and third party APIs. Vercel, the makers of Next.js and related companies are pioneering the way forward for jamstack development. Credit jamstack.org JAMstack stands for Javascript, API and Markup. The static part of the application is deployed to a CDN with javascript dynamically loading various pieces of dynamic content from backend APIs and rendering it. Why we chose Next.js Next.js is a massively popular react based framework for Jamstack development. When we were looking at building out a demo on top of M3O we had the choice of going down a number of routes but what really appealed to us was the how deliberate a lot of the choices were in how the Vercel team had constructed the Next.js framework. Being framework creators ourselves with the dominant framework Go Micro for Go, we could appreciate the efforts required and strong opinions needed to drive such adoption and success. Vercel has done a phenomenal job in this way. Beyond praising Vercel's efforts. The Next.js framework includes a lot of key components needed for the Jamstack including server side rendering, api routes and typescript support. For us these were mandatory feature requirements when building against not only our APIs but third party providers. Micro for the Backend Micro is an open source cloud platform for API development. With modern day complexity in writing software for the cloud, Micro has attempted to distill that down to a handful of primitives and a framework for building services in Go. Micro took learnings from the original Go Micro framework and focused on not just gRPC based service development but actually packaging together a runtime and platform which exposes those services automatically as APIs. What this means is we can write microservices on the backend using gRPC and protobuf and immediately provide value to consumers and clients on the frontend via HTTP/JSON. To learn more about that check out the project at micro.mu or the hosted platform at m3o.com. Building on Netlify We initially chose Netlify for hosting as we saw many people adopting it for Jamstack apps. Initially this proved really great for static content. As our apps got more complex and we started to build out the Distributed demo we found Netlify no longer scaled with our basic needs. The first example we can share is Netlify Functions for Next.js API routes. Next.js routes can be turned into Netlify Functions which are essentially hosted as AWS Lambda functions. It's a clever way of pushing certain requirements to the server side, like calling third party APIs with keys you don't want to expose to the client. Next.js is great in this regard and plugins like netlify-plugin-nextjs and next-on-netlify let us do this really quickly but the performance left a lot to be desired. Our APIs are primarily hosted in London on DigitalOcean and while Netlify has a CDN for static content, the Lambda functions are deployed in a single region in US-East on AWS. For those who've suffered this pain you know exactly what that means. We were making cross atlantic calls from JS in the client to api routes on lambda and then finally to our apis. Needless to say this didn't scale for us. We weren't able to reach out the Netlify team to get help and so in frustration had to go down the self hosted route. Note we did test out Vercel and found the experience to be faster but self hosting on DigitalOcean just made more sense for our demo needs. Switching to Self Hosted One of the things DigitalOcean now provides is App Platform, a container hosting solution which lets you pick regions, does TLS certificate management for your custom domain and automatic builds from Git. This turned out to be a perfect solution for self hosted Next.js apps. Next.js at the end of the day is a React and node.js based application. As much as you may want to separate out the static content to something like Netlify and functions on Lambda, it equally just makes sense to host the entire thing in one place and run many copies of it much like we did in the old php and rails days. Because the APIs are colocated with the frontend we find this experience fairly fast, sub 100ms for all the API calls but we know it's not an ideal demonstration of the Jamstack's architecture and so we'll be working towards hosting on Vercel in the future to showcase that experience. How It All Fits Together We're running Distributed as a Next.js application on the frontend talking to Micro APIs on the backend. All of this is constructed as API routes in Next.JS firing requests at our M3O platform and the various APIs we need. Let's walk through a typical request. For example, when loading a group we need to get the group info, user profile, chats and more. We could do this as a GraphQL APIbut that would require too much stitching together in terms of the schema on the backend. Instead we're using protobuf and RPC for rapid development there and Micro automagically exposes that as a HTTP/JSON API. So a typical flow is like so. Frontend makes a request to /api/groups/[id] which loads the api code in the Next.js app We validate the user is logged in by calling the /users/Validate endpoint and ff authenticated load the group data by id using /groups/Read Skipping ahead, we'll load group messages via /threads/ListConversations and private messages using /chats/ListMessages We can check for already read messages via a "seen" API and then subscribe to the streams API for instant messaging Finally we render everything based on the content loaded for the user .gist-data { max-height: 600px; overflow: auto;} Here's a code "snippet" for those interested. From an MVP standpoint this is just a very quick and rapid way for us to build against numerous separate APIs on the backend all hosted in the same place. For anyone interested in the "call" function. It's simply a small function we're using to call the Micro APIs on the backend. Remember Micro turns any RPC based service into a HTTP/JSON API automatically via an API gateway. M3O provides hosting for all this. Performance & Productivity Aside from the structural benefits of a framework like Next.js we find it really unlocks significant productivity by providing an opinionated approach to frontend development. That coupled with Micro on the backend and our APIs hosted on M3O it's enabled us to rapidly ship this MVP within the space of 4-6 weeks with mostly 1 person doing the work. That really speaks to the power of the combination of Next.js and Micro. For this demo we built APIs for user management, group messaging, websocket streaming, sending invite emails and audio/video through Twilio WebRTC. One can only imagine where it would go with a dedicated team and full product focus. On the performance side, Next.js is blazingly fast by all measures. Whether it be the local reload for development or the server side rendering. It all adds to a really snappy experience on both the development and consumption side of things. With the backend we tried to pair this with Go based APIs written with Micro to ensure not just speed of development but also speed of delivery. All in all, we think Next.js and Micro are the perfect pairing for any Jamstack and API based development. Conclusions Thanks for reading this post on how we built Distributed on the Jamstack using Next.js and Micro. Find the source code for distributed on Github. If you want to build and host your own version signup to M3O and start running the same services from our open source repository micro/services. Reach out on slack or twitter if you have any questions.

  • GitHub repo grpc-go

    The Go language implementation of gRPC. HTTP/2 based RPC

    Project mention: Killing PubSub connection when client of server side stream dies in go | reddit.com/r/golang | 2021-04-30
  • GitHub repo dapr

    Dapr is a portable, event-driven, runtime for building distributed applications across cloud and edge.

    Project mention: Show HN: Encore – Go framework for distributed systems | news.ycombinator.com | 2021-04-14

    Is this something like https://dapr.io/?

  • GitHub repo kratos

    A Go framework for microservices.

    Project mention: Don't Panic: Catching Panics in Errgroup | dev.to | 2021-01-16

    Kratos errgroup

  • GitHub repo backstage

    Backstage is an open platform for building developer portals

    Project mention: Ask HN: How do you keep track of releases/deployments of dozens micro-services? | news.ycombinator.com | 2021-04-20
  • GitHub repo jib

    🏗 Build container images for your Java applications.

    Project mention: Is Docker Dead in the Water? | reddit.com/r/programming | 2021-05-07

    The docker utility isn't the only way to build and run containers. There's also cri-o, podman, and crun among others for running containers. For building there is podman again, Jib for Java applications, and bazel plus many others. The docker approach of using a client to connect to a daemon required to run as root has turned out to be slow and insecure.

  • GitHub repo micro

    Distributed OS built for the Cloud (by micro)

    Project mention: Micro - A platform for microservices development | reddit.com/r/microservices | 2021-04-16
  • GitHub repo awesome-microservices

    A curated list of Microservice Architecture related principles and technologies.

  • GitHub repo chi

    lightweight, idiomatic and composable router for building Go HTTP services

    Project mention: Feasible to use labstack/echo with a jamstack? | reddit.com/r/golang | 2021-04-02

    Quick tip regarding the use of echo. Unlike many other language ecosystems (node, ruby, python, etc.), web frameworks on frowned upon by the go community and not commonly used in practice. Much of the ecosystem is designed to interoperate directly with the http package types... wrapping them with echo will inhibit this interpretability and carry few benefits. That said, routing specifically can be tricky, so checkout the https://github.com/go-chi/chi router.

  • GitHub repo qiankun

    📦 🚀 Blazing fast, simple and completed solution for micro frontends.

    Project mention: Choosing a Micro Frontend Framework | dev.to | 2021-04-04

    Others: FrintJS, qiankun, Berial, and Nuz

  • GitHub repo single-spa

    The router for easy microfrontends

    Project mention: Mount light weighted apps into Ember | dev.to | 2021-04-27

    Well, this can be achieved with certain router packages​ like https://single-spa.js.org. However, let's try a different approach. For demo, let's choose Ember and Preact and define two routes in Ember

  • GitHub repo falcon

    The no-nonsense, minimalist REST and app backend framework for Python developers, with a focus on reliability, correctness, and performance at scale.

    Project mention: Do you know any Python projects on Github that are examples of best practices and good architecture? | reddit.com/r/learnpython | 2021-05-05

    This may not be exactly what you asked for but I found contributing to open source projects really exposed me to different approaches I never would have considered and may not have fully grasped had I not had to actually dive into the code to solve an issue. Falcon is a great place to start and the guys are super friendly there.

  • GitHub repo up

    Deploy infinitely scalable serverless apps, apis, and sites in seconds to AWS.

    Project mention: Ask HN: Does anyone else find the AWS Lambda developer experience poor? | news.ycombinator.com | 2021-04-18

    I can also recommend serverless.com framework for local development and running lambdas offline.

    AWS provides a docker image for emulating DynamoDB which works great for local dev and will commonly be paired with lambdas.

    Another option I have used recently for implementing node web services is https://github.com/apex/up which also has a nice local dev experience.

  • GitHub repo Activiti

    Activiti is a light-weight workflow and Business Process Management (BPM) Platform targeted at business people, developers and system admins. Its core is a super-fast and rock-solid BPMN 2 process engine for Java. It's open-source and distributed under the Apache license. Activiti runs in any Java application, on a server, on a cluster or in the cloud. It integrates perfectly with Spring, it is extremely lightweight and based on simple concepts.

  • GitHub repo Tyk

    Tyk Open Source API Gateway written in Go, supporting REST, GraphQL, TCP and gRPC protocols

    Project mention: Microservices Design - API Gateway Pattern | dev.to | 2021-01-01

    Amazon API Gateway AWS provides fully managed service for creating, publishing, maintaining, monitoring, and securing REST, HTTP, and WebSocket where developers can create APIs that access AWS or other web services, as well as data stored in the AWS Cloud. Kong API Gateway Kong Gateway is an open-source, lightweight API gateway optimized for microservices, delivering unparalleled latency performance and scalability. If you just want the basics, this option will work for you. It is scalable easily horizontally by adding more nodes. It supports large and variable workloads with very low latency. Other API Gateways Apigee API Gateway MuleSoft Tyk.io Akana SwaggerHub Azure API Gateway Express API Gateway Karken D Choosing the right API gateway Some of the common baseline for evaluation criteria include simplicity, open-source vs propriety, scalability & flexibility, security, features, community, administrative (support, monitoring & deployment), environment provisioning(installation, configuration, hosting offering), pricing, and documentation. API Composition / Aggregation Some API requests in API Gateway map directly to single service API which can be served by routing request to the corresponding microservice. However, in the case of complex API operations that requires results from several microservices can be served by API composition/aggregation (a scatter-gather mechanism). In case of dependency of one another service where synchronous communication is required, the chained composition pattern has to be followed. The composition layer has to support a significant portion of ESB/integration capabilities such as transformations, orchestration, resiliency, and stability patterns. A root container is deployed with the special distributor and aggregator functionalities (or microservices). The distributor is responsible for breaking down into granular tasks and distributing those tasks to microservice instances. The aggregator is responsible for aggregating the results derived by business workflow from composed microservice. API Gateway and Aggregation API gateway with added features results in overambitious gateways that encourage designs that continue to be difficult to test and deploy. It is highly recommended to avoid aggregation and data transformation in the API Gateway. Domain smarts are better suited to be done in application code that follows the defined software development practices. Netflix API Gateway, Zuul 2 removed a lot of the business logic from Gateway that they had in Zuul to origin systems. For more details, refer here. Service Mesh and API Gateway Service mesh in microservices is a configurable network infrastructure layer that handles interprocess communication. This is akin to what is often termed as sidecar proxy or sidecar gateway. It provides a lot of functionalities such as:

  • GitHub repo rpcx

    A zero cost, faster multi-language bidirectional microservices framework in Go, like alibaba Dubbo, but with more features, Scale easily. Try it. Test it. If you feel it's better, use it! 𝐉𝐚𝐯𝐚有𝐝𝐮𝐛𝐛𝐨, 𝐆𝐨𝐥𝐚𝐧𝐠有𝐫𝐩𝐜𝐱!

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2021-05-07.

Index

What are some of the best open-source Microservice projects? This list will help you:

Project Stars
1 nodebestpractices 64,493
2 Nest 36,669
3 Kong 28,869
4 istio 26,839
5 go-kit 20,081
6 Nacos 17,454
7 eShopOnContainers 17,410
8 go-micro 15,912
9 grpc-go 13,649
10 dapr 13,020
11 kratos 12,481
12 backstage 11,457
13 jib 10,550
14 micro 10,028
15 awesome-microservices 9,987
16 chi 9,345
17 qiankun 9,303
18 single-spa 8,969
19 falcon 8,363
20 up 8,179
21 Activiti 7,694
22 Tyk 6,211
23 rpcx 5,612