Resque
virtual-kubelet
Our great sponsors
Resque | virtual-kubelet | |
---|---|---|
5 | 10 | |
9,388 | 4,080 | |
0.1% | 1.0% | |
4.1 | 6.7 | |
5 months ago | 7 days ago | |
Ruby | Go | |
MIT License | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Resque
-
Mike Perham of Sidekiq: “If you build something valuable, charge money for it.”
The free version acts exactly like Resque, the previous market leader in Ruby background jobs. If it was good enough reliability for GitHub and Shopify to use for years, it was good enough for Sidekiq OSS too.
Here's Resque literally using `lpop` which is destructive and will lose jobs.
https://github.com/resque/resque/blob/7623b8dfbdd0a07eb04b19...
-
Add web scraping data into the database at regular intervals [ruby & ror]
You can use a background job queue like Resque to scrape and process data in the background, and a scheduler like resque-scheduler to schedule jobs to run your scraper periodically.
-
How to run a really long task from a Rails web request
So how do we trigger such a long-running process from a Rails request? The first option that comes to mind is a background job run by some of the queuing back-ends such as Sidekiq, Resque or DelayedJob, possibly governed by ActiveJob. While this would surely work, the problem with all these solutions is that they usually have a limited number of workers available on the server and we didn’t want to potentially block other important background tasks for so long.
-
Building a dynamic staging platform
Background jobs are another limitation. Since only the Aha! web service runs in a dynamic staging, the host environment's workers would process any Resque jobs that were sent to the shared Redis instance. If your branch hadn't updated any background-able methods, this would be no big deal. But if you were hoping to test changes to these methods, you would be out of luck.
-
Autoscaling Redis applications on Kubernetes 🚀🚀
Redis Lists are quite versatile and used as the backbone for implementing scalable architectural patterns such as consumer-producer (based on queues), where producer applications push items into a List, and consumers (also called workers) process those items. Popular projects such as resque, sidekiq, celery etc. use Redis behind the scenes to implement background jobs.
virtual-kubelet
-
Bare-Metal Kubernetes, Part I: Talos on Hetzner
Speaking of k8s, anyone here know of ready-made solutions for getting XCode (i.e. xcodebuild) running in pods? As far as I'm aware, there are no good solutions for getting XCode running on Linux, so at the moment I'm just futzing about with a virtual-kubelet[0] implementation that spawns MacOS VMs. This works just fine, but the problem seems like such an obvious one that I expect there to be some existing solution(s) I just missed.
[0]:https://github.com/virtual-kubelet/virtual-kubelet/
-
Keeping Airflow tasks “cloud-native”
Have you looked into virtual kubelet yet? It allows you to make a virtual node in your on-prem cluster that schedules workloads on services like AWS Fargate or Azure Container Instances.
-
Similar to AWS Fargate provider?
If you are serious about implementing this yourself, you may want to look into virtual kubelet: https://virtual-kubelet.io/
- Nomad vs. Kubernetes
-
Deploy on prem Kubernetes. What is the best approach paid and unpaid to deploy a cluster on premise with burst to azure/aws? The only need is the ability to have some static pods. I do have a preference for free/open source solutions.
I just stumbled upon this project a while back and don't have experience with it, so I don't know how well it works and what caveats you may face, but there's Virtual Kubelet, which aims to do just that, i.e. running a virtual Kubernetes node outside the cluster. Its Kip provider sounds like the thing you're looking for.
-
Create a fake node
Are you looking something like the following: https://github.com/virtual-kubelet/virtual-kubelet
-
How to use the GitOps model to create, update and manage applications at the edge with KubeEdge and Argo
Kubeedge docs are light on self-justification... How does https://github.com/kubeedge/kubeedge differ from https://github.com/virtual-kubelet/virtual-kubelet or just running a regular kubelet on that edge machine?
-
Autoscaling Redis applications on Kubernetes 🚀🚀
If this sounds interesting, do check out Virtual Nodes in Azure Kubernetes Service to see how you can use them to seamlessly scale your applications to Azure Container Instances and benefit from quick provisioning of pods, and only pay per second for their execution time. The virtual nodes add-on for AKS, is based on the open source project Virtual Kubelet which is an open source Kubernetes kubelet implementation.
-
Infrastructure Engineering - Diving Deep
Use cases like these are made possible by projects like KubeEdge , K3s and Virtual Kubelets. You can read more about how they power the edge with different architectures and compromises here.
-
Evolving Container Security with Linux User Namespaces
This is a complicated question to answer.
This isn't my expertise (the cluster orchestration system), but I can answer to the best of my abilities: Titus, today is a system that sits on top of Kubernetes, and uses Kubernetes components to do its thing, but we've substituted many of the systems with our own. For example, closer to my area of knowledge, we've used our own executor / provider along with the Virtual Kubelet project (https://github.com/virtual-kubelet/virtual-kubelet) instead of Kubelet.
We're exploring where we can leverage the Kubernetes ecosystem, adapt components, or help contribute changes back that others can leverage to enable our use of more COTS components of Kubernetes.
tl;dr: We're swapping out the engines while in flight
What are some alternatives?
Sidekiq - Simple, efficient background processing for Ruby
kubeedge - Kubernetes Native Edge Computing Framework (project under CNCF)
Shoryuken - A super efficient Amazon SQS thread based message processor for Ruby
kubevirt - Kubernetes Virtualization API and runtime in order to define and manage virtual machines.
RabbitMQ - Open source RabbitMQ: core server and tier 1 (built-in) plugins
kubefed - Kubernetes Cluster Federation
Sneakers - A fast background processing framework for Ruby and RabbitMQ
keda - KEDA is a Kubernetes-based Event Driven Autoscaling component. It provides event driven scale for any container running in Kubernetes
Sucker Punch - Sucker Punch is a Ruby asynchronous processing library using concurrent-ruby, heavily influenced by Sidekiq and girl_friday.
cni - Container Network Interface - networking for Linux containers
good_job - Multithreaded, Postgres-based, Active Job backend for Ruby on Rails.
charts - ⚠️(OBSOLETE) Curated applications for Kubernetes