ialacol VS lens

# This is the configuration file for DevSpace # # devspace use namespace private-ai # suggest to use a namespace instead of the default name space # devspace deploy # deploy the skeleton of the app and the dependencies (ialacol) # devspace dev # start syncing files to the container # devspace purge # to clean up version: v2beta1 deployments: # This are the manifest our private app deployment # The app will be in "sleep mode" after `devspace deploy`, and start when we start # syncing files to the container by `devspace dev` private-ai-app: helm: chart: # We are deploying the so-called Component Chart: https://devspace.sh/component-chart/docs name: component-chart repo: https://charts.devspace.sh values: containers: - image: ghcr.io/loft-sh/devspace-containers/python:3-alpine command: - "sleep" args: - "99999" service: ports: - port: 8000 labels: app.kubernetes.io/name: private-ai-app ialacol: helm: # the backend for the AI app, we are using ialacol https://github.com/chenhunghan/ialacol/ chart: name: ialacol repo: https://chenhunghan.github.io/ialacol # overriding values.yaml of ialacol helm chart values: replicas: 1 deployment: image: quay.io/chenhunghan/ialacol:latest env: # We are using MPT-30B, which is the most sophisticated model at the moment # If you want to start with some small but mightym try orca-mini # DEFAULT_MODEL_HG_REPO_ID: TheBloke/orca_mini_3B-GGML # DEFAULT_MODEL_FILE: orca-mini-3b.ggmlv3.q4_0.bin # MPT-30B DEFAULT_MODEL_HG_REPO_ID: TheBloke/mpt-30B-GGML DEFAULT_MODEL_FILE: mpt-30b.ggmlv0.q4_1.bin DEFAULT_MODEL_META: "" # Request more resource if needed resources: {} # pvc for storing the cache cache: persistence: size: 5Gi accessModes: - ReadWriteOnce storageClass: ~ cacheMountPath: /app/cache # pvc for storing the models model: persistence: size: 20Gi accessModes: - ReadWriteOnce storageClass: ~ modelMountPath: /app/models service: type: ClusterIP port: 8000 annotations: {} # You might want to use the following to select a node with more CPU and memory # for MPT-30B, we need at least 32GB of memory nodeSelector: {} tolerations: [] affinity: {}

You might be wondering why running Kubernetes is necessary for this project. This article was actually created during the development of a testing CI for the OSS project ialacol. The goal was to have a basic smoke test that verifies the Helm charts and ensures the endpoint returns a 200 status code. You can find the full source of the testing CI YAML here.

We are deploying a Helm release orca-mini-3b using Helm chart ialacol

Lens

Generally I felt as if I was diving in the deepest of waters without the correct equipement and that was horrifying. Unfortunately to me, I had to dive even deeper before getting equiped with tools like ArgoCD, and k8slens. I had to start working with... HELM.

Indeed you can, with several "paid" features removed, like log tailing and pod shells. They deliberately hobbled the product. If you want to use Lens, my advice is pay for the supported version.

yes I know there is lens, but it does not allow me to see logs of multiple pods at same time and what is even more important it is not friendly for ephemeral clusters - in my case with help of kind I am recreating whole cluster each time from scratch

Let's wait for few seconds for the pods to become green, I am using Lens, it's awesome btw.

You probably don't need Rancher unless you need a GUI or manage multiple clusters, Lens or k9s might be a better fit for your use case.

`openlens` is now preferred over `Lens`, it has everything you need and none of the fluff that Lens wants to charge you for.