promptfoo VS cloudseeder

Jumping in because I'm a big believer in (1) local LLMs, and (2) evals specific to individual use cases.

[0] https://github.com/typpo/promptfoo

Congrats on the launch!

I've been interested in automatic testset generation because I find that the chore of writing tests is one of the reasons people shy away from evals. Recently landed eval testset generation for promptfoo (https://github.com/typpo/promptfoo), but it is non-RAG so more simplistic than your implementation.

Was also eyeballing this paper https://arxiv.org/abs/2401.03038, which outlines a method for generating asserts from prompt version history that may also be useful for these eval tools.

Thanks for the promptfoo mention. For anyone else who might prefer deterministic, programmatic evaluation of LLM outputs, I've been building promptfoo: https://github.com/typpo/promptfoo

Example asserts include basic string checks, regex, is-json, cosine similarity, etc.

We made a simple way to lock QBittorrent into a VPN in a container [1]. It's probably simple enough to follow what we did in the config script to set it up for your use case (all open source [2]).

[1] https://www.youtube.com/watch?v=PrH6Ci_4eig

[2] https://github.com/ipv6rslimited/cloudseeder

You should never use an LLM/AI Chatbot with a third party service for anything that could be confidential, private in nature, or may have to do with your security.

All of the LLM/AI providers can read your contents. The only thing between them and your chats is "trust," and we've all had promises broken on us before.

Trust is not security.

Shameless plug: You can definitely use IPv6.rs and Cloud Seeder [1][2] to run Ollama and OpenWebUI on your own computer (confidentially) and access it via TLS remotely (via phone, etc.).

[1] https://ipv6.rs/cloudseeder

This was a tremendous write-up. I appreciate the detail including your ingressd setup. I agree, though, that this is a pain. It's for this reason that we made Cloud Seeder [1] so you can have hands-free setup of your homelab and IPv6rs for painless ingress [2]

[1] https://github.com/ipv6rslimited/cloudseeder

[2] https://ipv6.rs

Submission: This is a simple terminal for 'output/read-only' usage in a Fyne GoLang app. You can basically run a command and it will feel like its running in a terminal and the output will go to said terminal. This strips out all ansi, escape and control codes.

It's part of Cloud Seeder [1] which we wrote open source and in a way so that everyone can use the parts [2][3] easily.

[1] https://github.com/ipv6rslimited/cloudseeder - 1 click host with hands free upgrades your favorite self hosted appliance

[2] https://github.com/ipv6rslimited/configurator - make a config screen from a json

[3] https://github.com/ipv6rslimited/tray - make a system tray from a json

Ollama is really organized - it relies on llama but the UX and organization it provides makes it legit. We recently made a one-click wizard to run Open WebUI and Ollama together, self hosted and remotely accessible but locally hosted [1]

[1] https://github.com/ipv6rslimited/cloudseeder

Salutations HN,

We're excited to introduce Cloud Seeder, a tool designed to make self-hosting easy.

Problem Identification:

We realized there were many discussions about self-hosting but they never transitioned into action. After surveying potential users, two major obstacles emerged:

1. Lack of an external IP address.

2. The complexity of setting up and maintaining a server.

Our Solution - Cloud Seeder:

To make self-hosting straightforward, we developed Cloud Seeder. It's a platform that supports one-click deployment for over 20+ different appliances (and growing upon request), ranging from Mastodon (a federated social network that communicates with Meta's Threads) to Nextcloud (a robust, self-hosted alternative to major collaboration platforms).

Cloud Seeder is fully open source. The code is available on GitHub [1].

We believe Cloud Seeder will play a crucial role in fostering a more decentralized internet.

We hope you'll give this a try and start self hosting!

We also include a Configurator [2] and Tray [3] on github which lets you dynamically (from just a json) create dialog boxes and system trays. We use this in our application and thought it might be useful to people who don't want to spend time on GUI but would love to reach people with such UX.

[1] https://github.com/ipv6rslimited/cloudseeder

[2] https://github.com/ipv6rslimited/configurator

[3] https://github.com/ipv6rslimited/tray