?Best LLM service for a tiny home server

This page summarizes the projects mentioned and recommended in the original post on /r/LocalLLaMA

Our great sponsors
  • WorkOS - The modern identity platform for B2B SaaS
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • SaaSHub - Software Alternatives and Reviews
  • Home Assistant

    :house_with_garden: Open source home automation that puts local control and privacy first.

  • My plan is to make a home-server by putting a Google Coral M.2 Accelerator in an m92p tiny thinkstation (after I've given it a Xeon E3-1265L V2 and 32gb of DDR3 RAM ofc, annnd once I connect the M.2(E) to the mini-PCIe socket and get past the BIOS whitelisting :facepalm: might take me a year to get around to, but heck when I do, I'll be the queen of 2015!). The only other things I want to serve are a 3-user Nextcloud instance, and a very basic home-assistant instance.

  • AGiXT

    AGiXT is a dynamic AI Agent Automation Platform that seamlessly orchestrates instruction management and complex task execution across diverse AI providers. Combining adaptive memory, smart features, and a versatile plugin system, AGiXT delivers efficient and comprehensive AI solutions.

  • Even if my (for example, privateGPT) LLM is glacially slow I'd still love to be able to say "Mr Holmes, have Mrs Doubtfire verb the data object in order to verb a product for me, please." (eg: analyse the wikipedia article on the peace of westfalia in order to ELI5 a short summary of it). Hopefully she'd crunch away at the data, and at my convenience, I could have her brief me on her conclusions. I'm sure folks here would do something more clever using AGiXT, or having the old girl prepare lesson-plans for Mycroft to deliver (I just think that sort of thing is world-changing-bonkers for anyone wanting to learn anything, perhaps for kids one day), but I'd have to work up to that.

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

    WorkOS logo
  • Nextcloud

    ☁️ Nextcloud server, a safe home for all your data

  • My plan is to make a home-server by putting a Google Coral M.2 Accelerator in an m92p tiny thinkstation (after I've given it a Xeon E3-1265L V2 and 32gb of DDR3 RAM ofc, annnd once I connect the M.2(E) to the mini-PCIe socket and get past the BIOS whitelisting :facepalm: might take me a year to get around to, but heck when I do, I'll be the queen of 2015!). The only other things I want to serve are a 3-user Nextcloud instance, and a very basic home-assistant instance.

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts