Our great sponsors
-
-
gpt_index
Discontinued LlamaIndex (GPT Index) is a project that provides a central interface to connect your LLM's with external data. [Moved to: https://github.com/jerryjliu/llama_index]
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
Today, if someone wants to know what the HN community knows/thinks about a topic, they can either:
A) Search past HN comments on hn.algolia.com, or
B) Post a new 'Ask HN'.
LLMs could provide a new way to find answers within a corpus. These have been described elsewhere, e.g.
- https://github.com/openai/openai-cookbook/blob/main/examples/Question_answering_using_embeddings.ipynb
- https://news.ycombinator.com/item?id=34477543
I keep expecting someone (maybe minimaxir or simonw?) to post a 'Show HN: Get your question answered by the collective wisdom of HN', but I no one has so far (unless I missed the submission?).
Is someone already working on this?
The things I've seen all use hosted language models. For example https://github.com/jerryjliu/gpt_index depends on LangChain, which wraps APIs from hosted LLMs: https://langchain.readthedocs.io/en/latest/reference/modules...
AFAIK there's no GPT-3-like LLM that's easy to run at home, because the number of parameters is so so large. Your gaming PC's GPU won't have enough RAM to hold the model.
Apparently there are two ways to access it on GCP:
https://github.com/ashish01/hn-data-dumps
HN has an API[0], with a bit of effort you can make one yourself.
[0]https://github.com/HackerNews/API