Ask HN: Bypassing GPT-4 8k tokens limit

• compress-gpt

  2 218 4.0 Python

  Self-extracting GPT prompts for ~70% token savings

• Text_summary

  1 3 2.6 Python

you could use ntlk to summarize the text before you send it GPT-4.

I have a script that uses NLTK to do this. It needs cleaned up but it could be a starting point.

https://github.com/gnuconcepts/Text_summary

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• llama_index

  75 30,639 10.0 Python

  LlamaIndex is a data framework for your LLM applications

  

Check out llama_index at https://github.com/jerryjliu/llama_index. What it does: it creates an index over your data using OpenAI embeddings vectors, using the OpenAI Ada model. When querying, it compiles as much context out of this index as fits into GPT, based on similarity to your prompt. Be cautious however: when I experimented with this, GPT-4 support with it‘s larger context size was not there yet. I have landed https://github.com/hwchase17/langchain/pull/1778, but I never wound up submitting another, yet similar patch (to llama_index? Don‘t remember). Make sure that the GPT-4 context is really fully used, and not some smaller size is assumed. Also, ensure that GPT-4 is used as the LLM in the first place: the defaults used to be the older models.

• langchain

  152 56,526 10.0 Python

  Discontinued ⚡ Building applications with LLMs through composability ⚡ [Moved to: https://github.com/langchain-ai/langchain] (by hwchase17)

  

Check out llama_index at https://github.com/jerryjliu/llama_index. What it does: it creates an index over your data using OpenAI embeddings vectors, using the OpenAI Ada model. When querying, it compiles as much context out of this index as fits into GPT, based on similarity to your prompt. Be cautious however: when I experimented with this, GPT-4 support with it‘s larger context size was not there yet. I have landed https://github.com/hwchase17/langchain/pull/1778, but I never wound up submitting another, yet similar patch (to llama_index? Don‘t remember). Make sure that the GPT-4 context is really fully used, and not some smaller size is assumed. Also, ensure that GPT-4 is used as the LLM in the first place: the defaults used to be the older models.

• flash-attention

  26 10,642 9.4 Python

  Fast and memory-efficient exact attention

  

Longer sequence length in transformers is an active area of research (see e.g the great work from the Flash-attention team - https://github.com/HazyResearch/flash-attention), and I'm sure will improve things dramatically very soon.

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

Suggest a related project