Omni-tokenizer Alternatives

Similar projects and alternatives to omni-tokenizer based on common topics and language

• modelfusion-ollama-nextjs-starter

  2 61 7.9 TypeScript omni-tokenizer VS modelfusion-ollama-nextjs-starter

  Starter examples for using Next.js and the Vercel AI SDK with Ollama and ModelFusion.

• modelfusion-llamacpp-nextjs-starter

  2 23 7.1 TypeScript omni-tokenizer VS modelfusion-llamacpp-nextjs-starter

  Starter examples for using Next.js and the Vercel AI SDK with Llama.cpp and ModelFusion.

• SurveyJS

  surveyjs.io featured

  Open-Source JSON Form Builder to Create Dynamic Forms Right in Your App. With SurveyJS form UI libraries, you can build and style forms in a fully-integrated drag & drop form builder, render them in your JS app, and store form submission data in any backend, inc. PHP, ASP.NET Core, and Node.js.

  

• Chevrotain

  3 2,403 6.7 TypeScript omni-tokenizer VS Chevrotain

  Parser Building Toolkit for JavaScript

  

• gpt-tokenizer

  1 383 3.3 TypeScript omni-tokenizer VS gpt-tokenizer

  JavaScript BPE Tokenizer Encoder Decoder for OpenAI's GPT-2 / GPT-3 / GPT-4. Port of OpenAI's tiktoken with additional features.

• vscode-blockman

  8 341 3.9 TypeScript omni-tokenizer VS vscode-blockman

  VSCode extension to highlight nested code blocks

• gpt4-tokenizer-visualizer

  3 20 4.2 TypeScript omni-tokenizer VS gpt4-tokenizer-visualizer

  GPT4 Tokenizer Visualizer

• leac

  1 5 3.6 TypeScript omni-tokenizer VS leac

  Lexer / tokenizer

  

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.