Top 23 Lexer Open-Source Projects

• flex

  9 3,436 8.4 C

  The Fast Lexical Analyzer - scanner generator for lexing in C and C++ (by westes)

Project mention: How to provide input file for Flex++? | /r/Compilers | 2023-09-05

I am using Flex++, which is Flex for C++ and I am having trouble setting the input file. Flex++ uses the FlexLexer class provided in FlexLexer.h to create the lexer object(https://github.com/westes/flex/blob/master/src/FlexLexer.h). In my main function I have

• participle

  8 3,312 6.8 Go

  A parser library for Go

Project mention: Parser generator | /r/golang | 2023-09-14

I know of tools like Kaitai that do essentially what I want, just curious about my options. I guess something like https://github.com/alecthomas/participle could work to parse the format, then I'd have to generate code based on the parsed data or something

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• logos

  15 2,627 8.3 Rust

  Create ridiculously fast Lexers (by maciejhirsz)

Project mention: Beating the fastest lexer generator in Rust | /r/rust | 2023-07-11

This is mighty impressive! I've been trying to get some motivation for the mythical rewrite of the proc macro in Logos, and this might just do it for me :D. I'll have a proper look later today and see if any of your findings have something that can be generalized. Also really surprised to see aarch64 doing better than x86_64 since the latter is what I've been optimizing for!

• Chevrotain

  3 2,399 6.7 TypeScript

  Parser Building Toolkit for JavaScript

  

Project mention: Ohm: A library and language for building parsers, interpreters, compilers, etc. | news.ycombinator.com | 2023-10-31

How does this compare with Chevrotain[1]?

More specifically, can I build lexers with Ohm? Can it generate a syntax diagram from a grammar?

[1]: https://github.com/chevrotain/chevrotain

• csstree

  1 1,836 7.1 JavaScript

  A tool set for CSS including fast detailed parser, walker, generator and lexer based on W3C specs and browser implementations

  

Project mention: Building a lightweight CSS formatter | dev.to | 2023-06-12

After thinking about the problem for a while I realised that I could enlist the help of CSSTree to do some of the work. The CSS Analyzer is based on CSSTree's AST, so I know how the thing works and the dependency is already on the page, so no need to download more dependencies. Prettier + Postcss cost almost 340kB to download, which isn't huge, but it would be nice if we could reduce that amount.

• Diagon

  7 1,431 5.1 C++

  Interactive ASCII art diagram generators. :star2:

Project mention: Architecture diagrams enable better conversations | news.ycombinator.com | 2023-08-22

A few more ASCII-based tool that you could add into your workflow are https://arthursonzogni.com/Diagon/#Sequence which can be used to generate the ASCII that you input into something like Typogram (https://google.github.io/typograms/).

For example, input:

  Renderer -> Browser: BeginNavigation()

• verible

  6 1,197 9.3 C++

  Verible is a suite of SystemVerilog developer tools, including a parser, style-linter, formatter and language server

  

Project mention: How to instance module with auto-completion for verilog in neovim? | /r/neovim | 2023-08-25

I want to write Verilog/SystemVerilog with neovim(I use Lazyvim,nvim-lspconfig,mason.nvim, mason-lspconfig.nvim and nvim-cmp) . Now I use Verible to format and lint. But it seems that it cannot complete the signals when I want to instance a module and type a "." . So is there a better way to interconnect modules?

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

• libfsm

  2 912 8.4 C

  DFA regular expression library & friends

• moo

  1 807 2.4 JavaScript

  Optimised tokenizer/lexer generator! 🐄 Uses /y for performance. Moo. (by no-context)

  

• graphql-go-tools

  27 638 9.6 Go

  GraphQL Router / API Gateway framework written in Golang, focussing on correctness, extensibility, and high-performance. Supports Federation v1 & v2, Subscriptions & more.

  

Project mention: Scaling GraphQL Subscriptions in Go with Epoll and Event Driven Architecture | dev.to | 2024-03-04

If you're interested in the full implementation of the resolver, you can find it on GitHub.

• ExprTK

  10 577 0.0 C++

  C++ Mathematical Expression Parsing And Evaluation Library https://www.partow.net/programming/exprtk/index.html

• JFlex

  1 575 4.3 Java

  The fast scanner generator for Java™ with full Unicode support

  

• edge

  2 557 5.9 TypeScript

  Node.js template with a breath of fresh air (by edge-js)

  

• php-parser

  0 515 3.3 JavaScript

  :herb: NodeJS PHP Parser - extract AST or tokens (by glayzzle)

  

• RE-flex

  3 484 7.4 C++

  A high-performance C++ regex library and lexical analyzer generator with Unicode support. Extends Flex++ with Unicode support, indent/dedent anchors, lazy quantifiers, functions for lex and syntax error reporting and more. Seamlessly integrates with Bison and other parsers. (by Genivia)

  

Project mention: RE/flex 3.3.4 released - a scanner generator for C++ | /r/Compilers | 2023-05-31

• grmtools

  2 472 7.7 Rust

  Rust grammar tool libraries and binaries

  

• Melang

  8 413 7.7 C

  A script language of time-sharing scheduling coroutine in single thread

Project mention: A Time-Sharing Scheduling Coroutine Script Language | /r/programming | 2023-12-05

• CSLY

  2 336 9.2 C#

  a C# embeddable lexer and parser generator (.Net core)

• lexer

  11 323 2.3 JavaScript

  A lexical analyzer based on DFA that is built using JS and supports multi-language extensions / 一个基于DFA的支持多语言扩展的JS版开源词法分析器

• lazy-json

  2 231 7.2 PHP

  🐼 Framework-agnostic package to load JSON of any dimension and from any source into Laravel lazy collections recursively.

Project mention: 🐼 Lazy JSON: load JSON of any size into Lazy Collections | /r/PHP | 2023-11-30

Hello everybody, happy to share that Lazy JSON has been released.

• parser

  1 212 7.9 C#

  A lexer and parser for GraphQL in .NET (by graphql-dotnet)

  

• phplrt

  1 206 8.8 PHP

  PHP Language Recognition Tool

  

• matcheroni

  4 192 9.5 C++

  A minimalist single-header library for building pattern-matchers, lexers, and parsers.

Project mention: Matcheroni, a tiny C++20 header library for building lexers/parsers | /r/regex | 2023-07-06

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

Lexer related posts

• Scaling GraphQL Subscriptions in Go with Epoll and Event Driven Architecture

  2 projects | dev.to | 4 Mar 2024

• 🐼 Lazy JSON: load JSON of any size into Lazy Collections

  1 project | /r/PHP | 30 Nov 2023

• GraphQL Router / API Gateway Framework Written in Golang

  1 project | news.ycombinator.com | 15 Nov 2023

• Cosmo Router: High Performance Federation v1 & v2 Router / Gateway

  2 projects | dev.to | 14 Sep 2023

• Parser generator

  1 project | /r/golang | 14 Sep 2023

• WunderGraph Cosmo: a open source alternative to Apollo Federation, GraphOS, Studio, etc...

  3 projects | dev.to | 13 Sep 2023

• Introducing Open Federation - a MIT-Licensed specification to build federated GraphQL APIs

  3 projects | dev.to | 12 Sep 2023
• A note from our sponsor - InfluxDB
  www.influxdata.com | 5 May 2024

  Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality. Learn more →

Index

Sponsored

SaaSHub - Software Alternatives and Reviews

SaaSHub helps you find the best software and product alternatives

www.saashub.com