Open-source projects categorized as Lexer | Edit details

Top 23 Lexer Open-Source Projects

  • GitHub repo participle

    A parser library for Go

    Project mention: GO App has very bad performance vs. NodeJS version. Seeking answers. | reddit.com/r/golang | 2021-05-16

    Maybe instead of regex you could look into other options.

  • GitHub repo flex

    The Fast Lexical Analyzer - scanner generator for lexing in C and C++ (by westes)

    Project mention: A work in progress C compiler from scratch | news.ycombinator.com | 2021-08-03

    I wrote a C compiler using flex [1] and bison [2]. The glue between them is a bit hacky.

    At some point ANTLR [3] looked promising, but these days I'd probably write a lexer and recursive descent parser by hand, then generate LLVM IR.

    [1] https://github.com/westes/flex

  • Scout APM

    Scout APM: A developer's best friend. Try free for 14-days. Scout APM uses tracing logic that ties bottlenecks to source code so you know the exact line of code causing performance issues and can get back to building a great product faster.

  • GitHub repo Chevrotain

    Parser Building Toolkit for JavaScript

    Project mention: Why are you building a programming language? | reddit.com/r/ProgrammingLanguages | 2021-09-05

    I don't think I'll have time to make one any time soon, unfortunately. My original plan was to write a compiler in TypeScript using Chevrotain, and see if it's possible to compile down to TypeScript's AST and feed that into its own compiler programmatically. Basically piggybacking on Microsoft's hard work (work smart, not hard). I don't know if it's possible, but it's what I'd try first.

  • GitHub repo logos

    Create ridiculously fast Lexers (by maciejhirsz)

    Project mention: [X-Post] Skiff, a gradually typed functional language written in Rust | reddit.com/r/rust | 2021-09-06

    Hi all, Skiff is a functional language that I've been working on over the past few months that's implemented (almost) entirely from scratch in Rust (so I thought you all might be interested). The one major dependency is the excellent lexing library Logos.

  • GitHub repo libfsm

    DFA regular expression library & friends

    Project mention: DFA regular expression library and friends | news.ycombinator.com | 2021-07-01
  • GitHub repo verible

    Verible is a suite of SystemVerilog developer tools, including a parser, style-linter, and formatter.

    Project mention: Tools like Scitools Understand but support Verilog | reddit.com/r/FPGA | 2021-09-19

    https://github.com/chipsalliance/verible (may not do actual syntax checking)

  • GitHub repo php-parser

    :herb: NodeJS PHP Parser - extract AST or tokens (PHP5 and PHP7) (by glayzzle)

  • Nanos

    Run Linux Software Faster and Safer than Linux with Unikernels.

  • GitHub repo JFlex

    The fast scanner generator for Java™ with full Unicode support

  • GitHub repo ExprTK

    C++ Mathematical Expression Parsing And Evaluation Library

    Project mention: library for solving equations from string? | reddit.com/r/cpp | 2021-10-08
  • GitHub repo Diagon

    Interactive ASCII art diagram generators. :star2:

    Project mention: Vim for presentation? | reddit.com/r/vim | 2021-10-07

    Doing presentations directly in vim is kinda insane, but if you are insane, I have the tool for you here.

  • GitHub repo graphql-go-tools

    Tools to write high performance GraphQL applications using Go/Golang.

    Project mention: Tech writer looking for an OSS project to join for Hacktoberfest | reddit.com/r/DevelopingAPIs | 2021-10-04
  • GitHub repo CSLY

    a C# embeddable lexer and parser generator (.Net core)

    Project mention: June 2021 monthly "What are you working on?" thread | reddit.com/r/ProgrammingLanguages | 2021-06-02

    I am still working on CSLY, a parser generator for C#.

  • GitHub repo lazy-json

    🐼 Framework agnostic package to load heavy JSON into Laravel lazy collections.

    Project mention: Lazy JSON | reddit.com/r/PHP | 2021-05-21

    Lazy JSON is a framework agnostic package that loads heavy JSON in Laravel lazy collections to limit memory usage and allow a fluent syntax to work with the JSON data.

  • GitHub repo pl0c

    Self-hosting PL/0 to C compiler to teach basic compiler construction from a practical, hands-on perspective.

    Project mention: Let's write a compiler, part 3: A parser | news.ycombinator.com | 2021-08-16

    I don’t think there will be an AST. https://briancallahan.net/blog/20210814.html: “We will be writing a single-pass compiler for a simple language and immediately output our final output code as soon as our compiler has enough information to do so”

    That “as soon as” implies code will be generated before the entire program has been parsed.

    Also, for me single-pass implies “no AST”, as you would need at least one pass to construct one, and iterating over an AST counts as another pass for me.

  • GitHub repo charly-vm

    Fibers, Closures, C-Module System | NaN-boxing, bytecode-VM written in C++

    Project mention: The Charly Programming Language | news.ycombinator.com | 2021-10-06
  • GitHub repo ravi-compiler

    Parser and compiler for Ravi and Lua

    Project mention: A new compiler for Lua and Ravi that can also generate code AOT | reddit.com/r/Compilers | 2021-07-09

    The compiler is work in progress but can handle a large subset of Ravi code at the moment. Full details are available at the Github project.

  • GitHub repo miss_hit

    MATLAB Independent, Small & Safe, High Integrity Tools

    Project mention: Static analyzers for GNU Octave | reddit.com/r/learnprogramming | 2021-03-11

    While I have no experience with the specific ecosystems you're interested in, the term you're looking for is a "linter." I found one for GNU Octave/MATLAB.

  • GitHub repo lazy-json-pages

    🐼 Framework agnostic package using asynchronous HTTP requests and PHP generators to load paginated items of JSON APIs into Laravel lazy collections.

    Project mention: Lazy JSON Pages | reddit.com/r/laravel | 2021-08-08

    Lazy JSON Pages is a framework agnostic package that uses asynchronous HTTP requests and generators to load paginated items of JSON APIs into Laravel lazy collections.

  • GitHub repo lexer

    A lexical analyzer based on DFA that made by JS and supports multi-language extension

    Project mention: GitHub : A lexical analyzer based on DFA that is built using JS and supports multi-language extensions | reddit.com/r/javascript | 2021-10-15
  • GitHub repo scintillua

    Scintillua adds dynamic Lua LPeg lexers to Scintilla.

    Project mention: Libraries for text data parsing? | reddit.com/r/lua | 2021-10-12

    Well, Here's an example of a lpeg lua lexer

  • GitHub repo KAI

    KAI is a distributed computing model written in modern C++ and is cross-plaftorm. Using custom language translators and an executor, KAI provides full reflection, persistence and cross-process communications without having to modify existing source code. KAI Comes with an automated, generational tricolor garbage collector, and Console- and Window-based interfaces.

    Project mention: Will C++ ever be as safe as Rust? Does The Standard even care? | reddit.com/r/programming | 2021-10-11

    All I'll add is that clever code is not necessarily good code. For instance, I wrote this and I doubt anyone in my team, or many people in general, could maintain or majorly refactor it.

  • GitHub repo wheel-lang

    A small toy imperative language (with some OOP features) for demonstrating and practicing language design/implementation.

    Project mention: Practical introduction to algebraic datatypes (ADTs) in TypeScript | reddit.com/r/typescript | 2021-08-10

    Yeah, when building my language, manually constructing the sum types was definitely annoyingly verbose. I found that you can mostly get a good experience for pattern matching with switch statements, but it definitely could be better.

  • GitHub repo miso-from-html

    Lex, parse and pretty print HTML as Miso View syntax

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2021-10-15.


What are some of the best open-source Lexer projects? This list will help you:

Project Stars
1 participle 2,246
2 flex 2,235
3 Chevrotain 1,743
4 logos 1,214
5 libfsm 673
6 verible 530
7 php-parser 420
8 JFlex 413
9 ExprTK 340
10 Diagon 317
11 graphql-go-tools 220
12 CSLY 154
13 lazy-json 141
14 pl0c 75
15 charly-vm 72
16 ravi-compiler 42
17 miss_hit 42
18 lazy-json-pages 39
19 lexer 27
20 scintillua 20
21 KAI 12
22 wheel-lang 10
23 miso-from-html 7
Find remote jobs at our new job board 99remotejobs.com. There are 34 new remote jobs listed recently.
Are you hiring? Post a new remote job listing for free.
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives