A modern Flex/Bison replacement that generates lexers and parsers in C, Java, and Python.

• Lexer Generator - Convert regex patterns to efficient DFA-based lexers
• Parser Generator - Generate LALR(1) parsers from BNF grammars
• Multi-language Output - Generate code in C, Java, or Python
• Flex/Bison Compatible - Uses familiar .l and .y file formats
• Web GUI - Browser-based interface via WebAssembly

Visit the web demo to try OpenLexer in your browser.

# Install Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

# Clone and build
git clone https://github.com/example/openlexer.git
cd openlexer
cargo build --release

openlexer gen-lexer --lexer rules.l --lang python --output output/

openlexer gen-parser --parser grammar.y --lang java --output output/

OpenLexer generates well-organized code following language-specific best practices:

• Lexer Only: Generates Lexer.java with public Lexer class
• Parser Only: Generates Parser.java with public Parser class (includes inline lexer)
• Both: Generate separate files and compile together:

  openlexer gen-lexer --lexer rules.l -L java -o output/
  openlexer gen-parser --parser grammar.y -L java -o output/
  javac output/Lexer.java output/Parser.java
  java -cp output Parser "3 + 4 * 2"

  Parser automatically detects and uses external Lexer.class if available.

• Lexer: Generates lexer.c
• Parser: Generates parser.c
• Use -DLEXER_NO_MAIN -DPARSER_NO_MAIN flags to disable test drivers when linking

• Lexer: Generates lexer.py (importable module)
• Parser: Generates parser.py (can import external lexer)

📖 See docs/FILE_ORGANIZATION.md for detailed file organization guide

%%
[0-9]+      { return NUMBER; }
"+"         { return PLUS; }
"-"         { return MINUS; }
"*"         { return TIMES; }
"/"         { return DIVIDE; }
"("         { return LPAREN; }
")"         { return RPAREN; }
[ \t\n]+    { /* skip whitespace */ }
%%

%token NUMBER PLUS MINUS TIMES DIVIDE LPAREN RPAREN

%%

expr:
    expr PLUS term   { $$ = $1 + $3; }
  | expr MINUS term  { $$ = $1 - $3; }
  | term             { $$ = $1; }
  ;

term:
    term TIMES factor { $$ = $1 * $3; }
  | term DIVIDE factor { $$ = $1 / $3; }
  | factor            { $$ = $1; }
  ;

factor:
    LPAREN expr RPAREN { $$ = $2; }
  | NUMBER             { $$ = $1; }
  ;

%%

# Generate Python lexer
openlexer gen-lexer --lexer calc.l --lang python --output calc_py/

# Generate Python parser
openlexer gen-parser --parser calc.y --lang python --output calc_py/

.l file (Flex format)
    |
    v
+-------------------+
| Regex Parser      |
+-------------------+
    |
    v
+-------------------+
| NFA Construction  | (Thompson's algorithm)
+-------------------+
    |
    v
+-------------------+
| DFA Construction  | (Subset construction)
+-------------------+
    |
    v
+-------------------+
| Code Generation   | --> lexer.py / lexer.c / Lexer.java
+-------------------+


.y file (Bison format)
    |
    v
+-------------------+
| Grammar Parser    |
+-------------------+
    |
    v
+-------------------+
| FIRST/FOLLOW Sets |
+-------------------+
    |
    v
+-------------------+
| LALR(1) Tables    |
+-------------------+
    |
    v
+-------------------+
| Code Generation   | --> parser.py / parser.c / Parser.java
+-------------------+

Run the desktop GUI:

cargo run --features gui --bin openlexer-gui

MIT License - see LICENSE for details.

Contributions are welcome! Please open an issue or PR.