Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Yes—ANTLR 4 can generate a Python 2 lexer and parser when you select its Python 2 target explicitly. Generate the files with antlr4 -Dlanguage=Python2 Expr.g4, install the antlr4-python2-runtime package, then pass input through a lexer, token stream, and parser. Python 2 is a legacy target: ANTLR’s repository notes that support is dropped as of version 4.14, so use a compatible legacy tool/runtime pair and plan Python 3 migration for new work.

What you need to know before starting

ANTLR (ANother Tool for Language Recognition) is a parser generator. You describe a language in a grammar file; the ANTLR tool generates lexer and parser code for a selected target language. The generated recognizers use that target’s runtime library.

  • For Python 2, select the target with -Dlanguage=Python2. The Python 2 and Python 3 targets are distinct because the languages are not fully compatible.
  • Install the Python 2 runtime with pip install antlr4-python2-runtime.
  • Keep the ANTLR tool and runtime on compatible versions. The project notes that Python 2 support is dropped as of version 4.14; it does not establish one universally compatible legacy version pair here.

The ANTLR download page lists version 4.13.2, released August 3, 2024. That fact does not make it a blanket recommendation for Python 2: verify compatibility between the exact tool and runtime versions you choose before relying on the setup.

Build a small arithmetic parser

This example recognizes integers, parentheses, and the four basic arithmetic operators. It parses an expression and prints its parse tree; it does not calculate the expression’s numeric result.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

1. Write the grammar

Save the following as Expr.g4:

grammar Expr;

prog : expr EOF ;
expr : expr ('*'|'/') expr
     | expr ('+'|'-') expr
     | INT
     | '(' expr ')'
     ;
INT  : [0-9]+ ;
WS   : [ trn]+ -> skip ;

prog is the start rule used by the driver. It requires an expression followed by EOF, so the parser consumes the complete input. The INT lexer rule matches one or more digits, and WS skips spaces, tabs, and line breaks.

The order of the recursive alternatives matters: multiplication and division appear before addition and subtraction, giving them higher precedence in this grammar. Parentheses provide explicit grouping.

2. Generate the Python 2 recognizer

Run the ANTLR tool with the Python 2 target:

antlr4 -Dlanguage=Python2 Expr.g4

This assumes the antlr4 command is already available in your environment. The generated files include ExprLexer.py, ExprParser.py, and ExprListener.py. If you also want a visitor, request one during generation:

antlr4 -Dlanguage=Python2 -visitor Expr.g4

That command generates the visitor class as well. Generated classes provide the recognizer and tree-traversal extension points; they do not supply application-specific behavior such as evaluating an expression.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

3. Install the Python 2 runtime

Install the runtime in the Python 2 environment that will run the generated code:

pip install antlr4-python2-runtime

ANTLR’s download page says its runtimes are provided as source code, so no additional installation is required beyond obtaining the runtime for the target. For a Python 2 project, the package command above is the relevant setup step. Because Python 2 support is legacy, check that the runtime version you install is compatible with the tool version used to generate the files.

4. Create the driver

Save this as Driver.py in the same directory as the generated files:

import sys
from antlr4 import FileStream, CommonTokenStream
from ExprLexer import ExprLexer
from ExprParser import ExprParser

def main(argv):
    input_stream = FileStream(argv[1])
    lexer = ExprLexer(input_stream)
    stream = CommonTokenStream(lexer)
    parser = ExprParser(stream)
    tree = parser.prog()
    print(tree.toStringTree(recog=parser))

if __name__ == '__main__':
    main(sys.argv)

The program reads the path in argv[1], creates a character input stream, tokenizes it, and invokes the grammar’s prog start rule. The result is a parse tree, which the final line renders as text.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

5. Parse an input file

Put this expression in input.txt:

10+20*30

Run the driver with Python 2:

python Driver.py input.txt

The tree is equivalent to (prog (expr 10 + (expr 20 * 30)) <EOF>). The multiplication expression is nested beneath the addition expression, showing that the grammar gives multiplication higher precedence.

Use a listener or a visitor for application behavior

Both approaches traverse the parse tree produced by the same lexer and parser pipeline. Choose based on how you want application code to interact with that tree.

Approach How traversal works Best fit State to manage
Listener ANTLR’s tree walker calls rule-entry and rule-exit methods as it traverses the tree. Side effects, reporting, or reacting to recognized constructs. Store any information needed across callbacks in the listener.
Visitor Your code explicitly visits children and can return a value from each visit. Expression evaluation or other work that naturally returns results. Traversal and result handling are explicit in your visitor methods.

Listener example

For a grammar with rules named key and value, a listener can respond when a key rule finishes:

class KeyPrinter(MyGrammarListener):
    def exitKey(self, ctx):
        print("Oh, a key!")

After calling the grammar’s start rule to get tree, walk it with the listener:

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
printer = KeyPrinter()
walker = ParseTreeWalker()
walker.walk(printer, tree)

Import the generated listener and ParseTreeWalker in the surrounding program. The example prints when the walker exits each key rule; it does not return a value for that node.

Visitor alternative

Use a generated visitor when each node’s result should be computed and passed back to its caller. For arithmetic evaluation, for example, visitor methods can return numeric results from integer nodes and combine child results at operator nodes. Unlike a listener’s walker-driven callbacks, the visitor implementation chooses when to visit each child and what to return. The grammar still only describes syntax; you must implement those semantics yourself.

Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Support on Ko-Fi

Keep the Python 2 setup maintainable

  • Use a Python 2 environment isolated from current Python 3 projects, and record the exact ANTLR tool and runtime versions that work together.
  • Do not assume the newest ANTLR release still supports the Python 2 target. The project states that Python 2 support is dropped as of version 4.14.
  • For new work, prefer Python 3. For an existing Python 2 application, treat this integration as a compatibility-maintenance task and plan migration.

For broader grammar design and language implementation exercises, the ANTLR project points readers to The Definitive ANTLR 4 Reference.

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.