Parsing EDIFACT in Python with syntax trees

Edison An EDIFACT parser with a real syntax tree. Edison is a Python library for parsing EDIFACT messages. Instead of treating a message as a flat list of segments, it models messages as a syntax tree: segments can define groups of child segments, recursively. You define the syntax tree once, then parse, validate, or pretty-print messages against that structure. It works by converting the syntax tree to a state machine of possible transitions between segments. Highlights Define EDIFACT syntax trees directly in Python. Segments can define groups of child segments, which can be conditional, mandatory, or repeatable. Attach parsing callbacks to the segments you care about. Select syntax tree by message version. Handle escaped EDIFACT separators and EBCDIC control characters. Properly tokenize segments with specified lengths like BLB Export transition graphs with pydot. Installation Edison requires Python 3.12 or newer. pip install edison-parser Quick start Subclass BaseParser, define one or more schemas, and attach callback method names to the schema nodes you want to process. from edison import BaseParser class PIZZAQParser(BaseParser): schemas = { (0, 0): [ ["UNH"], ["CVR"], ["REF"], # one type of REF segment [ "PIZ", [ ["TYP", [], "parse_typ"], ["REF", [], "parse_ref"], # another type of REF segment ], ], ["UNT"], ], } def __init__(self): self.summary = {} self.pizza = None def parse_typ(self, segment: str): # Executed when the parser sees a "TYP" segment. The segment is passed as a string. self.pizza = segment[4:] def parse_ref(self, segment: str): # Executed when the parser sees a "REF" segment. The segment is passed as a string. # Only executed on the "second" type of "REF" segment! refs = segment[4:].split("*") self.summary[self.pizza] = len(refs) edi = """ UNH+1+PIZZAQ:1:1:1A+00012480825035' CVR+USD' REF+LUIS' PIZ+1' TYP+MARGARITA' REF+1*2' PIZ+2' TYP+PESTO' REF+3' UNT+19+1' """ # Create the parser and parse this EDIFACT message. # The `summary` attribute is populated by the callbacks. parser = PIZZAQParser() parser.parse(edi) parser.summary # {"MARGARITA": 2, "PESTO": 1} # Validate and pretty-print the message. PIZZAQParser.validate(edi) # True/False PIZZAQParser.show(edi) # to print Here is the output of PIZZAQParser.show(edi). Tree-based structure. Schema format A schema is a dictionary keyed by the minimum supported version: schemas = { (0, 0): [ ["UNH"], ["BGM"], ["UNT"], ], } Each segment node accepts these positional fields: ["TAG", children, action, repetition, flag] Main API Advanced usage Define multiple syntax trees when the same message evolves across versions. The parser selects the highest schema version less than or equal to the version argument, while parsing callbacks can stay the same. from edison import BaseParser class RouteParser(BaseParser): schemas = { (0, 0): [ ["UNH"], ["ORG", [], "parse_origin"], ["DST", [], "parse_destination"], ["UNT"], ], (2, 0): [ ["UNH"], [ "LEG", [ ["ORG", [], "parse_origin"], ["DST", [], "parse_destination"], ], ], ["UNT"], ], } def __init__(self): self.route = {} def parse_origin(self, segment: str): self.route["origin"] = segment[4:] def parse_destination(self, segment: str): self.route["destination"] = segment[4:] legacy = RouteParser() legacy.parse("UNH'ORG+PAR'DST+NCE'UNT'", version=(1, 0)) modern = RouteParser() modern.parse("UNH'LEG+1'ORG+PAR'DST+NCE'UNT'", version=(2, 0)) assert legacy.route == modern.route == { "origin": "PAR", "destination": "NCE", } assert RouteParser.validate("UNH'ORG+PAR'DST+NCE'UNT'", version=(1, 0)) assert RouteParser.validate("UNH'LEG+1'ORG+PAR'DST+NCE'UNT'", version=(2, 0)) When version is omitted, incomplete, or newer than all known schemas, Edison uses the latest schema. If a requested version is older than the first schema, find_transitions() raises ValueError. repetition controls whether the transition graph can come back to the same node: 1 means the node can be used once in a given sequence. Any value greater than 1 means the node is repeatable. The transition graph only distinguishes 1 from "repeatable"; it does not count exact occurrence limits such as 2 versus 99999. Mandatory nodes constrain valid paths through the tree. Once a mandatory node is next in a sequence, the parser cannot skip over it to reach a later sibling, a parent, or a child that would bypass it. A mandatory child does not make its parent mandatory by itself: if the whole group is required, mark the group node as "M" too. from edison import BaseParser class OrderParser(BaseParser): schemas = { (0, 0): [ ["UNH"], [ "ITM", [ ["DSC"], ["QTY", [], "parse_quantity", 1, "M"], ["PRI"], ], None, 99999, "M", ], ["UNT"], ], } def __init__(self): self.quantities = [] def parse_quantity(self, segment: str): self.quantities.append(segment[4:]) assert OrderParser.validate("UNH'ITM'DSC'QTY+2'PRI'ITM'QTY+1'UNT'") assert not OrderParser.validate("UNH'UNT'") # ITM is mandatory. assert not OrderParser.validate("UNH'ITM'PRI'UNT'") # QTY is mandatory. assert not OrderParser.validate("UNH'ITM'QTY+2'QTY+3'UNT'") # QTY repeats once. Use header and footer when several schemas share envelope segments. They are automatically prepended and appended to every schema at class compilation time. from edison import BaseParser class EnvelopeParser(BaseParser): header = [["UNH", [], "on_segment"]] footer = [["UNT", [], "on_segment"]] schemas = {(0, 0): [["BGM", [], "on_segment"]]} def __init__(self): self.segments = [] def set_up(self): self.segments.clear() def on_segment(self, segment: str): self.segments.append(segment) def tear_down(self): self.count = len(self.segments) set_up() runs before parsing starts, and tear_down() runs after parsing finishes. Internally, Edison builds a directed graph of transitions between segments, when the parser class is created. Each node is a specific segment, and each edge is a valid transition to the next segment. With this, parsing is just a walk through the graph, and validation is checking if the path exists. You can visualize the transition graph with build_dot_object(), which returns a pydot.Dot object. pydot is installed with Edison. To write PNG files, make sure Graphviz is available on your system. from edison import BaseParser class Parser(BaseParser): schemas = { (0, 0): [ ["UNH"], [ "BGM", [ ["DTM"], ], ], ["UNT"], ], } dot = Parser.build_dot_object() dot.write_png("transitions.png") Here is the resulting graph for the above schema. You can pass the same version selector used by parsing, and hide upward links for a simpler tree-like graph: dot = Parser.build_dot_object(version=(13, 2), include_up_links=False) dot.write_png("transitions-v13.png") Because build_dot_object() returns the underlying pydot.Dot object, you can customize it before writing. Edison understands escaped EDIFACT separators and can normalize EBCDIC separator characters often found in binary payloads. from edison import normalize_ebcdic_separators, escape, unescape assert escape("REC+") == r"REC\+" assert unescape(r"REC\+") == "REC+" assert normalize_ebcdic_separators("UNB\x1dSENDER\x1fREC") == "UNB+SENDER:REC" The segment tokenizer also handles escaped quotes, so content such as IFT+1+AIRLINE\'SERVER' stays in a single EDIFACT segment. For the common case of reading EDIFACT envelope boundaries, use extract_envelope_headers() to lazily return the first UNB/UNH segments or extract_envelope_footers() to lazily return the last UNT/UNZ segments: from edison import extract_envelope_footers, extract_envelope_headers edi = "UNB+SENDER+RECEIVER'UNH+1+PIZZAQ:1:1:1A'BGM+1'UNT+3+1'UNZ+1+1'" headers = extract_envelope_headers(edi) footers = extract_envelope_footers(edi) assert headers.unb == "UNB+SENDER+RECEIVER" assert headers.unh == "UNH+1+PIZZAQ:1:1:1A" assert footers.unt == "UNT+3+1" assert footers.unz == "UNZ+1+1" Development This repository uses uv, tox, pytest, ruff, black, ty, and pre-commit hooks. uv tool install tox --with tox-uv tox -e py Build distributions with: uv build --sdist --wheel -o dist/ Run the full pre-commit suite with: uv tool install prek==0.4.10 prek run --all-files CI runs tests through tox on CPython 3.12 and 3.14 across Linux, Windows, and macOS, then builds source and wheel distributions.