Changelog

All notable changes to this project will be documented in this file.

0.1.0a11 - 2026-07-23

Added

• Configurable Custom URI Schemes (extra_authority_schemes, extra_opaque_schemes):

  • Implemented parser options extra_authority_schemes and extra_opaque_schemes in parse_to_ast with dynamic URI regex regeneration.

• Implemented scheme name validation and reserved scheme blacklist (link, image, xref, note, tip, warning, to, cc, bcc, date, time, ssn, id, etc.) to prevent syntax collisions with core AsciiDoc macros and attributes.

• Native Python Local TCK Test Harness:

• Integrated local TCK test suite (tests/tck_harness/) into standard pytest runs via tests/test_local_tck.py.

• 3-Tiered Testing Strategy & Efficiency Guidelines:

  • Established a 3-tiered testing workflow (Tier 1: Dev Loop, Tier 2: Pre-Commit, Tier 3: Pre-Release) with quiet reporting (-q) in AGENTS.md and docs/contributing.adoc.

• Bare URL & Email Autolinks (Issue #76):

  • Full support for parsing bare URLs and email addresses (user@domain.com) into Ref nodes with role: "bare" and target="mailto:...".

  • Implemented trailing punctuation stripping (.,;:!?)>]}), angle-bracket delineation (<https://...>), backslash escaping (https://..., user@domain.com), and round-trip serialization back to raw string forms.

• Standardized Verbatim Properties on Literal Blocks:

  • Added property accessors (id, style, and literal_title) to Literal block nodes to ensure interface parity with Listing blocks.

• Section Nesting TCK Test (Draft):

  • Added a local TCK test (tests/tck_harness/tests/block/section/nesting-*) demonstrating section nesting and level transitions: child nesting (level 1 → 2 → 3) and returning to a sibling section at a lower numerical level.

Fixed

• Block Style Attribute Preservation in ASGResolver:

  • Updated ASGResolver and lark_parser to preserve block style attributes on resolved ASG block nodes for structural equivalence with reference ASG outputs.

• Recursive Formatted Paragraph Text Reconstruction in ASTSyntaxAuditor:

• Fixed ASTSyntaxAuditor.visit_paragraph to recursively traverse nested formatted inline spans (bold, italic, monospace, etc.) via child.walk(), ensuring syntax auditing properly inspects text inside formatted inline structures.

• Non-Destructive ASGResolver:

• Updated ASGResolver.resolve() to deep-copy the input AST prior to resolution, eliminating in-place AST mutation side-effects and preserving caller AST nodes.

• Graceful Permissive Include Directives (Issue #87):

• Updated Preprocessor and parse_to_ast to handle missing include files gracefully under permissive parsing (strict=False), emitting a PreprocessorWarning and returning an unresolved directive placeholder string (Unresolved directive in ...).

• Inline Macro Prefix vs. URI Terminal Priority Conflict:

• Fixed `link:, image:, icon:, xref:, and anchor: inline macros with URL targets (e.g. text) by introducing dedicated high-priority prefix terminals (LINK_PREFIX.5, etc.) that prevent URI.3` from stealing the URL before the macro rule can match.

• Bare URLs Swallowing Trailing Formatting Delimiters:

  • Fixed URI.3 regex to use a negative lookbehind so trailing formatting characters (*, _, ++++) and punctuation are not consumed as part of the URL token. This allows constructs like https://example.com` to parse correctly as bold-formatted bare links.

• Nested Link Elements in Link Labels:

  • Added unwrapping logic in `inline_link transformer to flatten nested Ref nodes that arise when a link label is itself a URL string (e.g. https://example.com`), preventing invalid nested anchor elements.

Changed

• Configurable & Streamlined URI Scheme Architecture:

• Refactored URI.3 terminal to a streamlined, safe default set (http/https, ftp/ftps, file, irc/ircs, ws/wss, git, ssh, mailto, data, tel, sms).

• Added support for passing custom authority and opaque schemes via parse_to_ast(..., extra_authority_schemes=[...], extra_opaque_schemes=[...]).

• Implemented strict scheme validation (validate_custom_scheme) enforcing length bounds (2–10 chars), character rules ([a-z0-9\-]+), and a comprehensive blacklist preventing collisions with inline macros (link, image, xref), block labels (note, tip, warning), headers, and common colon-followed terms (to, cc, bcc, date, time, ssn, id).

0.1.0a10 - 2026-07-20

Added

• Bare URL Links Support:

• Added robust support for parsing, serializing, and rendering bare URL and mailto references (e.g., https://google.com or mailto:zopemaven@gmail.com) inside paragraphs and inlines.

• Handled proper Docutils rendering fallback and lossless AST round-trip serialization for bare URLs without trailing brackets [].

• Multi-Block List Continuations (+):

  • Implemented complete, native parsing and structural resolution for list continuation delimiters (+), enabling subsequent paragraphs, admonitions, and nested lists to attach seamlessly to parent list items.

• Architectural & Design Documentation:

  • Formally documented *Recursive Include Coordinate Mapping and Permissive Parsing & Error Recovery* pipelines in docs/architecture.adoc.

• AsciiDoc Language Specification & Feature Matrix:

• Expanded FEATURE_MATRIX.adoc with 10 newly tracked language features and linked them directly to Peggy prototyping grammars in the AsciiDoc Parsing Lab.

• Promoted several feature rows (such as Collapsible Blocks, STEM Blocks, Index Terms, Inline Anchors, Conditional Directives, Comment Blocks, Checklists, and List Continuations) to Fully Supported.

• API Documentation:

• Generated complete API reference stubs for all core system modules.

Changed

• Stricter Parsing & Post-AST Syntax Auditor:

  • Implemented an advanced ASTSyntaxAuditor node visitor that acts as a post-parsing syntax validation engine.

  • Transitioned from an ad-hoc hybrid preprocessor validation layer to a structured, post-AST semantic checker, guaranteeing much stricter and faster compliance checks.

• Enabled precise, coordinate-correct syntax error tracking (mapping errors directly back to the physical source file and line) across multi-level include files.

Fixed

• Strict Type Safety:

  • Resolved dynamic attribute typing error on Node objects to achieve 100% type safety compliance under Mypy’s strict configuration.

• Technical Debt Pruning:

  • Marked resolved parser issues and documentation stubs as complete in TECHNICAL-DEBT.adoc.

0.1.0a9 - 2026-07-19

Added

• Full Eclipse AsciiDoc SDR-1 Open Block Support:

• Implemented the new standard variable-length tilde-based open block delimiter form (~~~~, ~~~~~, ~~~~~~, and long ~{7,}).

• Supported recursive multi-level open block nesting by varying delimiter lengths.

• Maintained 100% backward compatibility for the legacy two-hyphen (--) open block delimiter form.

• Restructured and updated the draft TCK open block test suite (organizing into containing-paragraph and legacy-delimiter tests) in tests/tck_harness/ in alignment with MR feedback.

• Developed a comprehensive unit test suite in tests/test_inline_transformer_unit.py testing 100% of all logical bodies, branching conditions, error fallbacks, and attribute substitutions in InlineTransformer (including math stems, keyboard macros, and split target-window link parsers).

• Boosted inline_transformer.py coverage from 53% to 69%.

• High-Fidelity Cyclic Include Loop Diagnostics:

• Implemented stateful include_stack tracking of inclusion paths (including file paths, relative base directories, line numbers, and line text) during preprocessing (Issue #82).

• Developed a static cycle scanner that detects mutual inclusion loops and prints elegant compiler-grade diagnostic reports with visual caret highlights and source strings.

• Component-Aware Path Traversal Protections:

  • Implemented safe commonpath-based boundary checks under safe_mode in preprocessor.py to prevent directory traversal and secure sibling folders (Issue #83).

• Enhanced Docutils/Sphinx Rendering:

• Integrated native %hidden and options="hidden" attribute-mapping on toctree block open macros, converting them directly to Sphinx addnodes.toctree(hidden=True) to prevent redundant/duplicated visual index lists.

• Implemented missing AST node visitor methods in DocutilsRenderer, supporting Header, Author, Revision, PageBreak, AttributeEntry, Attributes, and Include to ensure smooth rendering and zero compilation crashes.

• Upgraded Page Breaks Support:

  • Upgraded Page Breaks status to Fully Supported in the Feature Matrix, natively translating PageBreak AST nodes into explicit <!-- page break --> raw HTML markers in the Docutils renderer backend.

• Comprehensive AST and ASG Nodes Unit Tests:

  • Developed a comprehensive unit test suite in tests/test_nodes_unit.py systematically instantiating, verifying, and serializing all Node subclasses, coordinate tracking, custom appends, and properties.

  • Verified verbatim-block properties (code, stripped_code, callouts) matching automated, manual, and bare HTML callout regex styles under Unix and Windows line endings.

  • Boosted nodes.py coverage from 62% to 68%, elevating overall codebase test coverage to 70%.

• 93%+ Docutils Backend Test Coverage:

• Expanded tests/test_docutils_backend.py with comprehensive integration tests targeting section layouts, inline stems/math, block stems, audio/video raw embeds, image alternate text, table cell specifier alignments, and styles.

• Boosted docutils_backend.py statement coverage to 93%.

Fixed

• Non-Destructive Lookahead-Based Description List Grammar:

  • Refactored description list markers (DLIST_MARKER_2 through DLIST_MARKER_5) in grammar.lark using regex positive lookaheads ((?=[ \t\n]|$)) to prevent greedily matching inline double-colons (::) inside standard paragraph text (Issue #72 sibling).

  • Simplified the COLON terminal to a literal ":" to allow multiple consecutive colons to cleanly parse as individual colon text nodes.

• PyPI Dependency Upgrades and Workaround Reversion:

  • Migrated to stable PyPI releases of sphinx-asciidoctrine==0.1.0a1 and asciidocstring==0.1.0a5 (which natively handles blank lines between definition list items).

• Reverted the temporary preprocessor single-line docstring workaround back to a standard multi-line description list.

• Inline Stem Parsing Priorities:

  • Added explicit .10 priority suffixes to inline_stem, inline_asciimath, and inline_latexmath rules in grammar.lark to prevent Earley parser ambiguity where preceding text caused them to be incorrectly evaluated as nested plain text sequences.

• Block Macro Attribute Parsers:

• Fixed attributes parsing for block macros (such as image::, audio::, and video::) in lark_parser.py’s block_macro by dynamically scanning children for any transformed attribute_content token, ensuring alternate text (alt) and custom properties are correctly parsed and populated on AST block nodes.

• 100% Preprocessor Test Coverage:

• Reached 100% statement and branch coverage in preprocessor.py by pruning unused/redundant legacy paths and adding extensive unit testing of defensive edge-case pathways.

• Robust Multi-Level Open Block Nesting:

  • Added full test coverage for mixed standard (tilde-based) and legacy (hyphen-based) open block structures to ensure compatibility and robustness under extreme nested conditions.

• Visual Identity & Theme Overhaul:

  • Re-themed the documentation to a premium forest-green dark color scheme and customized the RTD sidebar with multi-level sub-navigation line legibility, hover styles, and link color states.

  • Redesigned ASCII compiler pipeline flowcharts in docs/index.adoc and docs/architecture.adoc to cleanly illustrate the bidirectional AST-to-source serialization capability of the compiler.

• Standardized all documentation formatting on spec-compliant single-asterisk bold spans (*text*), resolving visual discrepancies and anomalies caused by unconstrained double-asterisks (**text**).

0.1.0a8 - 2026-07-16

Added

• Parser Error Handling (Issue #78):

  • Replaced raw Lark parsing exceptions with a structured user-facing AsciiDocSyntaxError compiler error, detailing line and column numbers alongside highlighted caret indicators.

• Programmatic TCK JSON Runner:

  • Replaced fragile log-regex parsing in tests/test_tck.py with a lightweight Node.js event-driven runner (bin/run-tck-json.mjs) that consumes the native TCK stream, boosting test speed and reliability.

• Parameterized Includes in Preprocessor (Issue #79):

  • Procedural Stream Preprocessor: Updated the Preprocessor in src/asciidoctrine/preprocessor.py to support parameterized includes (leveloffset, lines, tag, tags) using an immutable, procedural top-to-bottom stream-filtering state machine.

• C-Level Substring Short-Circuit Optimization: Implemented optimized short-circuiting (":" not in line) running in compiled C to bypass dynamic interpreter loop overhead for over 95% of standard document lines, increasing performance.

• Stateful Include Filtering:

  • leveloffset Shifting: Shifts relative (e.g. +1, -1) and absolute (e.g. 2) section heading levels inside includes.

  • lines Slicing: Slices single (1..5), multiple (1..2;4..5 or "1..2,4..5"), and open-ended (5..) line ranges.

• tag/tags Filtering: Extracts marked regions of code blocks while stripping tag boundary comment lines (e.g., // tag::name[]) from the output stream.

• Comprehensive Test Harness: Authored unit tests verifying individual and combined preprocessor attributes under a strict Red/Green TDD framework, resulting in 100% test suite and TCK compliance.

• High-Fidelity Cyclic Include Loop Diagnostics (Issue #82):

  • Implemented metadata-enriched include_stack to track inclusion paths including line numbers and line text.

• Implemented an on-loop static cycle scanner that reads on-disk files and builds an elegant compiler-grade diagnostic report listing all mutual includes in a loop simultaneously, complete with raw source strings and caret highlights.

Fixed

• Dynamic Sphinx Versioning (Issue #3.C):

• Replaced hardcoded version strings in docs/conf.py with dynamic metadata-driven package version lookup using importlib.metadata.

• Unified Python Target Versions (Issue #3.C):

• Aligned and unified Python target constraints across pyproject.toml (requires-python >=3.10), Ruff (py310), and Mypy (3.12) configurations.

• O(N) Table Cell Parsing Regex (Issue #6):

• Replaced nested table cell lookaheads in grammar.lark with a sequential, flat non-backtracking lookahead pattern to eliminate exponential backtracking risks.

• Experimental Inline Macros Parsing Bug (Issue #80):

  • EBNF Grammar Priority Adjustment: Assigned explicit priority of .10 to inline_kbd, inline_button, and inline_menu grammar rules in grammar.lark. This resolves the greedy priority conflicts in Lark’s Earley parsing engine when experimental macros are embedded inside standard prose sentences, preventing them from being flattened into generic standard text nodes.

  • TCK & Integration Tests: Added comprehensive test cases ensuring standalone and paragraph-embedded experimental macro syntax parses cleanly with 100% compliance across both local unit tests and the Technology Compatibility Kit (TCK) suite.

• Secure Directory Traversal Guard (Issue #83):

  • Replaced prefix-based boundary checking with component-aware os.path.commonpath checks under safe_mode to secure sibling directories from path traversal vulnerabilities.

• Pytest Recursion Exclusions (Issue #84):

  • Configured norecursedirs inside pyproject.toml to exclude tests/include_fixtures, preventing pytest and asciidoctest from scanning test inclusion files.

0.1.0a7 - 2026-07-13

Added

• Footnotes and Named Footnoterefs Support:

  • EBNF Grammar & AST Transformation: Added complete, prioritized EBNF grammar rules in `grammar.lark for standard auto-numbered [1], named definitions [2], and subsequent references [2]`. Implemented strict coordinate tracking and resolved bracket/comma lexer conflicts.

• Docutils/Sphinx Rendering: Developed dynamic footnote rendering inside DocutilsRenderer to collect definitions, auto-number them, render them as aligned label/body pairs, and position footnote tables elegantly at the bottom of the output document.

• Lossless AST Serialization: Added full round-trip formatting inside AsciiDocSerializerVisitor to preserve footnotes and definitions with exact character accuracy.

• TCK Conformance Tests: Authored and integrated two complete local TCK specification tests inside vendor/asciidoc-tck/ and tests/tck_harness/ for auto-numbered and named footnote macros.

• Expanded Feature Matrix Audit:

  • Cross-referenced FEATURE_MATRIX.adoc with the 1,748-line official specification outline.adoc to add 10 new feature rows tracking specification progress, Peggy prototyping, TCK status, and explicit support levels (including checklists, block/inline passthroughs, TOC macro, and autolinks).

• Developer Experience (DX) Error Handling Roadmap:

• Created a detailed architectural feature design (Issue #78) for strict diagnostics (visual carets), forgiving puppet-driven error recovery, and a formatting linter.

0.1.0a6 - 2026-07-12

Added

• Pre-Release Verification Checklist: Added a comprehensive step-by-step developer and automation release checklist inside AGENTS.md to prevent formatting, type-safety, and wheel-matching regressions in future releases.

Fixed

• Strict Type Safety compliance: Fully resolved all strict static analysis and typing errors under Mypy, specifically adding proper generic type annotations on LocationDict and initializing properties dynamically assigned on Document.

• Standardized Import Sorting and Formatting: Cleaned up internal imports and resolved layout lints to achieve 100% style compliance with Ruff formatters and linters.

0.1.0a5 - 2026-07-12

Fixed

• Earley Parser Description List Term Grouping: Corrected parsing priorities of nested lists to natively prefer grouping multiple consecutive terms under a single description list item (resolving TCK compliance failures on multi-term definitions).

• Block ASG Conformance: Implemented recursive block-level attribute cleaning inside the ASG Resolver to filter out syntactic positional/style keys (like '1', 'positional', and 'style') from resolved block-level metadata, achieving 100% compliant and clean ASG output for all blocks including Stem and Verse.

• Pyodide Sandbox Wheel Parsing: Added standard dynamic wheel version mapping inside functional tests via tomllib to prevent version mismatch crashes in local/CI sandbox executions.

0.1.0a4 - 2026-07-12

Added

• Native macOS Browser Automation Support: Established global and project-scoped developer instructions (AGENTS.md) and customized skills to natively execute high-fidelity Playwright/Selenium browser previews and layout audits on macOS.

• Refructured Sphinx Documentation Landing Page: Redesigned docs/index.adoc as a clean, polished onboarding guide, removing internal-only engineering blueprints to provide a professional, user-focused documentation site.

Fixed

• Delimiter Token-Splitting and Block Swallowing (Issue #76): Integrated strict lookahead/lookbehind boundaries on OPEN_BLOCK_DELIM to prevent Lark’s dynamic lexer from splitting thematic breaks (---) or listing delimiters (----) under Earley parser predictions, guaranteeing sections are never swallowed inside open blocks.

• High-Performance Thematic Break Matching: Optimized THEMATIC_BREAK_MARKER by removing slow, backtracking regex lookbehinds ((?<!...)) in favor of lookahead-only patterns, resulting in massive performance gains across large files.

• Unified Newline LF Normalization and Detection: Implemented robust pre-parsing LF normalization that automatically detects and stores the document’s original line ending type (CRLF vs LF) and the presence of a trailing newline.

• Trailing Newline and Round-Trip Accuracy: Updated the native serializer to automatically restore original newline styles and trailing newlines, fixing the trailing newline stripping bug and guaranteeing 100% exact, character-for-character round-trip accuracy.

• Sphinx Title Promotion and Heading Nesting: Corrected Docutils node hierarchy rendering where document titles were ignored and sections were flattened into flat root h1 headings, wrapping them under a single root section.

• Fixed Table Cell Pipe Delimiter Splitting (ARCHITECTURE.adoc): Manually escaped internal vertical pipes (\|) within table cells to prevent row-splitting and cell corruption under the lexer.

0.1.0a3 - 2026-07-11

Added

• Native AsciiDoc Serializer (Issue #74): Created AsciiDocSerializerVisitor and the public serialize_to_asciidoc() function to serialize any unresolved AST back into standard, clean, valid AsciiDoc source.

• NodeTransformer Class (Issue #73): Introduced NodeTransformer to facilitate programmatic AST/ASG modification, replacement, pruning, and expansion.

• Native Inline Link Parsing (Issue #75): Standardized native parsing of inline links/URIs as Ref nodes instead of simple text elements.

• Expanded Functional & Unit Tests: Added regression tests for NodeTransformer, serialize_to_asciidoc under unit suites and Pyodide functional test setups.

Fixed

• Consecutive Block Attributes Merging (Issue #72): Resolved a structural Lark Earley ambiguity by lowering the priority of attribute and anchor rules, allowing consecutive metadata lines (e.g. [.role]\n[source,python]) to merge correctly.

• Positional Attributes Parsing (Issue #71): Corrected parsing of mixed shorthand, named, and positional attributes without early return collisions.

0.1.0a2 - 2026-07-07

Added

• Nested & Mixed Description Lists: Complete parsing and ASG structure mapping of recursive definition/description lists.

• Advanced Table Cell Formatting: Full support for inline cell options, colspans, rowspans, text alignment patterns, and cell style operators.

• Listing Block Metadata & ASG Resolution: Direct properties mapping (id, language, style, listing_title) on Listing nodes with compliant resolved attributes propagation.

• Callout Stripping and Matching: Implemented robust matching/stripping of callout annotations (<1>) from verbatim listing blocks.

• Location Accuracy & Consolidation: Consolidated node coordinates with precise line/column inclusive tracking.

0.1.0a1 - 2026-07-07

Added

• Initial Release: Published and packaged the baseline AsciiDoc parser.

• Structured AST: Migration from dictionary-based AST to class-based node objects in nodes.py.

• Nested List Support: Improved handling of multi-level unordered and ordered lists.

• Modern Grammar: Standardized bold and italic markers and refined word parsing.

• Example Suite: Data-driven tests for .adoc examples.

• Packaging: Added pyproject.toml and professional documentation.

• Admonition Blocks: Full support for [NOTE], [TIP], [IMPORTANT], [WARNING], and [CAUTION].

• Sidebar Blocks: Implementation of **** delimited sidebar blocks with full block nesting.

• Attribute Parsing: Added generic [...] attribute list parsing for blocks.

• Source/Code Blocks: Foundation for source blocks via [source,lang] attributes on literal blocks.

• Example Blocks: Support for ==== delimited example blocks.

Fixed

• Resolved ast module naming conflict.

• Fixed list marker ambiguity in recursive grammar rules.

[1]

text

[2]

text