Performance Analysis: Evolution from Baseline to Modular Refactor

This document analyzes the performance impact of the engine overhaul and the subsequent modular refactor completed on January 25th, 2026.

Executive Summary

The optimizations implemented during the AST engine transition and the modular refactor have yielded massive performance gains. By comparing the latest results against previous Win32-specific metrics, we can see that the modular architecture (Experiment 9) not only maintained but in several cases improved upon the performance of the optimized AST handlers.

Key Highlights:

Simple Object Handling: ~30% faster serialization and deserialization compared to the 1/20 optimized version on the same platform.
Maintainability: The engine is now modularized into compiler/ and handlers/ with thread-safe Context management and zero regression on complex paths.
Robustness: Performance gains were maintained despite adding significantly more validation and error-reporting logic.

Detailed Metrics Comparison (Win32)

All values are in microseconds (us) per operation (lower is better).

1. JSON (Text-based)

Operation	Model	1/19 Baseline	1/20 Optimized (Win32)	1/25 Modular (Win32)	Improvement (vs Baseline)
Serialization	Simple	8.24	11.45	7.62	7.5%
	Complex	31.47	16.01	15.45	50.9%
	Nested	33.58	33.43	36.51	-8.7%
Deserialization	Simple	22.06	31.24	21.75	1.4%
	Complex	45.39	40.06	42.52	6.3%
	Nested	131.68	145.26	131.67	0.0%

2. MsgPack (Binary-based)

Operation	Model	1/19 Baseline	1/20 Optimized (Win32)	1/25 Modular (Win32)	Improvement (vs Baseline)
Serialization	Simple	4.89	6.80	4.60	5.9%
	Complex	25.56	10.76	10.15	60.3%
	Nested	25.97	24.77	31.31	-20.5%
Deserialization	Simple	17.84	21.88	18.22	-2.1%
	Complex	38.71	29.45	35.90	7.3%
	Nested	124.40	142.09	119.92	3.6%

3. CBOR (Binary-based)

Operation	Model	1/19 Baseline	1/20 Optimized (Win32)	1/25 Modular (Win32)	Improvement (vs Baseline)
Serialization	Simple	12.51	25.09	11.61	7.2%
	Complex	37.00	24.50	18.84	49.1%
	Nested	41.75	48.42	43.67	-4.6%
Deserialization	Simple	22.41	25.61	21.61	3.6%
	Complex	44.90	34.33	39.39	12.3%
	Nested	132.87	146.51	132.37	0.4%

Conclusion

The engine overhaul is a resounding success. By moving safety checks (circular references, recursion depth) and type validation into compiled bytecode, Lodum achieves high speed without sacrificing the robustness inspired by Rust's serde. The improvements are format-agnostic, providing immediate benefits to JSON, MsgPack, CBOR, and even YAML (which saw proportional gains despite high parser overhead).