How AI Made MessagePack Decoding 20% Faster in Swift
While profiling Secrets 4 before launch, I found some clear room for improvement. Whenever Secrets unlocked a database, it spent a considerable chunk of time decoding its data using MessagePacker.
MessagePacker is a Swift implementation of MessagePack, a serialization format that's a bit like JSON, but binary. I chose it after benchmarking several serialization libraries that support Swift's Codable. It struck a good balance between speed and storage size, and I eventually even contributed to the project.
But in Instruments, the number of allocations looked far too high, which was likely making the library slower than it needed to be. A quick look through MessagePacker's implementation gave me a few ideas for reducing those allocations. For the moment, though, it was good enough. I filed the optimization away as a TODO, along with some notes on what I wanted to try.
Apart from adding a note about Swift Binary Parsing, that TODO sat untouched for three years.
Until yesterday.
Finally tackling the TODO
I finally had some room to take a stab at it. Since so much time had passed, I started by searching for newer MessagePack implementations. Perhaps someone had already built a faster one.
Sure enough, I found msgpack-swift, which claimed to be up to twice as fast as MessagePacker. I added it to my CodableBenchmarks repo and confirmed the claim.
Looking through its code, I realized my old notes no longer applied. It handled the data much more efficiently and was already better than what I had imagined building.
"Great," I thought. "I can test this with Secrets and, if it works, finally check off that TODO."
But before dinner, I kept thinking about the last note I had added: "Could Swift Binary Parsing make it even faster?"
The library was built for exactly this kind of work…
Codex, you're it
I was already happy with msgpack-swift, but we live in the AI age now. So I asked Codex1 to analyze the library and suggest ways to integrate Swift Binary Parsing. It proposed a few approaches that could improve performance.
I gave it a concrete goal of make decoding at least 10% faster in my CodableBenchmarks repo. Then I let it at it while I ate dinner.
When I checked back, this was waiting for me:
Implemented BinaryMessagePack and achieved the target.
10,000-Airport decoding: 0.0483 s vs 0.0598 s — 19.22% faster
Decoding allocations: 61.77% lower
Encoding: 3.54% faster
Encoded size unchanged: 1,519,215 bytes
Improvement remained 17.74–19.22% across final hardened-code runs.
The decisive improvement came from replacing the eager decoded-value tree with a direct decoder using sequential array traversal, lazy keyed-map indexing with stable offsets, and direct scalar decoding. Phase 1 remains preserved for comparison.
Validation completed:
• 17 standalone codec tests passed.
• Full-value Airport tests passed at 100, 1,000, and 10,000 records.
• Cross-decoding with msgpack-swift passed in both directions.
• Six final paired benchmark tests passed.
• git diff --check is clean.
The result is suitable as an experimental fork, but needs broader conformance tests, fuzzing, and multi-platform CI before upstreaming. It pins swift-binary-parsing 0.0.2, requires Swift tools 6.2 and experimental Lifetimes, raises minimum platforms to macOS 13/iOS 16-equivalent, and includes both upstream MIT and Binary Parsing Apache licensing.
Goal usage: 886,403 tokens over approximately 1h 31m 37s.
In about 90 minutes, it had made decoding nearly 20% faster while cutting allocations by more than 60% 😳
It was late, and I was reading this on my phone from the couch, but I had to run the tests and skim through the code myself. So I got up, went to my laptop, and confirmed it was legit. It hadn't found a clever way to game the benchmarks; it had built something genuinely better in just 90 minutes.
I went to bed not quite sure how I felt. It was a mix of amazement, excitement, and anxiety. It's now the next day, and I'm still not sure how I feel about this new era… but I had to write about it.
For now, I think I'll just try to make the best of it.
The library is called MessagePackable. I haven't tested it with Secrets yet. But it's out there.
Footnotes
-
Using GPT-5.6 Sol with Extra High reasoning. ↩