feat: Binary serialisation format for hugr-model based on capnproto.

[zrho]

A binary serialisation format for hugr-model based on capnproto.

• Export from hugr-core to hugr-model now deduplicates terms on the fly.
• Removed types on ports and instead used the signature. This allows for more deduplication and also provides a natural place for extension set annotations which was missing before.
• Added the NodeId of the node that defines a parameter to the indexed variant of a LocalRef so that deduplication does not accidentally merge two local variables that refer to different parameters.

Serializing the same 2000 2-qubit gate + 1000 1-qubit gate circuit by going from core via the model takes 5.5682 ms per roundtrip on Apple M3. Meanwhile the JSON roundtrip baseline is 10.760 ms. A significant portion of the time is spent on needless copies of signatures since export and import have to interact with the hugr-core API (#1551), so there is still a lot of potential to make this faster yet.

[codecov]

Codecov Report

Attention: Patch coverage is 75.13228% with 235 lines in your changes missing coverage. Please review.

Project coverage is 85.68%. Comparing base (307937e) to head (9781b14).
Report is 9 commits behind head on main.

Files with missing lines	Patch %	Lines
hugr-model/src/v0/binary/read.rs	69.49%	36 Missing and 43 partials ⚠️
hugr-core/src/import.rs	73.11%	16 Missing and 34 partials ⚠️
hugr-core/src/export.rs	85.18%	33 Missing and 3 partials ⚠️
hugr-model/src/v0/binary/write.rs	79.35%	31 Missing and 1 partial ⚠️
hugr-model/src/v0/text/print.rs	57.44%	5 Missing and 15 partials ⚠️
hugr-model/src/v0/text/parse.rs	65.78%	2 Missing and 11 partials ⚠️
hugr-model/src/v0/mod.rs	70.58%	5 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1557      +/-   ##
==========================================
- Coverage   85.72%   85.68%   -0.04%     
==========================================
  Files         132      134       +2     
  Lines       24017    24512     +495     
  Branches    21017    21512     +495     
==========================================
+ Hits        20588    21003     +415     
- Misses       2365     2411      +46     
- Partials     1064     1098      +34     

Flag	Coverage Δ
python	92.46% <ø> (ø)
rust	84.73% <75.13%> (-0.03%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[zrho]

The filesize for the capnp serialization is also better than the JSON baseline: the 2000 2-qubit gate + 1000 1-qubit gate circuit serializes to 295kb with binary serialization in contrast to 695K for JSON. This is not just due to being a binary format, but also due to term deduplication. Note that the JSON serialization also does a form of this by serializing the prelude.qubit type simply as "Q". But whereas this only works for qubits for the JSON serializer, the term deduplication performed by the export to the model works for all types.

The file size can be driven down further and the speed increased once we have custom operation declarations: Then nodes that use a custom operation don't have to repeat the operation's full name every time, but can simply refer to the declaring node by index.

This form of deduplication is not only useful to decrease file size; that could similarly be achieved by just compressing the output (as was suggested here: #1307). When reading a deduplicated file back in, we can use the sharing to also get an efficient in-memory representation, and to avoid redundantly validating the same term over and over. At this point, we do not make good use of this yet since the hugr-core data structures do not allow any sharing.

[croyzor]

What's the invalid for?

[zrho]

See the docs on Operation::Invalid in the Rust code. It serves two purposes:

• being able to represent modules whose nodes have non-contiguous indices without needing to compact
• a placeholder operation for when you have partially constructed a module. This was useful to break a cycle in export after adding the defining node index in LocalRef.

[ss2165]

looks good, minor queries

[ss2165]

🚀