Choosing Between utilities and corekit

Which of the two support packages a module belongs in. Settled by the owner on #1712, over sorting modules by packet knowledge or by who uses them: this rule can be checked from the import graph alone, and needed the fewest moves.

The page’s questions, as one decision:

        flowchart TD
    NEW["a new support module,<br/>helper or ContextVar"] --> LAYER{"a schema-layer<br/>helper used<br/>outside it?"}
    LAYER -->|no| OWNED{"a helper in<br/>one protocol's<br/>schema?"}
    LAYER -->|yes| SCHEMA["corekit.schema<br/>one module per group"]
    OWNED -->|"used beyond that<br/>protocol, and generic"| SCHEMA
    OWNED -->|no| VAR{"a ContextVar?"}
    OWNED -->|"used by that<br/>protocol alone"| STAY["stays beside<br/>that schema"]
    VAR -->|"set or read by<br/>several modules"| CTX["utilities.contextvars"]
    VAR -->|no| UTIL{"usable by any<br/>code, importing<br/>no pcapkit but<br/>utilities?"}
    VAR -->|"used by one<br/>module only"| PRIV["private, in<br/>that module"]
    UTIL -->|yes| VER{"differs by<br/>interpreter<br/>version?"}
    UTIL -->|no| CORE["corekit"]
    VER -->|yes| COMPAT["utilities.compat, behind<br/>a sys.version_info gate"]
    VER -->|no| OWN["a utilities module,<br/>not compat"]
    

pcapkit.utilities is for cross-cutting services: exceptions, warnings, logging, the compatibility layer, configuration, and wrappers around third-party libraries such as chardet. Ask of a module: could any code use this service, and does it import nothing from pcapkit outside utilities? Both halves must hold, and nothing covers top-level, in-function and TYPE_CHECKING imports alike. tests/project/test_utilities_import_boundary_1735_unit.py checks every module under pcapkit/utilities/.

pcapkit.utilities.compat holds compatibility implementations and re-exports only. Where a name or behaviour differs by interpreter version, compat re-exports the right one behind a sys.version_info gate rather than a getattr or hasattr probe. Ruled in review of the threading excepthook and AppType union fixes for #1691 and #1694, whose threads do not hold it, and applied to the other probes on #1709. tests/project/test_stdlib_feature_probes_1709_unit.py fails on a getattr or hasattr call naming its attribute in a string literal, where the receiver is a builtin class or a module the file binds with an import statement, or where the attribute is one the test lists with the interpreter version that added it, unless the test exempts the call. A helper that must behave the same on every platform is neither a compatibility implementation nor a re-export, so it gets its own module: fromtimestamp() is in pcapkit.utilities.datetime, ruled in review of the timestamp fix for #2099, whose thread does not hold it.

pcapkit.corekit is for the types the packet model is built from: what pcapkit.protocols, pcapkit.const and pcapkit.foundation subclass, instantiate or decorate with. Ask: is this part of what a protocol or its schema is made of? A support module that fails the utilities question belongs here. pcapkit.corekit.decorators acts on a protocol’s file and falls back to protocol classes, so it moved out of utilities on #1735. corekit may import utilities; the rule forbids the reverse.

pcapkit.corekit.checksum, settled on #2580, fails the utilities question because ChecksumStatus subclasses EnumLookup. The protocols compute and verify checksums through it, so it imports pcapkit only from corekit and utilities, never from pcapkit.protocols, as tests/corekit/test_checksum_import_boundary_2580_unit.py checks. What needs IPv4 and IPv6 stays beside IP_Pseudo. The package has a module per checksum family, so a new family sits beside internet.

Shared Schema Helpers

A helper that code outside the schema layer calls lives in pcapkit.corekit.schema, one module per group, rather than in the module that first needed it; the schema layer is pcapkit.corekit.fields and pcapkit.protocols.schema. It keeps no re-export at its old path. Ruled by the owner in session on #2674, grouping these helpers in one package rather than placing each by the questions above. So pad_length() is in pcapkit.corekit.schema.padding, and keep_short_read() and replay_short_read() in pcapkit.corekit.schema.shortread. The header schema framework stays pcapkit.protocols.schema and its Schema; pcapkit.corekit.schema holds only the helpers shared across the layer.

A capture cut belongs to the built-in short read of Schema. Schema.unpack records the field the data ends inside, through _note_short_read, as __short_read__; _cut_short_read() packs back only the octets read; and keep_short_read() and replay_short_read() carry the record through info and cut a rebuild back by it. A protocol reads and rebuilds a cut header through them, never through an unpack override of its own or a length callable clamped to the octets present, which hides the cut from the record. Ruled in review of #2682 (#2680), and applied to option areas on #2694.

A rebuild is read at the width its parse read the header at. keep_short_read() records that width beside the record, as __short_width__, and Protocol.from_data reads the rebuild at it, so it holds the values and the next layer its parse did. Ruled by the owner on #2761, over a clamp inside IPv6_Ext, whose overrunning length showed the gap.

A field callback tells a zero the cut left from a zero read through the same record: Schema.unpack passes it to the fields after the one the data ends inside, as __short_read__ in their packet. A callback that would refuse such a zero leaves it to the short read, and refuses a read value with a ProtocolError, as the Opt Data Len helpers of pcapkit.protocols.internet.ipv6_ext.schema do (#2727).

A helper in one protocol’s own schema module moves there too once code outside that protocol uses it and it is generic enough to reuse, so other implementors can find it; a helper only its own protocol calls stays beside the schema. Ruled on #2674. A protocol’s own modules are its protocol, data and schema modules. So jumbo_payload_length(), which the IPv6 pseudo-header and the Scapy toolkit read as well as IPv6 itself, is in pcapkit.corekit.schema.jumbogram, while jumbogram_error(), which only IPv6 calls, stays in pcapkit.protocols.internet.ipv6.schema. The IPv6 option helpers in pcapkit.protocols.internet.ipv6_ext.schema belong to the HOPOPT and IPv6-Opts headers that share them, and stay there too.

The field modules and the protocols import these helpers, so a helper module imports neither, nor pcapkit.foundation, outside an if TYPE_CHECKING: block. The package’s __init__.py imports none of its modules, so a field module loads only the group it asks for. tests/project/test_schema_helpers_boundary_2674_unit.py fails on a helper module that does, on a module outside the layer that refers to a function defined in a field module or in pcapkit.protocols.schema, and on a module outside a protocol that refers to a function defined in that protocol’s schema module, by import or as an attribute of the module.

Context Variables

Where a ContextVar lives depends on which modules use it. Settled by the owner on #2616:

• One that more than one module sets or reads lives in pcapkit.utilities.contextvars, under a public name in its __all__. No other module re-exports it, so every reader imports it from there.

• One that only its own module uses stays in that module, private, as _exthdr_rebuilt does in pcapkit.protocols.internet.internet.

A shared variable kept in one of its users makes the others import a private name and ties them to that module’s place in the import graph. A module of bare declarations importing nothing from pcapkit passes the utilities question, so any module may import it – pcapkit.corekit.checksum included, which must not reach pcapkit.protocols. tests/project/test_shared_contextvars_2616_unit.py fails on a module that takes a context variable from anywhere else, by name or as an attribute of an imported module, and on a variable there that is private, missing from __all__, or read by one module alone.