3rd-Party Engines

Scapy Support

The Scapy engine for Extractor.

Note

Constructing this engine imports scapy.all, whose layer modules fill Scapy’s layer registries – conf.l2types and the bind_layers payload table – as an import side effect. A narrower import leaves them empty, and PcapReader then returns every frame as one opaque Raw layer without raising, saying so only on sys.stderr. Importing the layer modules by name repairs the link layer but leaves the payload table short, so dhcp.pcapng still reads as Ethernet / IP / UDP / Raw, and such a list goes stale as Scapy gains layer types.

scapy.all also loads scapy.layers.dcerpc, which reaches Scapy’s TLS layer and there triggers a CryptographyDeprecationWarning from cryptography about finite-field Diffie-Hellman. pcapkit never runs that key-exchange path, but the warning subclasses UserWarning rather than DeprecationWarning, so Python’s default filters show it.

pcapkit leaves it unfiltered on purpose: it is Scapy’s to emit and the consumer’s to silence, like every other category (see pcapkit.utilities.warnings):

import warnings

from cryptography.utils import CryptographyDeprecationWarning

warnings.filterwarnings('ignore', category=CryptographyDeprecationWarning)

class pcapkit.foundation.engines.scapy.Scapy(extractor)

Bases: EngineBase[ScapyPacket]

Scapy engine support.

Parameters:

extractor (Extractor) – Extractor instance.

__engine_name__: str = 'Scapy'

Engine name.

__engine_module__: str = 'scapy'

Engine module name.

__init__(extractor)

Initialise the engine.

Parameters:

extractor (Extractor) – Extractor instance.

run()

Call scapy.sendrecv.sniff() to extract PCAP files.

This method assigns self._extmp as an iterator from scapy.sendrecv.sniff(), reached through self._expkg – which __init__() binds to scapy.all, since that is the import that populates the layer registries sniff needs to dissect anything.

The global header is parsed and dumped first, by self.extractor.record_header.

Warns:

AttributeWarning – If self.extractor._exlyr and/or self.extractor._exptl is provided as the Scapy engine does not support such operations; or if self.extractor._exctx is provided, as the Scapy engine does not parse with pcapkit’s own protocol implementations; or if DTLS reassembly is enabled, for the same reason.

read_frame()

Read frames with Scapy engine.

Return type:

scapy.packet.Packet

Returns:

Parsed frame instance.

See also

Please refer to PCAP.read_frame for more operational information.

_expkg: _ScapyPackage

Engine extraction package, i.e. scapy.all.

_extmp: Iterator[scapy.packet.Packet]

Engine extraction temporary storage.

DPKT Support

The DPKT engine for Extractor.

Note

dpkt.pcapng.Reader reads only the first Interface Description Block of a file, and applies its link type and timestamp resolution to every packet. This engine therefore reads PCAP-NG with PCAPNGReader instead, which resolves each packet’s link type, if_tsresol and if_tsoffset from the interface it names, in its own section (#1379).

dpkt.pcapng.Reader also skips Simple Packet Blocks, which DPKT has no type for; PCAPNGReader reads them through pcapkit’s own PCAPNG and, since the block carries no timestamp, dates them 0 (#1520).

class pcapkit.foundation.engines.dpkt.DPKT(extractor)

Bases: EngineBase[DPKTPacket]

DPKT engine support.

A PCAP-NG file is read by PCAPNGReader rather than by dpkt.pcapng.Reader, so each packet takes the link type and the timestamp resolution and offset of the interface it was captured on.

It reads the Enhanced, the Simple and the (obsolete) Packet Block, and skips every other block without a warning. DPKT’s own reader skips the Simple Packet Block too.

Parameters:

extractor (Extractor) – Extractor instance.

__engine_name__: str = 'DPKT'

Engine name.

__engine_module__: str = 'dpkt'

Engine module name.

run()

Call dpkt.pcap.Reader to extract PCAP files.

This method assigns self._expkg as dpkt and self._extmp as an iterator from dpkt.pcap.Reader, or from PCAPNGReader for a PCAP-NG file.

The global header is parsed and dumped first, by self.extractor.record_header.

Warns:

AttributeWarning – If self.extractor._exlyr and/or self.extractor._exptl is provided as the DPKT engine does not support such operations; or if self.extractor._exctx is provided, as the DPKT engine does not parse with pcapkit’s own protocol implementations; or if DTLS reassembly is enabled, for the same reason.

Raises:

FormatError – If the file format is not supported, i.e., not a PCAP and/or PCAP-NG file.

read_frame()

Read frames with DPKT engine.

Return type:

dpkt.dpkt.Packet

Returns:

Parsed frame instance.

See also

Please refer to PCAP.read_frame for more operational information.

_expkg: _DPKTPackage

Engine extraction package, i.e. dpkt.

_extmp: Reader | PCAPNGReader

Engine extraction temporary storage.

class pcapkit.foundation.engines.dpkt.PCAPNGReader(file)

Bases: object

PCAP-NG reader for the DPKT engine, aware of every interface.

dpkt.pcapng.Reader reads only the first Interface Description Block (IDB) of a file, and applies its link type and timestamp resolution to every packet (#1379). This reader walks the blocks itself instead: it parses each Section Header Block and IDB with pcapkit’s own PCAPNG, keeps the interfaces of the current section, and resolves every packet’s link type, if_tsresol and if_tsoffset from the interface it names. Packet blocks are still decoded by DPKT.

It yields Enhanced Packet Blocks, (obsolete) Packet Blocks and Simple Packet Blocks, and skips every other block. dpkt.pcapng.Reader skips the Simple Packet Block too, and DPKT has no type for one, so that block is parsed with PCAPNG as well (#1520).

Parameters:

file (IO[bytes]) – Source PCAP-NG stream, positioned at its first Section Header Block.

datalink()

Link type of the interface the packet last yielded was captured on.

Return type:

LinkType

PyShark Support

The PyShark engine for Extractor.

Important

PyShark imports cleanly with its Python ceiling or tshark unmet, and fails only when used, so unsupported_reason() checks both up front.

Python 3.13 or older. pyshark 0.6 builds its event loop with asyncio.get_event_loop_policy().get_event_loop(). With no running loop, 3.10 and 3.11 return a loop silently, 3.12 returns one with a DeprecationWarning, and 3.14 raises RuntimeError: There is no current event loop in thread 'MainThread'; it extracts a capture on 3.13.

Wireshark’s tshark binary. pyshark shells out to it and parses nothing itself. It need not be on PATH: pyshark consults tshark_path in its config.ini first, then PATH on POSIX, both Program Files directories on Windows, and /Applications/Wireshark.app on macOS. The check uses pyshark’s own resolver, so a configured install off PATH is not refused.

Read access under AppArmor. From Ubuntu 25.04, tshark runs confined by an enforcing AppArmor profile, /etc/apparmor.d/tshark, which reads captures only under /tmp, so a capture anywhere else fails with You don’t have permission to read the file – which pyshark raises as TSharkCrashException. The check above cannot see this. To allow a directory, add a rule for it to the profile’s local include and reload the profile:

echo 'file r /path/to/captures/**,' | sudo tee -a /etc/apparmor.d/local/tshark
sudo apparmor_parser -r /etc/apparmor.d/tshark

class pcapkit.foundation.engines.pyshark.PyShark(extractor)

Bases: EngineBase[PySharkPacket]

PyShark engine support.

Parameters:

extractor (Extractor) – Extractor instance.

__engine_name__: str = 'PyShark'

Engine name.

__engine_module__: str = 'pyshark'

Engine module name.

PYTHON_CEILING = (3, 14)

First Python version PyShark does not work on, as a (major, minor) pair. Released 0.6 builds its event loop with asyncio.get_event_loop_policy().get_event_loop(), and Python 3.14 made asyncio.get_event_loop() raise RuntimeError when no current event loop exists instead of quietly creating one.

classmethod unsupported_reason()

Why this engine cannot run here, or None when it can.

Consulted when Extractor settles its engine, before the import test, because nothing that stops this engine is visible to an import. PyShark imports fine and fails when used, which without this hook escapes from run() as a hard error rather than degrading to the default engine with a warning.

The interpreter. pyshark 0.6 calls asyncio.get_event_loop_policy().get_event_loop() in Capture._setup_eventloop, in a fresh interpreter with no running loop. Measured: 3.10 and 3.11 return a loop silently, 3.12 returns one with a DeprecationWarning, and 3.14 raises RuntimeError: There is no current event loop in thread 'MainThread'; on 3.13 pyshark extracted a capture. Hence PYTHON_CEILING is (3, 14).

The tshark binary. pyshark wraps Wireshark’s command-line tool and parses nothing itself. The check asks pyshark’s own get_process_path() rather than shutil.which(), which would refuse setups that work: pyshark reads tshark_path from its config.ini first, then PATH on POSIX, both Program Files directories on Windows, and /Applications/Wireshark.app on macOS.

Note

Deliberately not cached: it runs per Extractor rather than per frame, far below the cost of opening the capture, and caching would keep a stale answer after Wireshark is installed or PATH is fixed.

Return type:

str | None

Returns:

A short phrase naming the limitation, or None.

run()

Call pyshark.FileCapture to extract PCAP files.

This method assigns self._expkg as pyshark and self._extmp as an iterator from pyshark.FileCapture.

The global header is parsed and dumped first, by self.extractor.record_header.

Warns:

AttributeWarning – Warns under the following circumstances:

• if self.extractor._exlyr and/or self.extractor._exptl is provided as the PyShark engine does not support such operations.

• if reassembly is enabled, as the PyShark engine does not support such operation.

• if self.extractor._exctx is provided, as the PyShark engine does not parse with pcapkit’s own protocol implementations.

read_frame()

Read frames with PyShark engine.

A tshark record that is not a packet, such as a PCAP-NG systemd Journal Export Block, is skipped. Frames are numbered by count, not by tshark’s frame number.

Return type:

pyshark.packet.packet.Packet

Returns:

Parsed frame instance.

See also

Please refer to PCAP.read_frame for more operational information.

close()

Close engine.

This method is to be used for closing the engine instance. It is to close the engine instance after the extraction process is finished.

A no-op if run() never opened the capture, which is the case when the Extractor constructor raises before it (#2107).

PyShark can raise TSharkCrashException with retcode: 255 here for a tshark that exited cleanly (#2633). When a subprocess.Popen.poll() reaps a tshark that exited before the event loop handled its SIGCHLD, the child watcher then finds no child and reports 255. Its cleanup kills tshark through asyncio, whose kill on CPython 3.9 to 3.12 goes through subprocess.Popen.send_signal(), which polls first (CPython gh-87744). That is background, not a gate: the check below reads the evidence the loss leaves, on every Python.

The Popen keeps the status its poll read, so that crash is logged at logging.DEBUG instead of raised, but only when the capture was read to its end and every tshark it still lists reports 255 to asyncio and 0 to its Popen. A tshark that failed, or exited 255 itself, keeps that status there and still raises, as does any other error. The one case this cannot tell apart is a tshark exiting 255 that the child watcher reaps just before the poll, which then finds no child and records 0.

_expkg: _PySharkPackage

Engine extraction package, i.e. pyshark.

_extmp: FileCapture

Engine extraction temporary storage.

PyPCAP Support

The PyPCAP engine for Extractor.

Important

PyPCAP publishes no wheels, so installing it compiles a C extension, which needs both the libpcap(3) headers and its shared library. It is therefore not in the all extra; install it on its own once libpcap is present:

pip install pypcapkit[PyPCAP]

On Python 3.12 and newer it cannot be installed at all, which is why the extra carries a python_version < '3.12' marker. PyPCAP 1.3.0 ships a pcap.c pre-generated by Cython 0.29.32 and never runs Cython at build time, and that generated C does not compile against the 3.12+ C API. With libpcap present:

Python	pip install pypcap
3.10	builds, imports
3.11	builds, imports
3.12	fails – ob_digit, curexc_traceback
3.14	fails – those, plus ma_version_tag and the _PyLong_AsByteArray arity

Upstream is unmaintained (see pynetwork/pypcap#116), so the cap is not expected to lift. Use PCAP_CT on 3.12 and newer.

Important

pcap-ct installs the same top-level pcap module as PyPCAP. This engine therefore checks which of the two it got and raises UnsupportedCall when it is pcap-ct, naming engine='pcap_ct' in the message. Running pcap-ct under the PyPCAP name would misreport the engine in use, whose install requirements and version coverage differ.

A pcap module that is installed but cannot load – its import raises OSError, say, because no libpcap(3) can be found – is reported the same way PCAP_CT reports it, so the extraction falls back to the default engine with a warning instead of aborting (#2132).

Important

PyPCAP is a libpcap(3) binding aimed primarily at live capture. Offline it performs no protocol dissection: each frame is the (timestamp, bytes) pair that pcap_next_ex() produced. Reassembly and flow tracing are therefore unavailable and are disabled – with an AttributeWarning – when requested.

The engine also reads PCAP savefiles only, and only from a file on disk. PCAP-NG is rejected with a FormatError, because libpcap(3) opens such a file without complaint and then yields no frames at all; and a non-file input is rejected with an UnsupportedCall, because pcap_open_offline() opens a savefile by name.

class pcapkit.foundation.engines.pypcap.PyPCAP(extractor)

Bases: EngineBase[RawFrame]

PyPCAP engine support.

PyPCAP is a binding over libpcap(3), primarily aimed at live capture. Offline it is a savefile reader and nothing more: iteration yields the (timestamp, bytes) pair from pcap_next_ex() and no protocol dissection is performed. Consequently this engine

• returns each frame as a (timestamp, bytes) tuple rather than as a parsed packet object, and

• disables both reassembly and flow tracing, warning as it does so, since neither can be derived without an IP or TCP layer to read.

It also requires the input to be a real file on disk, because pcap_open_offline() opens by name – there is no way to hand libpcap(3) an already-open Python stream.

Important

This engine is upstream PyPCAP specifically, which cannot be installed on Python 3.12 or newer: it ships no wheels and its pcap.c was pre-generated by Cython 0.29.32, which does not compile against the 3.12+ C API. pcap-ct is an independent reimplementation of the same interface that does install there, and PCAP_CT is the engine for it.

Because pcap-ct installs the same top-level pcap module, this engine checks which distribution it got and refuses the other one rather than running under the wrong name – see __init__().

Parameters:

extractor (Extractor) – Extractor instance.

Raises:

UnsupportedCall – If the installed pcap is pcap-ct rather than upstream PyPCAP.

__engine_name__: str = 'PyPCAP'

Engine name.

__engine_module__: str = 'pcap'

Engine module name. Note that this cannot separate the two distributions that own it – see pcapkit.foundation.engines._pcap_backend.probe() and unsupported_reason(), which is where that is done.

__engine_distribution__ = 'pypcap'

Distribution this engine drives.

classmethod unsupported_reason()

Why this engine cannot run here, or None when it can.

Consulted when Extractor settles its engine, before the import test, which cannot answer this: import pcap succeeds when pcap-ct is installed and upstream PyPCAP is not, so this engine would run on a distribution it does not drive and take credit for work pcap-ct did.

The condition is deliberately not a Python version ceiling, even though upstream cannot be installed on 3.12 or newer. What matters is which distribution is present: a version check would refuse an upstream build someone got working on a newer interpreter, and would say nothing on 3.10 and 3.11, where either distribution could be the one in place.

It also reports a pcap that is installed but cannot load, such as the OSError a loader raises when it cannot find libpcap(3). Extractor.import_test lets that through, so without this the extraction would die rather than fall back to the default engine (#2132). The verdict is engine_reason(), shared with PCAP_CT.

Return type:

str | None

Returns:

A phrase naming the real cause, or None. An absent pcap returns None, since the import test reports that case in its own words and duplicating it would produce two warnings for one problem.

property dlink: LinkType

Data link layer protocol, as reported by the capture handle.

property backend: str

The distribution, version and location this engine is driving.

More than one distribution provides pcap, so “the PyPCAP engine failed” is not a complete statement of what ran. This says which one it was, e.g. pypcap 1.3.0 (/.../pcap.cpython-310-x86_64-linux-gnu.so).

run()

Call pcap.pcap to extract PCAP files.

This method assigns self._expkg as pcap and self._extmp as an iterator from pcap.pcap.

Warns:

AttributeWarning – Warns under the following circumstances:

• if self.extractor._exlyr and/or self.extractor._exptl is provided as the PyPCAP engine does not support such operations.

• if self.extractor._exctx is provided, as the PyPCAP engine does not parse with pcapkit’s own protocol implementations.

• if reassembly and/or flow tracing is enabled, as the PyPCAP engine performs no protocol dissection and so cannot support either operation.

Raises:

• FormatError – If the file format is not supported, i.e., not a PCAP file. PCAP-NG is rejected explicitly rather than left to libpcap(3), which opens such a file without complaint and then yields no frames at all.

• UnsupportedCall – If the input is not a file on disk, as pcap_open_offline() can only open a savefile by name.

read_frame()

Read frames with PyPCAP engine.

Return type:

Tuple[float, bytes]

Returns:

The (timestamp, bytes) pair as yielded by pcap.pcap.

See also

Please refer to PCAP.read_frame for more operational information.

close()

Close engine.

This method closes the underlying pcap.pcap handle. It is idempotent, as Extractor._cleanup and Extractor.__exit__ may both reach it, and pcap.pcap.close is not safe to call twice.

_expkg: _PyPCAPPackage

Engine extraction package, i.e. pcap.

_extmp: Iterator[Tuple[float, bytes]]

Engine extraction temporary storage.

_backend: Probe

What the pcap import actually found, cf. backend.

_dlink: LinkType

Data link layer protocol, from the capture handle.

_closed: bool

Closed flag, so that the handle is not closed twice.

__init__(extractor)

Initialise the engine.

Warns:

EngineWarning – If both distributions that provide pcap are installed. Only one of them can win the import, so the other is shadowed and unreachable – a state worth naming, because engine= then selects an engine that cannot run and nothing else would say why.

Raises:

UnsupportedCall – If the installed pcap is pcap-ct rather than upstream PyPCAP, or is installed but cannot load. These are the conditions unsupported_reason() reports, kept here as well because that hook is only consulted by Extractor’s engine selection; constructing the engine directly bypasses it, and running pcap-ct under the PyPCAP name would misreport what did the work. Via Extractor the hook fires first and degrades to the default engine with a warning, so this is a backstop rather than the usual path.

pcap-ct Support

The pcap-ct engine for Extractor.

Important

pcap-ct is an independent ctypes reimplementation of the PyPCAP interface, by a different author, on top of the libpcap distribution. It is a separate engine rather than a second backend for PyPCAP: select it with engine='pcap_ct'.

It exists for coverage. Upstream PyPCAP stops at Python 3.11 (see PyPCAP Support above), while pcap-ct and libpcap both publish py3-none-any wheels, so installing this engine needs no compiler and no pcap.h:

pip install pypcapkit[PCAP_CT]

The two extras’ markers split the supported interpreters between them:

Engine	Python
PyPCAP (pypcap)	below 3.12
PCAP_CT (pcap-ct)	3.10 and newer

Warning

A system libpcap(3) is still required at run time. libpcap ships a vendored libpcap.so under _platform/{linux,macos,windows}/ but does not use it by default: its libpcap.cfg reads LIBPCAP = None as published, which sends the loader to ctypes.util.find_library(), so what gets mapped is the host’s libpcap.so.1.

The library mapped therefore follows the interpreter’s loader search path, so the same wheel can load different libpcap versions under different interpreters. Setting LIBPCAP = tcpdump in libpcap.cfg selects the vendored copy instead.

With no system libpcap(3) at all, import pcap raises OSError rather than ImportError, which would escape Extractor.import_test and abort the extraction. PCAP_CT.unsupported_reason detects it and reports it as an ordinary “engine unavailable”, so the extraction falls back to the default engine with a warning naming the missing library.

Warning

Both distributions are pre-releases only (pcap-ct 1.3.0b3, libpcap 1.11.0b29), which is also why pip install resolves them without --pre. pcap-ct documents itself as tracking the PyPCAP 1.2.3 interface rather than 1.3.0.

Every attribute this engine touches – the pcap.pcap(name=..., promisc=...) constructor, pcap.pcap.datalink, iteration yielding (timestamp, bytes), and pcap.pcap.close – is present on both and behaves identically, byte for byte and timestamp for timestamp, on pcap-ct 1.3.0b3 against pypcap 1.3.0 – a statement about those versions, not a guarantee from upstream. PCAP_CT is therefore not included in the all extra: a beta should be asked for by name.

Important

Being the same interface, pcap-ct has the same limits. It performs no protocol dissection: each frame is the (timestamp, bytes) pair that pcap_next_ex() produced. Reassembly and flow tracing are therefore unavailable regardless of which backend is installed, and are disabled – with an AttributeWarning – when requested; the adapters in pcapkit.toolkit.pcap_ct raise UnsupportedCall rather than fail obscurely. It also reads from a file on disk only, because pcap_open_offline() opens a savefile by name; a non-file input is rejected with an UnsupportedCall.

The timestamp is seconds since the epoch as a float. pcap-ct opens savefiles asking for nanosecond precision via pcap_open_offline_with_tstamp_precision(), so a microsecond-resolution savefile is scaled up by libpcap(3) rather than truncated.

Note

Both engines reject PCAP-NG with a FormatError, because accepting it would be unpredictable rather than merely limited. libpcap(3) can read a PCAP-NG savefile, but how well depends on the version the host provides – which, per the warning above, the libpcap wheel does not pin. Reading examples/captures/dhcp.pcapng:

System libpcap	Result
1.11.0	correct – the same frames and the same sub-second timestamps as PCAPNG
1.5.3	the same frames, but nonsense sub-second timestamps (1102274184.0000002 and the like), silently and with no error

Even on a good version, pcap_datalink() reports a single link type for the whole file, so a capture whose interfaces differ would have one interface’s link type applied to every frame. PCAPNG reads the per-interface blocks properly and does not depend on the host’s library at all, so PCAP-NG is routed there rather than read approximately – or wrongly – here.

class pcapkit.foundation.engines.pcap_ct.PCAP_CT(extractor)

Bases: EngineBase[RawFrame]

pcap-ct engine support.

pcap-ct is a ctypes reimplementation of the PyPCAP API on top of the libpcap package, which supplies the libpcap(3) bindings (and ships a vendored copy of the library that it does not, by default, use – see the warning below). Both distributions install a top-level pcap module and expose the same interface, so this engine is a sibling of PyPCAP rather than a replacement for it – see __engine_module__ for how the two are told apart, and the engine documentation for which one to install.

It exists because upstream PyPCAP cannot be installed on Python 3.12 or newer: it ships no wheels and its pcap.c was pre-generated by Cython 0.29.32, which does not compile against the Python 3.12+ C API. pcap-ct and libpcap both publish py3-none-any wheels, so installing this engine needs no compiler and no pcap.h.

Warning

It does still need a system libpcap(3) at run time, and this is easy to get wrong: libpcap ships a vendored libpcap.so under _platform/{linux,macos,windows}/ but does not use it by default. Its libpcap.cfg reads LIBPCAP = None as published, which sends its loader to ctypes.util.find_library(), so the library actually mapped is the host’s libpcap.so.1, and one wheel can load different libraries under two interpreters whose loader search paths differ.

With no system libpcap(3) at all, import pcap raises OSError – not ImportError – which unsupported_reason() exists partly to catch. And the libpcap(3) version in play is a property of the host rather than of the wheel, which is why the PCAP-NG note on run() does not rely on it. Setting LIBPCAP = tcpdump in libpcap.cfg selects the vendored copy instead.

Being the same interface, it has the same limits. Offline, pcap is a savefile reader and nothing more: iteration yields the (timestamp, bytes) pair from pcap_next_ex() and no protocol dissection is performed. Consequently this engine

• returns each frame as a (timestamp, bytes) tuple rather than as a parsed packet object, and

• disables both reassembly and flow tracing, warning as it does so, since neither can be derived without an IP or TCP layer to read.

It also requires the input to be a real file on disk, because pcap_open_offline() opens by name – there is no way to hand libpcap(3) an already-open Python stream.

Important

Both distributions are published as pre-releases only – pcap-ct 1.3.0b3 and libpcap 1.11.0b29 when measured – and pcap-ct documents itself as tracking the PyPCAP 1.2.3 API rather than 1.3.0. Every attribute this engine touches is present and behaves identically on both (measured on pcap-ct 1.3.0b3 against pypcap 1.3.0), but that is a statement about the versions tested, not a guarantee from upstream.

Parameters:

extractor (Extractor) – Extractor instance.

__engine_name__: str = 'PCAP_CT'

Engine name.

__engine_module__: str = 'pcap._pcap'

Engine module name. Note that this is pcap._pcap, not pcap: pcap-ct and upstream PyPCAP both install a top-level pcap, so importing that name cannot tell which of the two is present, and Extractor.import_test decides engine availability by import alone. pcap._pcap is the pcap-ct implementation module; upstream ships pcap as a single extension module rather than a package, so the submodule import fails there and the two engines stay distinguishable.

__engine_distribution__ = 'pcap-ct'

Distribution this engine drives.

classmethod unsupported_reason()

Why this engine cannot run here, or None when it can.

Consulted when Extractor settles its engine, before the import test, and answers what that test cannot.

Which distribution owns pcap. Upstream PyPCAP owns that name just as legitimately, and import pcap._pcap – what __engine_module__ names – separates them only while upstream ships a single extension module. Asking probe() states the check rather than inferring it from an import that happens to fail.

Whether a system libpcap(3) exists at all. pcap-ct is ctypes and needs no compiler, but at runtime it imports the libpcap distribution, whose Linux loader raises OSError – Cannot find libpcap.so library – when there is none. That is not an ImportError, so Extractor.import_test lets it through and the extraction dies rather than falling back. Reporting it here turns that into the ordinary “engine unavailable” path.

There is deliberately no Python version bound: this engine reads captures identically across the whole supported range. The floor in the PCAP_CT extra’s markers exists only because pyproject.toml advertises requires-python >= 3.8; below 3.10 the distributions are not installed, which the import test reports on its own.

Return type:

str | None

Returns:

A phrase naming the real cause, or None. A pcap that is merely absent returns None, since the import test reports that case in its own words.

property dlink: LinkType

Data link layer protocol, as reported by the capture handle.

property backend: str

The distribution, version and location this engine is driving.

More than one distribution provides pcap, so “the pcap-ct engine failed” is not a complete statement of what ran. This says which one it was, e.g. pcap-ct 1.3.0b3 (/.../pcap/__init__.py).

__init__(extractor)

Initialise the engine.

Warns:

EngineWarning – If both distributions that provide pcap are installed. Only one of them can win the import – measured on Python 3.10 with both present, the pcap-ct package wins and upstream’s extension module is shadowed – so the other is unreachable, and nothing else would say why engine='pypcap' had stopped working.

Raises:

UnsupportedCall – If the environment cannot support this engine, for either of the reasons unsupported_reason() describes. That hook is only consulted by Extractor’s engine selection, which degrades to the default engine with a warning; constructing the engine directly bypasses it, so this is the backstop for that path.

run()

Call pcap.pcap to extract PCAP files.

This method assigns self._expkg as pcap, self._handle as the pcap.pcap capture handle and self._extmp as an iterator over it.

Warns:

AttributeWarning – Warns under the following circumstances:

• if self.extractor._exlyr and/or self.extractor._exptl is provided as the pcap-ct engine does not support such operations.

• if self.extractor._exctx is provided, as the pcap-ct engine does not parse with pcapkit’s own protocol implementations.

• if reassembly and/or flow tracing is enabled, as the pcap-ct engine performs no protocol dissection and so cannot support either operation.

Raises:

• FormatError – If the file format is not supported, i.e., not a PCAP file. PCAP-NG is rejected explicitly even though libpcap(3) can read one – see the note below.

• UnsupportedCall – If the input is not a file on disk, as pcap_open_offline() can only open a savefile by name.

Note

The PCAP-NG rejection is a deliberate narrowing, and the reason is that the alternative is unpredictable rather than merely limited. libpcap(3) can read a PCAP-NG savefile, but how well depends on the version the host happens to provide – which, per the warning in this class’s docstring, is not something the libpcap wheel pins. Both measured on examples/captures/dhcp.pcapng:

• against libpcap 1.11.0 it reads correctly, yielding the same frames and the same sub-second timestamps as PCAPNG;

• against libpcap 1.5.3 it yields the same frames but with nonsense sub-second timestamps – 1102274184.0000002 and the like – silently, with no error.

Even on a good version it cannot represent more than one interface: pcap_datalink() returns a single link type for the whole file, so a capture whose interfaces differ would have one interface’s link type applied to every frame. PCAPNG reads the per-interface blocks properly and does not depend on the host’s library at all, so PCAP-NG is routed there rather than read approximately – or wrongly – here.

read_frame()

Read frames with pcap-ct engine.

Return type:

Tuple[float, bytes]

Returns:

The (timestamp, bytes) pair as yielded by pcap.pcap.

Note

The timestamp is seconds since the epoch as a float. pcap-ct opens savefiles with pcap_open_offline_with_tstamp_precision() asking for nanosecond precision, so a microsecond-resolution savefile is scaled up by libpcap(3) rather than truncated; the resulting value matches the record header’s ts_sec + ts_usec * 1e-6 exactly for examples/captures/in.pcap.

See also

Please refer to PCAP.read_frame for more operational information.

close()

Close engine.

This method closes the underlying pcap.pcap handle. It is idempotent, as Extractor._cleanup and Extractor.__exit__ may both reach it.

Note

pcap-ct 1.3.0b3’s own pcap.pcap.close happens to tolerate a second call, where upstream pypcap 1.3.0 segfaults on one. The _closed flag is kept regardless: it is not this engine’s business to depend on a pre-release’s undocumented forgiveness. An engine that was never opened has no _handle to close, and is skipped too.

_expkg: _PCAPCTPackage

Engine extraction package, i.e. pcap.

_handle: pcap

Capture handle, kept separately from the iterator it is read through so that close() does not depend on iter(handle) is handle.

_extmp: Iterator[Tuple[float, bytes]]

Engine extraction temporary storage.

_backend: Probe

What the pcap import actually found, cf. backend.

_dlink: LinkType

Data link layer protocol, from the capture handle.

_closed: bool

Closed flag, so that the handle is not closed twice.

PyPCAPFile Support

The PyPCAPFile engine for Extractor.

Important

PyPCAPFile is a pure Python savefile reader that decodes Ethernet, IPv4, TCP and UDP and nothing else. IPv6 reassembly is therefore unavailable and is disabled – with an AttributeWarning – when requested; IPv4 and TCP reassembly and TCP flow tracing remain available. PCAP-NG is rejected with a FormatError. PyPCAPFile 0.12.0 drops every record of a big-endian savefile, every record from a zero-length one on, and a truncated final record, so the engine reads the records itself, into PyPCAPFile’s own packet objects – see _read_records().

class pcapkit.foundation.engines.pypcapfile.PyPCAPFile(extractor)

Bases: EngineBase[PCAPFilePacket]

PyPCAPFile engine support.

PyPCAPFile is a pure Python savefile reader. It decodes Ethernet, IPv4, TCP and UDP, and nothing else – in particular there is no IPv6 decoder and no PCAP-NG support. Consequently this engine

• reads PCAP savefiles only, raising FormatError on PCAP-NG,

• reads the savefile’s records itself, into PyPCAPFile’s own packet objects, as PyPCAPFile 0.12.0 drops some of them – see _read_records(),

• disables IPv6 reassembly, warning as it does so, while leaving IPv4 and TCP reassembly and TCP flow tracing in place, and

• leaves TCP over IPv6 out of TCP reassembly and flow tracing, with one AttributeWarning per capture when the IPv6 packet is straight over Ethernet, and TCP behind a VLAN tag, in a tunnel or behind an extension header (an IPv6 one, or AH in IPv4) out without one – see pcapkit.toolkit.pypcapfile.

The engine stops decoding at the network layer rather than descending into the transport layer. PyPCAPFile decoders replace the payload bytes of the layer they decode, so descending further would discard the verbatim TCP segment that tcp_reassembly() needs in order to report an exact header/payload split.

Parameters:

extractor (Extractor) – Extractor instance.

__engine_name__: str = 'PyPCAPFile'

Engine name.

__engine_module__: str = 'pcapfile'

Engine module name.

LAYERS = 2

Number of layers to descend while decoding, i.e. link plus network. See the class docstring for why this stops short of the transport layer.

PYTHON_CEILING = (3, 12)

First Python version PyPCAPFile does not work on, as a (major, minor) pair. Released 0.12.0 imports imp from pcapfile.linklayer, and imp was removed in Python 3.12.

classmethod unsupported_reason()

Why this engine cannot run here, or None when it can.

Consulted when Extractor settles its engine, before the import test, which cannot answer this. pcapfile’s top-level package imports on Python 3.12 and newer; it is pcapfile.linklayer that fails, imported by pcapfile.savefile. So import pcapfile succeeds and the ModuleNotFoundError escapes from __init__() as a hard error instead of degrading to the default engine with a warning.

The version is checked rather than the import attempted, so the answer does not depend on which submodule happens to be imported first.

Return type:

str | None

Returns:

A short phrase naming the limitation, or None.

property dlink: LinkType

Data link layer protocol, as reported by the savefile header.

run()

Call pcapfile.savefile.load_savefile to extract PCAP files.

This method assigns self._expkg as pcapfile and self._extmp as an iterator over the lazily generated savefile packets.

pcapfile.savefile.load_savefile loads and validates the savefile header. The records are then read by _read_records(), not by the packet generator it returns, which drops records (see there). Like pcapfile.savefile._read_a_packet with layers=0, _read_records() does not hand back each frame’s bytes verbatim – it hexlifies the whole frame into ASCII text and stops there, exactly as pcapfile does for any layer at which it runs out of layers left to descend. It is _decode() that un-hexlifies and then decodes each frame to LAYERS depth using pcapfile.linklayer.clookup – the very call pcapfile makes internally. Doing it this way round is what lets the link layer type be inspected (and reported on) before the first frame is decoded.

Warns:

AttributeWarning – Warns under the following circumstances:

• if self.extractor._exlyr and/or self.extractor._exptl is provided as the PyPCAPFile engine does not support such operations.

• if self.extractor._exctx is provided, as the PyPCAPFile engine does not parse with pcapkit’s own protocol implementations.

• if IPv6 reassembly is enabled, as pcapfile has no IPv6 decoder.

• if DTLS reassembly is enabled, as it reads pcapkit’s own DTLS layer.

• if pcapfile has no decoder for the savefile’s link layer type, in which case frames are left undecoded.

Raises:

FormatError – If the file format is not supported, i.e., not a PCAP file. pcapfile reads libpcap savefiles only.

read_frame()

Read frames with PyPCAPFile engine.

Return type:

pcap_packet

Returns:

Parsed frame instance.

See also

Please refer to PCAP.read_frame for more operational information.

_expkg: _PCAPFilePackage

Engine extraction package, i.e. pcapfile.

_extmp: Iterator[pcap_packet]

Engine extraction temporary storage.

_dlink: LinkType

Data link layer protocol, from the savefile header.

_declf: Callable[..., Any] | None

Link layer decoder for this savefile, or None when pcapfile has none for its link layer type.

__init__(extractor)

Import pcapfile and the submodules it reads savefiles with.

Parameters:

extractor (Extractor) – Extractor instance.

Internal Definitions

PyPCAPFile._get_decoder(linktype)

Return the pcapfile link layer decoder for a link layer type.

Parameters:

linktype (int) – Link layer type code, from the savefile header.

Return type:

Callable[..., Any] | None

Returns:

The decoder class, or None when pcapfile has none.

Warns:

AttributeWarning – If no decoder is available, as frames will then be left hexlified – see run() – and no reassembly or flow tracing is possible.

PyPCAPFile._read_records(file, header)

Read the savefile’s records, every one of them.

pcapfile.savefile._read_a_packet (pypcapfile 0.12.0) drops records:

• It picks the byte order of each record header by testing hdrp[0].byteorder == 'big' – bytes against str, never true on Python 3. A big-endian record header is unpacked little-endian, its byte-swapped capture length overruns the file, and the savefile ends at its first record (#1512).

• It ends the savefile at any record whose frame data is empty or shorter than its capture length, so a zero-length record drops itself and every record after it, and a truncated final record is dropped (#1571).

This reads each record in the byte order of the header pcapfile loaded, into the pcapfile.structs.pcap_packet that function builds with layers=0 – the frame hexlified, pointing at the savefile’s own header – so _decode() and pcapkit.toolkit.pypcapfile see what pcapfile hands them. It stops at a tail too short for a record header, and keeps a record whose capture length runs past the data: the capture length as declared, the octets present as the frame.

Parameters:

• file (NamedStream) – The savefile, just past its header, where pcapfile.savefile.load_savefile leaves it.

• header (__pcap_header__) – The savefile header pcapfile loaded.

Yields:

pcap_packet – Undecoded savefile packets, one per record.

Warns:

ProtocolWarning – If a record’s capture length runs past the data.

PyPCAPFile._decode(packet, frnum)

Decode a raw savefile packet down to LAYERS depth.

A new pcapfile.structs.pcap_packet is built rather than the given one mutated, so that a decoding failure leaves the original intact.

Parameters:

• packet (pcap_packet) – Undecoded savefile packet, i.e. as loaded with layers=0, whose packet.packet is therefore the hexlified frame (see run()), not the raw bytes.

• frnum (int) – Frame number, for the warning message below.

Return type:

pcap_packet

Returns:

The decoded packet, or packet unchanged when it could not be decoded, or holds no octets at all.

Warns:

AttributeWarning – If pcapfile could not decode the frame, e.g. because the capture is truncated. One bad frame should not abort the extraction, but it should not pass silently either.

Backend Detection

PyPCAP, a Cython extension module, and pcap-ct, a ctypes package, both install a top-level pcap module, so import pcap resolves to whichever the import system finds first. This module tells PyPCAP and PCAP_CT which one they got, shared so the two engines cannot disagree. It only detects, and imports nothing from pcapkit but pcapkit.corekit.infoclass, which imports nothing from pcapkit.foundation, so it cannot introduce an import cycle.

pcapkit.foundation.engines._pcap_backend.PYPCAP = 'pypcap'

Distribution name of upstream PyPCAP.

pcapkit.foundation.engines._pcap_backend.PCAP_CT = 'pcap-ct'

Distribution name of pcap-ct.

pcapkit.foundation.engines._pcap_backend.DISTRIBUTIONS = ('pypcap', 'pcap-ct')

Every distribution known to provide pcap, in a fixed order so that messages naming several of them read the same way every time.

pcapkit.foundation.engines._pcap_backend.ENGINE_NAMES = {'pcap-ct': 'pcap_ct', 'pypcap': 'pypcap'}

The engine= string that drives each distribution. Kept here rather than in either engine so that a message pointing the user at the other engine cannot name one that does not exist.

class pcapkit.foundation.engines._pcap_backend.Probe(name, version, origin, failure, missing, installed)

Bases: Info

What one attempt to import pcap found.

Note

This is an Info subclass, so it is a Mapping rather than a tuple: its fields are reached by name, not by position, and it cannot be unpacked as a sequence.

name: str | None

Distribution that provided the imported module – PYPCAP, PCAP_CT, or None when the import did not succeed.

version: str | None

pcap.__version__, when there was a module to read it from.

origin: str | None

pcap.__file__, which is what distinguishes an extension module from a package directory to a human reading a bug report.

failure: str | None

Why the import failed, as a short phrase, or None on success.

missing: bool

Whether the failure was simply “not installed”, i.e. an ImportError. This matters because Extractor.import_test already reports that case perfectly well, whereas the other kind of failure escapes it – see probe().

installed: tuple[str, ...]

Distributions found installed, whether or not their module was importable. More than one means the collision described in this module’s docstring.

describe()

A one-line description fit for a warning or a bug report.

Return type:

str

Returns:

Something like pcap-ct 1.3.0b3 (/.../pcap/__init__.py), or a phrase naming the failure when there is no module to describe.

class pcapkit.foundation.engines._pcap_backend.Memo

Bases: object

One extraction’s probe(), taken at most once (#2113).

Engine selection asks unsupported_reason() in the Extractor constructor and again in run(), and the engine’s own constructor probes once more. While a memo is entered, probe() answers from it, probing on first use. Each extraction holds its own memo, so nothing is cached across extractions.

found: Probe | None

The probe taken under this memo, or None before the first.

pcapkit.foundation.engines._pcap_backend.probe()

Import pcap and report what was found.

Not cached, except within a Memo, which an extraction enters so that its engine selection and engine constructor share one probe (#2113). The cost after a successful first call is a sys.modules lookup plus the dist-info scan in installed_distributions(), and a process-wide cache would make the answer depend on when it was first asked – which the tests, and anything that manipulates sys.path, would have to work around. It is idempotent instead, via _purge().

Note

The bare except Exception is deliberate and is much of the point of this function. pcap-ct imports the libpcap distribution, whose Linux loader calls ctypes.util.find_library() and raises OSError – Cannot find libpcap.so library – when no system libpcap(3) is present. OSError is not an ImportError, so Extractor.import_test does not catch it and it escapes as a hard error instead of degrading to the default engine. Catching it here is what lets an engine report it as a reason instead.

Return type:

Probe

Returns:

A Probe describing the outcome.

pcapkit.foundation.engines._pcap_backend.identify(module)

Which distribution does an imported pcap module come from?

pcap-ct ships pcap as a package whose __init__ does from ._pcap import *, which binds the submodule as an attribute; upstream PyPCAP ships a single extension module, which has no such attribute. That is a structural difference rather than a cosmetic one, which is why it is preferred here over the alternatives:

• pcap.__version__ is 1.3.0b3 against 1.3.0 upstream, but that is a coincidence of release timing and would stop separating them the moment pcap-ct cuts a 1.3.0 final.

• pcap.ex_name looked like a pcap-ct marker and is not – measured present on upstream pypcap 1.3.0 as well.

Parameters:

module (ModuleType) – An already-imported pcap module.

Return type:

str

Returns:

PCAP_CT or PYPCAP.

pcapkit.foundation.engines._pcap_backend.installed_distributions()

Which of DISTRIBUTIONS are installed, per package metadata.

Asked of the metadata rather than of pcap itself, because that is the only way to see the distribution the import did not resolve to – which is exactly the collision worth reporting.

Only the existence of a dist-info is asked about, through importlib.metadata.distributions() rather than importlib.metadata.distribution(). From Python 3.15 the latter parses every candidate’s METADATA file to prefer one that has metadata, and since this runs on every pcap_ct extraction, that parse made the engine about 2.3x slower per packet (#2110). Either call finds a distribution exactly when the other does.

Return type:

tuple[str, ...]

Returns:

The installed subset of DISTRIBUTIONS, in that order.

pcapkit.foundation.engines._pcap_backend.wrong_backend_reason(wanted, found)

Why wanted cannot run, when some other distribution owns pcap.

Parameters:

• wanted (str) – The distribution the calling engine drives – PYPCAP or PCAP_CT.

• found (Probe) – The result of probe().

Return type:

str | None

Returns:

A reason naming what was found and which engine= string wants it, or None when wanted is what is installed, or when nothing is – an absent module is not this function’s business, since Extractor.import_test reports that.

pcapkit.foundation.engines._pcap_backend.engine_reason(wanted, found)

Why the engine driving wanted cannot run on found, if it cannot.

The verdict both engines’ unsupported_reason() give, kept in one place so that they agree (#2132).

Parameters:

• wanted (str) – The distribution the calling engine drives – PYPCAP or PCAP_CT.

• found (Probe) – The result of probe().

Return type:

str | None

Returns:

wrong_backend_reason()’s answer when another distribution owns pcap; a phrase carrying the import failure when pcap is installed but raised something other than ImportError – OSError: Cannot find libpcap.so library with no system libpcap(3), say; and None otherwise. An absent module is still None, since Extractor.import_test reports that.

pcapkit.foundation.engines._pcap_backend.collision_reason(found)

A description of both distributions being installed at once, if they are.

Return type:

str | None

Returns:

A phrase naming every installed distribution and which one import pcap actually resolved to, or None when at most one is installed.

Internal Definitions

pcapkit.foundation.engines._pcap_backend._purge()

Drop the pcap and libpcap module trees from sys.modules.

Called after a failed import so that the next probe reproduces the same failure. Without it a second attempt reports a different error.

Both pcap-ct’s and libpcap’s package initialisers open with from .__about__ import * ; del __about__, which is not safe to re-run: the del needs a name that only gets bound as a side effect of importing the submodule fresh. When the initialiser fails part-way – as it does with no system libpcap(3), where the real error is OSError: Cannot find libpcap.so library – Python removes the package it was executing but leaves the __about__ submodule cached, so the retry reaches the del with nothing bound and dies with NameError: name '__about__' is not defined.

That message names neither the missing library nor the package it came from, and it is what the user would otherwise be shown. libpcap is purged as well as pcap because the residue is in whichever of the two got part-way: purging only pcap moved the NameError from one to the other rather than removing it.