3rd-Party Engines¶

Scapy Support¶

This module contains the implementation for Scapy engine support, as is used by pcapkit.foundation.extraction.Extractor.

Note

Constructing this engine imports scapy.all, which is what populates Scapy’s layer registries – conf.l2types and the bind_layers payload table, both of which exist only as import side effects of the layer modules. Importing a narrower submodule leaves them empty, and PcapReader then returns every frame as one opaque Raw layer without raising, so the engine dissected nothing at all and said so only on sys.stderr. See Scapy.__init__() for why naming the layer modules individually is not a cheaper route to the same place.

One side effect is worth knowing about in advance: scapy.all loads scapy.layers.dcerpc, which reaches Scapy’s TLS layer and there triggers a CryptographyDeprecationWarning from cryptography about finite-field Diffie-Hellman. It concerns a key-exchange code path pcapkit never executes, but it subclasses UserWarning rather than DeprecationWarning, so Python’s default filters show it.

pcapkit deliberately does not filter it away – it is Scapy’s to emit and the consumer’s to silence, on the same footing as every other category (see pcapkit.utilities.warnings):

import warnings

from cryptography.utils import CryptographyDeprecationWarning

warnings.filterwarnings('ignore', category=CryptographyDeprecationWarning)

class pcapkit.foundation.engines.scapy.Scapy(extractor)[source]¶

Bases: EngineBase[ScapyPacket]

Scapy engine support.

Parameters:: extractor (Extractor) – Extractor instance.

__engine_name__ = 'Scapy'¶: Engine name.

__engine_module__ = 'scapy'¶: Engine module name.

__init__(extractor)[source]¶

Initialise the engine.

Parameters:: extractor (Extractor) – Extractor instance.

run()[source]¶

Call scapy.sendrecv.sniff() to extract PCAP files.

This method assigns self._extmp as an iterator from scapy.sendrecv.sniff(), reached through self._expkg – which __init__() binds to scapy.all, since that is the import that populates the layer registries sniff needs to dissect anything.

Warns:: AttributeWarning – If self.extractor._exlyr and/or self.extractor._exptl is provided as the Scapy engine currently does not support such operations; or if self.extractor._exctx is provided, as the Scapy engine does not parse with pcapkit’s own protocol implementations.

read_frame()[source]¶

Read frames with Scapy engine.

Return type:: Packet
Returns:: Parsed frame instance.

See also

Please refer to PCAP.read_frame for more operational information.

_expkg: scapy.all¶: Engine extraction package.

_extmp: Iterator[ScapyPacket]¶: Engine extraction temporary storage.

DPKT Support¶

This module contains the implementation for DPKT engine support, as is used by pcapkit.foundation.extraction.Extractor.

class pcapkit.foundation.engines.dpkt.DPKT(extractor)[source]¶

Bases: EngineBase[DPKTPacket]

DPKT engine support.

Parameters:: extractor (Extractor) – Extractor instance.

__engine_name__ = 'DPKT'¶: Engine name.

__engine_module__ = 'dpkt'¶: Engine module name.

run()[source]¶

Call dpkt.pcap.Reader to extract PCAP files.

This method assigns self._expkg as dpkt and self._extmp as an iterator from dpkt.pcap.Reader.

Warns:: AttributeWarning – If self.extractor._exlyr and/or self.extractor._exptl is provided as the DPKT engine currently does not support such operations; or if self.extractor._exctx is provided, as the DPKT engine does not parse with pcapkit’s own protocol implementations.
Raises:: FormatError – If the file format is not supported, i.e., not a PCAP and/or PCAP-NG file.

read_frame()[source]¶

Read frames with DPKT engine.

Return type:: Packet
Returns:: Parsed frame instance.

See also

Please refer to PCAP.read_frame for more operational information.

_expkg: dpkt¶: Engine extraction package.

_extmp: Reader¶: Engine extraction temporary storage.

PyShark Support¶

This module contains the implementation for PyShark engine support, as is used by pcapkit.foundation.extraction.Extractor.

Important

PyShark has two requirements beyond installing it, and neither is visible to an import – the package imports cleanly and then fails when used, which is why unsupported_reason() checks both up front.

Python 3.13 or older. pyshark 0.6 builds its event loop with asyncio.get_event_loop_policy().get_event_loop(). Measured in a fresh interpreter with no running loop: 3.10 and 3.11 return a loop silently, 3.12 returns one with a DeprecationWarning, and 3.14 raises RuntimeError: There is no current event loop in thread 'MainThread'. Python 3.13 was not available to measure and is expected to work, being on the deprecated-but-functional side of that change.

Wireshark’s tshark binary. pyshark shells out to it and parses nothing itself. It need not be on PATH: pyshark consults tshark_path in its config.ini first, then PATH on POSIX, both Program Files directories on Windows, and /Applications/Wireshark.app on macOS. The check delegates to pyshark’s own resolver for exactly that reason, so a correctly configured install off PATH is not refused.

class pcapkit.foundation.engines.pyshark.PyShark(extractor)[source]¶

Bases: EngineBase[PySharkPacket]

PyShark engine support.

Parameters:: extractor (Extractor) – Extractor instance.

__engine_name__ = 'PyShark'¶: Engine name.

__engine_module__ = 'pyshark'¶: Engine module name.

PYTHON_CEILING = (3, 14)¶: First Python version PyShark does not work on, as a (major, minor) pair. Released 0.6 builds its event loop with asyncio.get_event_loop_policy().get_event_loop(), and Python 3.14 made asyncio.get_event_loop() raise RuntimeError when no current event loop exists instead of quietly creating one.

classmethod unsupported_reason()[source]¶

Why this engine cannot run here, or None when it can.

Consulted by pcapkit.foundation.extraction.Extractor.run() before the import test, because neither of the two things that stop this engine is visible to an import. PyShark imports perfectly well and then fails when it is used, which without this hook escapes from run() as a hard error rather than degrading to the default engine with a warning.

The interpreter. pyshark 0.6 does asyncio.get_event_loop_policy().get_event_loop() at pyshark/capture/capture.py:183, in a fresh interpreter with no running loop. Measured on four interpreters: 3.10 and 3.11 return a loop silently, 3.12 returns one with a DeprecationWarning, and 3.14 raises RuntimeError: There is no current event loop in thread 'MainThread'. Hence PYTHON_CEILING is (3, 14). Python 3.13 was not available on the machine this was measured on; it is expected to work, since it is on the deprecated-but-functional side of that progression, and that expectation is the one thing here that is inferred rather than observed.

The tshark binary. pyshark is a wrapper around Wireshark’s command-line tool and does no parsing itself, so it is useless without it. The check delegates to pyshark’s own get_process_path() rather than calling shutil.which(), because the two are not equivalent and which would refuse setups that work: pyshark looks at tshark_path in its config.ini first, and then at PATH on POSIX, at both Program Files directories on Windows, and at /Applications/Wireshark.app on macOS. Asking pyshark gets all of that for free and cannot disagree with what pyshark will do a moment later.

Note

Deliberately not cached, and the cost was measured rather than assumed: the failing path – which is the expensive one, since it exhausts every candidate – takes about 290 microseconds with 39 PATH entries, against about 120 for a bare shutil.which(). This runs once per Extractor, not once per frame, so it is far below the cost of opening the capture. Caching would trade that for an answer about the environment that cannot change within the process – so installing Wireshark, or fixing PATH, would not take effect until restart. The Windows path does more work than the POSIX one (two Program Files directories, and shutil.which() there would multiply by PATHEXT), but it is still a bounded handful of os.stat() calls.

Return type:: str | None
Returns:: A short phrase naming the limitation, or None.

run()[source]¶

Call pyshark.FileCapture to extract PCAP files.

This method assigns self._expkg as pyshark and self._extmp as an iterator from pyshark.FileCapture.

Warns:

AttributeWarning – Warns under following circumstances:

if self.extractor._exlyr and/or self.extractor._exptl is provided as the PyShark engine currently does not support such operations.
if reassembly is enabled, as the PyShark engine currently does not support such operation.
if self.extractor._exctx is provided, as the PyShark engine does not parse with pcapkit’s own protocol implementations.

read_frame()[source]¶

Read frames with PyShark engine.

Return type:: Packet
Returns:: Parsed frame instance.

See also

Please refer to PCAP.read_frame for more operational information.

close()[source]¶

Close engine.

This method is to be used for closing the engine instance. It is to close the engine instance after the extraction process is finished.

_expkg: pyshark¶: Engine extraction package.

_extmp: FileCapture¶: Engine extraction temporary storage.

PyPCAP Support¶

This module contains the implementation for PyPCAP engine support, as is used by pcapkit.foundation.extraction.Extractor.

Important

PyPCAP publishes no wheels, so installing it compiles a C extension and needs both the libpcap(3) headers and its shared library on the system – a header alone is not enough. It is therefore not part of the all extra, and is installed on its own once libpcap is available:

pip install pypcapkit[PyPCAP]

On Python 3.12 and newer it cannot be installed at all, which is why the extra carries a python_version < '3.12' marker. PyPCAP 1.3.0 ships a pcap.c pre-generated by Cython 0.29.32 and never runs Cython at build time, and that generated C does not compile against the 3.12+ C API. Measured on four interpreters with libpcap present and found:

Python	`pip install pypcap`
3.10	builds, imports
3.11	builds, imports
3.12	fails – `ob_digit`, `curexc_traceback`
3.14	fails – those, plus `ma_version_tag` and the `_PyLong_AsByteArray` arity

Upstream is unmaintained – one doc-only commit since 1.3.0, and its Python 3.12 issue (pynetwork/pypcap#116) has been open and uncommented since May 2024 – so the cap is not expected to lift. Use PCAP_CT on 3.12 and newer.

Important

pcap-ct installs the same top-level pcap module as PyPCAP. This engine therefore checks which of the two it got and raises UnsupportedCall when it is pcap-ct, naming engine='pcap_ct' in the message. Running pcap-ct under the PyPCAP name would report an engine that is not the one in use, and the two have different install requirements and different version coverage to report.

Important

PyPCAP is a libpcap(3) binding aimed primarily at live capture. Offline it performs no protocol dissection: each frame is the (timestamp, bytes) pair that pcap_next_ex() produced. Reassembly and flow tracing are therefore unavailable and are disabled – with an AttributeWarning – when requested.

The engine also reads PCAP savefiles only, and only from a file on disk. PCAP-NG is rejected with a FormatError, because libpcap(3) opens such a file without complaint and then yields no frames at all; and a non-file input is rejected with an UnsupportedCall, because pcap_open_offline() opens a savefile by name.

class pcapkit.foundation.engines.pypcap.PyPCAP(extractor)[source]¶

Bases: EngineBase[RawFrame]

PyPCAP engine support.

PyPCAP is a binding over libpcap(3), primarily aimed at live capture. Offline it is a savefile reader and nothing more: iteration yields the (timestamp, bytes) pair from pcap_next_ex() and no protocol dissection is performed. Consequently this engine

returns each frame as a (timestamp, bytes) tuple rather than as a parsed packet object, and
disables both reassembly and flow tracing, warning as it does so, since neither can be derived without an IP or TCP layer to read.

It also requires the input to be a real file on disk, because pcap_open_offline() opens by name – there is no way to hand libpcap(3) an already-open Python stream.

Important

This engine is upstream PyPCAP specifically, which cannot be installed on Python 3.12 or newer: it ships no wheels and its pcap.c was pre-generated by Cython 0.29.32, which does not compile against the 3.12+ C API. pcap-ct is an independent reimplementation of the same interface that does install there, and PCAP_CT is the engine for it.

Because pcap-ct installs the same top-level pcap module, this engine checks which distribution it got and refuses the other one rather than running under the wrong name – see __init__().

Parameters:: extractor (Extractor) – Extractor instance.
Raises:: UnsupportedCall – If the installed pcap is pcap-ct rather than upstream PyPCAP.

__engine_name__ = 'PyPCAP'¶: Engine name.

__engine_module__ = 'pcap'¶: Engine module name. Note that this cannot separate the two distributions that own it – see pcapkit.foundation.engines._pcap_backend.probe() and unsupported_reason(), which is where that is done.

__engine_distribution__ = 'pypcap'¶: Distribution this engine drives.

classmethod unsupported_reason()[source]¶

Why this engine cannot run here, or None when it can.

Consulted by pcapkit.foundation.extraction.Extractor.run() before the import test, because the import test cannot answer this question: import pcap succeeds when pcap-ct is installed and upstream PyPCAP is not, so the guard is satisfied and this engine would run on a distribution it does not drive – reporting PyPCAP for work that pcap-ct did.

The condition is deliberately not a Python version ceiling, even though upstream cannot be installed on 3.12 or newer. What matters is which distribution is actually present: a version check would refuse a hypothetical upstream build that someone got working on a newer interpreter, and would say nothing useful on 3.10 and 3.11, where both distributions install happily and either could be the one in place.

Return type:: str | None
Returns:: A phrase naming the real cause, or None. An absent pcap returns None, since the import test reports that case in its own words and duplicating it would produce two warnings for one problem.

property dlink: LinkType¶: Data link layer protocol, as reported by the capture handle.

property backend: str¶

The distribution, version and location this engine is driving.

Two distributions provide pcap, so “the PyPCAP engine failed” is not a complete statement of what ran. This says which one it was, e.g. pypcap 1.3.0 (/.../pcap.cpython-310-x86_64-linux-gnu.so).

run()[source]¶

Call pcap.pcap to extract PCAP files.

This method assigns self._expkg as pcap and self._extmp as an iterator from pcap.pcap.

Warns:

AttributeWarning – Warns under following circumstances:

if self.extractor._exlyr and/or self.extractor._exptl is provided as the PyPCAP engine currently does not support such operations.
if reassembly and/or flow tracing is enabled, as the PyPCAP engine performs no protocol dissection and so cannot support either operation.

Raises:

FormatError – If the file format is not supported, i.e., not a PCAP file. PCAP-NG is rejected explicitly rather than left to libpcap(3), which opens such a file without complaint and then yields no frames at all.
UnsupportedCall – If the input is not a file on disk, as pcap_open_offline() can only open a savefile by name.

read_frame()[source]¶

Read frames with PyPCAP engine.

Return type:: tuple[float, bytes]
Returns:: The (timestamp, bytes) pair as yielded by pcap.pcap.

See also

Please refer to PCAP.read_frame for more operational information.

close()[source]¶

Close engine.

This method closes the underlying pcap.pcap handle. It is idempotent, as Extractor._cleanup and Extractor.__exit__ may both reach it, and pcap.pcap.close() is not safe to call twice.

_expkg: pcap¶: Engine extraction package.

_extmp: Iterator[RawFrame]¶: Engine extraction temporary storage.

_dlink: Enum_LinkType¶: Data link layer protocol, from the capture handle.

_closed: bool¶: Closed flag, so that the handle is not closed twice.

pcap-ct Support¶

This module contains the implementation for pcap-ct engine support, as is used by pcapkit.foundation.extraction.Extractor.

Important

pcap-ct is an independent ctypes reimplementation of the PyPCAP interface, by a different author, on top of the libpcap distribution. It is a separate engine rather than a second backend for PyPCAP: select it with engine='pcap_ct'.

Its reason for existing is coverage. Upstream PyPCAP stops at Python 3.11 (see the note under PyPCAP Support above); pcap-ct and libpcap both publish py3-none-any wheels, so installing this engine needs no compiler and no pcap.h:

pip install pypcapkit[PCAP_CT]

Verified end to end on Python 3.10.20 and 3.14.7: both read examples/captures/in.pcap through engine='pcap_ct' and return the same six frames with identical timestamps. So the two engines together cover every interpreter the project supports, and PCAP_CT covers all of it on its own:

Engine	Python
`PyPCAP` (`pypcap`)	3.10, 3.11
`PCAP_CT` (`pcap-ct`)	3.10 and newer

Warning

A system libpcap(3) is still required at run time, and this is the easiest thing to get wrong about pcap-ct. libpcap ships a vendored libpcap.so under _platform/{linux,macos,windows}/ but does not use it by default: its libpcap.cfg reads LIBPCAP = None as published, which sends the loader to ctypes.util.find_library(), so what gets mapped is the host’s libpcap.so.1.

Measured, not assumed: the same libpcap 1.11.0b29 wheel loaded /usr/lib64/libpcap.so.1.5.3 under one interpreter on this host and linuxbrew’s 1.11.0 under another, purely because their loader search paths differ. Setting LIBPCAP = tcpdump in libpcap.cfg selects the vendored copy instead, which reports libpcap 1.10.6.

With no system libpcap(3) at all, import pcap raises OSError rather than ImportError, which would escape Extractor.import_test and abort the extraction. PCAP_CT.unsupported_reason detects it and reports it as an ordinary “engine unavailable”, so the extraction falls back to the default engine with a warning naming the missing library.

Warning

Both distributions are pre-releases. pcap-ct 1.3.0b3 and libpcap 1.11.0b29 are the newest published versions, and neither project has ever published a stable release – which is also why pip install resolves them without --pre. pcap-ct further documents itself as tracking the PyPCAP 1.2.3 interface rather than 1.3.0.

Every attribute this engine touches – the pcap.pcap(name=..., promisc=...) constructor, datalink(), snaplen, iteration yielding (timestamp, bytes), and close() – is present on both and was measured to behave identically, byte for byte and timestamp for timestamp, on pcap-ct 1.3.0b3 against pypcap 1.3.0. That is a statement about the versions tested, not a guarantee from upstream. PCAP_CT is therefore not included in the all extra: a beta should be asked for by name.

Important

Being the same interface, pcap-ct has the same limits. It performs no protocol dissection: each frame is the (timestamp, bytes) pair that pcap_next_ex() produced. Reassembly and flow tracing are therefore unavailable regardless of which backend is installed, and are disabled – with an AttributeWarning – when requested; the adapters in pcapkit.toolkit.pcap_ct raise UnsupportedCall rather than fail obscurely. It also reads from a file on disk only, because pcap_open_offline() opens a savefile by name; a non-file input is rejected with an UnsupportedCall.

The timestamp is seconds since the epoch as a float. pcap-ct opens savefiles asking for nanosecond precision via pcap_open_offline_with_tstamp_precision(), so a microsecond-resolution savefile is scaled up by libpcap(3) rather than truncated; for examples/captures/in.pcap the result matches each record header’s ts_sec + ts_usec * 1e-6 exactly.

Note

PCAP-NG is rejected with a FormatError, for both engines, and the reason is that accepting it would be unpredictable rather than merely limited. libpcap(3) can read a PCAP-NG savefile, but how well depends on the version the host provides – which, per the warning above, the libpcap wheel does not pin. Both measured on examples/captures/dhcp.pcapng:

System libpcap	Result
1.11.0	correct – the same four frames and the same sub-second timestamps as `PCAPNG`
1.5.3	four frames, but nonsense sub-second timestamps (`1102274184.0000002` and the like), silently and with no error

Even on a good version, pcap_datalink() reports a single link type for the whole file, so a capture whose interfaces differ would have one interface’s link type applied to every frame. PCAPNG reads the per-interface blocks properly and does not depend on the host’s library at all, so PCAP-NG is routed there rather than read approximately – or wrongly – here.

class pcapkit.foundation.engines.pcap_ct.PCAP_CT(extractor)[source]¶

Bases: EngineBase[RawFrame]

pcap-ct engine support.

pcap-ct is a ctypes reimplementation of the PyPCAP API on top of the libpcap package, which supplies the libpcap(3) bindings (and ships a vendored copy of the library that it does not, by default, use – see the warning below). Both distributions install a top-level pcap module and expose the same interface, so this engine is a sibling of PyPCAP rather than a replacement for it – see __engine_module__ for how the two are told apart, and the engine documentation for which one to install.

It exists because upstream PyPCAP cannot be installed on a current interpreter: it ships no wheels and its pcap.c was pre-generated by Cython 0.29.32, which does not compile against the Python 3.12+ C API. pcap-ct and libpcap both publish py3-none-any wheels, so installing this engine needs no compiler and no pcap.h.

Warning

It does still need a system libpcap(3) at run time, and this is easy to get wrong: libpcap ships a vendored libpcap.so under _platform/linux,macos,windows/ but does not use it by default. Its libpcap.cfg reads LIBPCAP = None as published, which sends its loader to ctypes.util.find_library(), so the library actually mapped is the host’s libpcap.so.1. Measured on this host: the same libpcap 1.11.0b29 wheel loaded /usr/lib64/libpcap.so.1.5.3 under one interpreter and linuxbrew’s 1.11.0 under another, purely because their loader search paths differ.

Two consequences. With no system libpcap(3) at all, import pcap raises OSError – not ImportError – which unsupported_reason() exists partly to catch. And the libpcap(3) version in play is a property of the host rather than of the wheel, which is why the PCAP-NG note on run() does not rely on it. Setting LIBPCAP = tcpdump in libpcap.cfg selects the vendored copy instead – verified loading, and reporting libpcap 1.10.6.

Being the same interface, it has the same limits. Offline, pcap is a savefile reader and nothing more: iteration yields the (timestamp, bytes) pair from pcap_next_ex() and no protocol dissection is performed. Consequently this engine

returns each frame as a (timestamp, bytes) tuple rather than as a parsed packet object, and
disables both reassembly and flow tracing, warning as it does so, since neither can be derived without an IP or TCP layer to read.

It also requires the input to be a real file on disk, because pcap_open_offline() opens by name – there is no way to hand libpcap(3) an already-open Python stream.

Important

Both distributions are published as pre-releases only – pcap-ct 1.3.0b3 and libpcap 1.11.0b29 at the time of writing – and pcap-ct documents itself as tracking the PyPCAP 1.2.3 API rather than 1.3.0. Every attribute this engine touches is present and behaves identically on both (measured on pcap-ct 1.3.0b3 against pypcap 1.3.0), but that is a statement about the versions tested, not a guarantee from upstream.

Parameters:: extractor (Extractor) – Extractor instance.

__engine_name__ = 'PCAP_CT'¶: Engine name.

__engine_module__ = 'pcap._pcap'¶: Engine module name. Note that this is pcap._pcap, not pcap: pcap-ct and upstream PyPCAP both install a top-level pcap, so importing that name cannot tell which of the two is present, and Extractor.import_test decides engine availability by import alone. pcap._pcap is the pcap-ct implementation module; upstream ships pcap as a single extension module rather than a package, so the submodule import fails there and the two engines stay distinguishable.

__engine_distribution__ = 'pcap-ct'¶: Distribution this engine drives.

classmethod unsupported_reason()[source]¶

Why this engine cannot run here, or None when it can.

Consulted by pcapkit.foundation.extraction.Extractor.run() before the import test, and it answers two questions the import test cannot.

Which distribution owns :mod:`pcap`. Upstream PyPCAP owns that name just as legitimately, and import pcap._pcap – what __engine_module__ names – separates them only as long as upstream keeps shipping a single extension module. Asking probe() states the check rather than inferring it from an import that happens to fail.

Whether a system :manpage:`libpcap(3)` exists at all. This is the important one, and it is measured rather than theoretical. pcap-ct needs no compiler because it is ctypes, but at runtime it imports the libpcap distribution, whose Linux loader calls ctypes.util.find_library() and raises OSError – Cannot find libpcap.so library – when there is none. OSError is not an ImportError, so Extractor.import_test lets it through and the extraction dies rather than falling back. Reporting it here turns that into the ordinary “engine unavailable” path.

There is deliberately no Python version bound in either direction. Measured on 3.10.20 and 3.14.7: pcap-ct 1.3.0b3 with libpcap 1.11.0b29 installs and reads examples/captures/in.pcap identically on both, so this engine covers the whole range the project supports. The floor in the PCAP_CT extra’s markers exists only because pyproject.toml still advertises requires-python >= 3.6, and on an interpreter below 3.10 the distributions simply will not be installed – which the import test then reports correctly on its own.

Return type:: str | None
Returns:: A phrase naming the real cause, or None. An pcap that is merely absent returns None, since the import test reports that case in its own words.

property dlink: LinkType¶: Data link layer protocol, as reported by the capture handle.

property backend: str¶

The distribution, version and location this engine is driving.

Two distributions provide pcap, so “the pcap-ct engine failed” is not a complete statement of what ran. This says which one it was, e.g. pcap-ct 1.3.0b3 (/.../pcap/__init__.py).

__init__(extractor)[source]¶

Initialise the engine.

Warns:: EngineWarning – If both distributions that provide pcap are installed. Only one of them can win the import – measured on Python 3.10 with both present, the pcap-ct package wins and upstream’s extension module is shadowed – so the other is unreachable, and nothing else would say why engine='pypcap' had stopped working.
Raises:: UnsupportedCall – If the environment cannot support this engine, for either of the reasons unsupported_reason() describes. That hook is only consulted by Extractor.run, which degrades to the default engine with a warning; constructing the engine directly bypasses it, so this is the backstop for that path.

run()[source]¶

Call pcap.pcap to extract PCAP files.

This method assigns self._expkg as pcap, self._handle as the pcap.pcap capture handle and self._extmp as an iterator over it.

Warns:

AttributeWarning – Warns under following circumstances:

if self.extractor._exlyr and/or self.extractor._exptl is provided as the pcap-ct engine currently does not support such operations.
if reassembly and/or flow tracing is enabled, as the pcap-ct engine performs no protocol dissection and so cannot support either operation.

Raises:

FormatError – If the file format is not supported, i.e., not a PCAP file. PCAP-NG is rejected explicitly even though the vendored libpcap(3) can read one – see the note below.
UnsupportedCall – If the input is not a file on disk, as pcap_open_offline() can only open a savefile by name.

Note

The PCAP-NG rejection is a deliberate narrowing, and the reason is that the alternative is unpredictable rather than merely limited. libpcap(3) can read a PCAP-NG savefile, but how well depends on the version the host happens to provide – which, per the warning in this class’s docstring, is not something the libpcap wheel pins. Both measured on examples/captures/dhcp.pcapng:

against libpcap 1.11.0 it reads correctly, yielding the same four frames and the same sub-second timestamps as PCAPNG;
against libpcap 1.5.3 it yields the four frames but with nonsense sub-second timestamps – 1102274184.0000002 and the like – silently, with no error.

Even on a good version it cannot represent more than one interface: pcap_datalink() returns a single link type for the whole file, so a capture whose interfaces differ would have one interface’s link type applied to every frame. PCAPNG reads the per-interface blocks properly and does not depend on the host’s library at all, so PCAP-NG is routed there rather than read approximately – or wrongly – here.

read_frame()[source]¶

Read frames with pcap-ct engine.

Return type:: tuple[float, bytes]
Returns:: The (timestamp, bytes) pair as yielded by pcap.pcap.

Note

The timestamp is seconds since the epoch as a float. pcap-ct opens savefiles with pcap_open_offline_with_tstamp_precision() asking for nanosecond precision, so a microsecond-resolution savefile is scaled up by libpcap(3) rather than truncated; the resulting value matches the record header’s ts_sec + ts_usec * 1e-6 exactly for examples/captures/in.pcap.

See also

Please refer to PCAP.read_frame for more operational information.

close()[source]¶

Close engine.

This method closes the underlying pcap.pcap handle. It is idempotent, as Extractor._cleanup and Extractor.__exit__ may both reach it.

Note

pcap-ct 1.3.0b3’s own pcap.pcap.close() happens to tolerate a second call, where upstream pypcap 1.3.0 segfaults on one. The _closed flag is kept regardless: it is not this engine’s business to depend on a pre-release’s undocumented forgiveness, and the flag also covers the case of an engine that was never opened.

_expkg: pcap¶: Engine extraction package.

_handle: Handle¶: Capture handle, kept separately from the iterator it is read through so that close() does not depend on iter(handle) is handle.

_extmp: Iterator[RawFrame]¶: Engine extraction temporary storage.

_dlink: Enum_LinkType¶: Data link layer protocol, from the capture handle.

_closed: bool¶: Closed flag, so that the handle is not closed twice.

PyPCAPFile Support¶

This module contains the implementation for PyPCAPFile engine support, as is used by pcapkit.foundation.extraction.Extractor.

Important

PyPCAPFile is a pure Python savefile reader that decodes Ethernet, IPv4, TCP and UDP and nothing else. IPv6 reassembly is therefore unavailable and is disabled – with an AttributeWarning – when requested; IPv4 and TCP reassembly and TCP flow tracing remain available. PCAP-NG is rejected with a FormatError.

class pcapkit.foundation.engines.pypcapfile.PyPCAPFile(extractor)[source]¶

Bases: EngineBase[PCAPFilePacket]

PyPCAPFile engine support.

PyPCAPFile is a pure Python savefile reader. It decodes Ethernet, IPv4, TCP and UDP, and nothing else – in particular there is no IPv6 decoder and no PCAP-NG support. Consequently this engine

reads PCAP savefiles only, raising FormatError on PCAP-NG, and
disables IPv6 reassembly, warning as it does so, while leaving IPv4 and TCP reassembly and TCP flow tracing in place.

The engine stops decoding at the network layer rather than descending into the transport layer. PyPCAPFile decoders replace the payload bytes of the layer they decode, so descending further would discard the verbatim TCP segment that tcp_reassembly() needs in order to report an exact header/payload split.

Parameters:: extractor (Extractor) – Extractor instance.

__engine_name__ = 'PyPCAPFile'¶: Engine name.

__engine_module__ = 'pcapfile'¶: Engine module name.

LAYERS = 2¶: Number of layers to descend while decoding, i.e. link plus network. See the class docstring for why this stops short of the transport layer.

PYTHON_CEILING = (3, 12)¶: First Python version PyPCAPFile does not work on, as a (major, minor) pair. Released 0.12.0 imports imp from pcapfile.linklayer, and imp was removed in Python 3.12.

classmethod unsupported_reason()[source]¶

Why this engine cannot run here, or None when it can.

Consulted by pcapkit.foundation.extraction.Extractor.run() before the import test, because the import test cannot answer this question. pcapfile’s top-level package imports perfectly well on Python 3.12 and newer – it is pcapfile.linklayer that fails, and pcapfile.savefile imports it – so a guard that only tries import pcapfile is satisfied and the ModuleNotFoundError then escapes from __init__() as a hard error instead of degrading to the default engine with a warning, which is what happens when the package is simply absent.

The version is checked rather than the import attempted so that the answer does not depend on which of the package’s submodules happens to be imported first, and so it is the same answer on a machine that has never installed pcapfile at all.

Return type:: str | None
Returns:: A short phrase naming the limitation, or None.

property dlink: LinkType¶: Data link layer protocol, as reported by the savefile header.

run()[source]¶

Call pcapfile.savefile.load_savefile() to extract PCAP files.

This method assigns self._expkg as pcapfile and self._extmp as an iterator over the lazily generated savefile packets.

The savefile is loaded with layers=0, so that each frame arrives with its bytes verbatim, and read_frame() then decodes it to LAYERS depth using pcapfile.linklayer.clookup() – the very call pcapfile makes internally. Doing it this way round is what lets the link layer type be inspected (and reported on) before the first frame is decoded.

Warns:

AttributeWarning – Warns under following circumstances:

if self.extractor._exlyr and/or self.extractor._exptl is provided as the PyPCAPFile engine currently does not support such operations.
if IPv6 reassembly is enabled, as pcapfile has no IPv6 decoder.
if pcapfile has no decoder for the savefile’s link layer type, in which case frames are left undecoded.

Raises:

FormatError – If the file format is not supported, i.e., not a PCAP file. pcapfile reads libpcap savefiles only.

read_frame()[source]¶

Read frames with PyPCAPFile engine.

Return type:: pcap_packet
Returns:: Parsed frame instance.

See also

Please refer to PCAP.read_frame for more operational information.

_expkg: pcapfile¶: Engine extraction package.

_extmp: Iterator[PCAPFilePacket]¶: Engine extraction temporary storage.

_dlink: Enum_LinkType¶: Data link layer protocol, from the savefile header.

_declf: Callable[..., Any] | None¶: Link layer decoder for this savefile, or None when pcapfile has none for its link layer type.

Internal Definitions¶

class pcapkit.foundation.engines.pypcapfile._NamedStream(stream, name)[source]¶

Bases: object

Read-only proxy that gives a stream the name attribute.

pcapfile.savefile.load_savefile() dereferences input_file.name unconditionally, on the way into its trace helper. That is fine for the BufferedReader Extractor normally holds, but not for the SeekableReader it substitutes when the caller supplied a non-seekable stream – that class exposes no name, so the load would fail with AttributeError before a single byte was read.

Parameters:

stream (BinaryIO) – Underlying binary stream.
name (str) – Name to report as name.

name¶: Name of the underlying stream.

read(size=-1)[source]¶

Read from the underlying stream.

Parameters:: size (int) – Number of bytes to read; all remaining bytes if negative.
Return type:: bytes

PyPCAPFile._get_decoder(linktype)[source]¶

Return the pcapfile link layer decoder for a link layer type.

Parameters:: linktype (int) – Link layer type code, from the savefile header.
Return type:: Callable[..., Any] | None
Returns:: The decoder class, or None when pcapfile has none.
Warns:: AttributeWarning – If no decoder is available, as frames will then be left as raw bytes and no reassembly or flow tracing is possible.

PyPCAPFile._decode(packet, frnum)[source]¶

Decode a raw savefile packet down to LAYERS depth.

A new pcapfile.structs.pcap_packet is built rather than the given one mutated, so that a decoding failure leaves the original intact.

Parameters:

packet (pcap_packet) – Undecoded savefile packet, i.e. as loaded with layers=0.
frnum (int) – Frame number, for the warning message below.

Return type:

pcap_packet

Returns:

The decoded packet, or packet unchanged when it could not be decoded.

Warns:

AttributeWarning – If pcapfile could not decode the frame, e.g. because the capture is truncated. One bad frame should not abort the extraction, but it should not pass silently either.

Backend Detection¶

Two unrelated PyPI distributions install a top-level module named pcap – PyPCAP, a Cython binding shipped as a single extension module, and pcap-ct, a ctypes reimplementation shipped as a package. They therefore collide, and import pcap resolves to whichever the import system finds first. PyPCAP and PCAP_CT each have to know which one they actually got rather than assume, and this module is the one place that answers it – deliberately shared, since the two engines must agree and two copies of the detection would be two chances to disagree. It is only detection, and it imports nothing from pcapkit, so it cannot introduce an import cycle.

pcapkit.foundation.engines._pcap_backend.PYPCAP = 'pypcap'¶: Distribution name of upstream PyPCAP.

pcapkit.foundation.engines._pcap_backend.PCAP_CT = 'pcap-ct'¶: Distribution name of pcap-ct.

pcapkit.foundation.engines._pcap_backend.DISTRIBUTIONS = ('pypcap', 'pcap-ct')¶: Every distribution known to provide pcap, in a fixed order so that messages naming several of them read the same way every time.

pcapkit.foundation.engines._pcap_backend.ENGINE_NAMES = {'pcap-ct': 'pcap_ct', 'pypcap': 'pypcap'}¶: The engine= string that drives each distribution. Kept here rather than in either engine so that a message pointing the user at the other engine cannot name one that does not exist.

class pcapkit.foundation.engines._pcap_backend.Probe(*args: VT, **kwargs: VT)[source]¶

Bases: Info

What one attempt to import pcap found.

Note

This is an Info subclass, so it is a Mapping rather than a tuple: its fields are reached by name, not by position, and it cannot be unpacked as a sequence.

name: str | None¶: Distribution that provided the imported module – PYPCAP, PCAP_CT, or None when the import did not succeed.

version: str | None¶: pcap.__version__, when there was a module to read it from.

origin: str | None¶: pcap.__file__, which is what distinguishes an extension module from a package directory to a human reading a bug report.

failure: str | None¶: Why the import failed, as a short phrase, or None on success.

missing: bool¶: Whether the failure was simply “not installed”, i.e. an ImportError. This matters because Extractor.import_test already reports that case perfectly well, whereas the other kind of failure escapes it – see probe().

installed: tuple[str, ...]¶: Distributions found installed, whether or not their module was importable. More than one means the collision described in this module’s docstring.

describe()[source]¶

A one-line description fit for a warning or a bug report.

Return type:: str
Returns:: Something like pcap-ct 1.3.0b3 (/.../pcap/__init__.py), or a phrase naming the failure when there is no module to describe.

pcapkit.foundation.engines._pcap_backend.probe()[source]¶

Import pcap and report what was found.

Not cached. The cost after a successful first call is a sys.modules lookup, and a cache would make the answer depend on when it was first asked – which the tests, and anything that manipulates sys.path, would have to work around. It is idempotent instead, via _purge().

Note

The bare except Exception is deliberate and is much of the point of this function. pcap-ct imports the libpcap distribution, whose Linux loader calls ctypes.util.find_library() and raises OSError – Cannot find libpcap.so library – when no system libpcap(3) is present. OSError is not an ImportError, so Extractor.import_test does not catch it and it escapes as a hard error instead of degrading to the default engine. Catching it here is what lets an engine report it as a reason instead.

Return type:: Probe
Returns:: A Probe describing the outcome.

pcapkit.foundation.engines._pcap_backend.identify(module)[source]¶

Which distribution does an imported pcap module come from?

pcap-ct ships pcap as a package whose __init__ does from ._pcap import *, which binds the submodule as an attribute; upstream PyPCAP ships a single extension module, which has no such attribute. That is a structural difference rather than a cosmetic one, which is why it is preferred here over the alternatives:

pcap.__version__ is 1.3.0b3 against 1.3.0 today, but that is a coincidence of release timing and would stop separating them the moment pcap-ct cuts a 1.3.0 final.
pcap.ex_name looked like a pcap-ct marker and is not – measured present on upstream pypcap 1.3.0 as well.

Parameters:: module (ModuleType) – An already-imported pcap module.
Return type:: str
Returns:: PCAP_CT or PYPCAP.

pcapkit.foundation.engines._pcap_backend.installed_distributions()[source]¶

Which of DISTRIBUTIONS are installed, per package metadata.

Asked of the metadata rather than of pcap itself, because that is the only way to see the distribution the import did not resolve to – which is exactly the collision worth reporting.

Return type:: tuple[str, ...]
Returns:: The installed subset of DISTRIBUTIONS, in that order.

pcapkit.foundation.engines._pcap_backend.wrong_backend_reason(wanted, found)[source]¶

Why wanted cannot run, when some other distribution owns pcap.

Parameters:

wanted (str) – The distribution the calling engine drives – PYPCAP or PCAP_CT.
found (Probe) – The result of probe().

Return type:

str | None

Returns:

A reason naming what was found and which engine= string wants it, or None when wanted is what is installed, or when nothing is – an absent module is not this function’s business, since Extractor.import_test reports that.

pcapkit.foundation.engines._pcap_backend.collision_reason(found)[source]¶

A description of both distributions being installed at once, if they are.

Return type:: str | None
Returns:: A phrase naming every installed distribution and which one import pcap actually resolved to, or None when at most one is installed.

Internal Definitions¶

pcapkit.foundation.engines._pcap_backend._purge()[source]¶

Drop the pcap and libpcap module trees from sys.modules.

Called after a failed import so that the next probe reproduces the same failure. Without it a second attempt reports something else entirely, which was measured rather than imagined.

Both pcap-ct’s and libpcap’s package initialisers open with from .__about__ import * ; del __about__, which is not safe to re-run: the del needs a name that only gets bound as a side effect of importing the submodule fresh. When the initialiser fails part-way – as it does with no system libpcap(3), where the real error is OSError: Cannot find libpcap.so library – Python removes the package it was executing but leaves the __about__ submodule cached, so the retry reaches the del with nothing bound and dies with NameError: name '__about__' is not defined.

That message names neither the missing library nor the package it came from, and it is what the user would otherwise be shown. libpcap is purged as well as pcap because the residue is in whichever of the two got part-way: purging only pcap moved the NameError from one to the other rather than removing it.