Engine Support

pcapkit.foundation.engines is a collection of engines support for pcapkit, including but not limited to the built-in PCAP and PCAP-NG file support, Scapy, PyShark, DPKT, PyPCAP, pcap-ct and PyPCAPFile 3rd party engine support.

See also

For more information on customisation and extension, please refer to Customisation & Extensions.

• Base Engine
  • Internal Definitions
  • Type Variables
• Built-in Engines
  • PCAP Support
  • PCAP-NG Support
• 3rd-Party Engines
  • Scapy Support
  • DPKT Support
  • PyShark Support
  • PyPCAP Support
  • pcap-ct Support
  • PyPCAPFile Support
  • Backend Detection

All engines are implemented as EngineBase subclasses, which are responsible for parsing the input files and extracting the network packets for further processing. Below is a brief diagram of the class hierarchy of pcapkit.foundation.engines:

        flowchart LR
    A{{EngineMeta}} -.->|metaclass| B(EngineBase)

    subgraph built-in [Built-in Engines]
        %% direction TD

        PCAP
        PCAPNG
    end
    B --> built-in

    subgraph third-party [3rd Party Engines]
        %% direction TD

        Scapy
        DPKT
        PyShark
        PyPCAP
        PCAP_CT
        PyPCAPFile
    end
    B --> third-party

    B --> C(Engine)
    C --> D([user customisation ...])

    click A "/pcapkit/foundation/engines/engine.html#pcapkit.foundation.engines.engine.EngineMeta"
    click B "/pcapkit/foundation/engines/engine.html#pcapkit.foundation.engines.engine.EngineBase"
    click C "/pcapkit/foundation/engines/engine.html#pcapkit.foundation.engines.engine.Engine"
    click D "/ext.html#extractor-engines"

    click PCAP "/pcapkit/foundation/engines/builtin.html#pcapkit.foundation.engines.pcap.PCAP"
    click PCAPNG "/pcapkit/foundation/engines/builtin.html#pcapkit.foundation.engines.pcapng.PCAPNG"

    click Scapy "/pcapkit/foundation/engines/3rdparty.html#pcapkit.foundation.engines.scapy.Scapy"
    click DPKT "/pcapkit/foundation/engines/3rdparty.html#pcapkit.foundation.engines.dpkt.DPKT"
    click PyShark "/pcapkit/foundation/engines/3rdparty.html#pcapkit.foundation.engines.pyshark.PyShark"
    click PyPCAP "/pcapkit/foundation/engines/3rdparty.html#pcapkit.foundation.engines.pypcap.PyPCAP"
    click PCAP_CT "/pcapkit/foundation/engines/3rdparty.html#pcapkit.foundation.engines.pcap_ct.PCAP_CT"
    click PyPCAPFile "/pcapkit/foundation/engines/3rdparty.html#pcapkit.foundation.engines.pypcapfile.PyPCAPFile"
    

Not every engine can do everything Extractor offers, and the ones that cannot say so rather than quietly doing less:

Engine	Gap, and how it is surfaced
PyShark	no reassembly – disabled with an AttributeWarning
PyPCAP	no protocol dissection at all, hence no reassembly and no flow tracing – both disabled with an AttributeWarning; PCAP savefiles on disk only, otherwise FormatError / UnsupportedCall
PCAP_CT	the same gaps as PyPCAP above – it reads the same interface, so no dissection, no reassembly and no flow tracing, and PCAP savefiles on disk only
PyPCAPFile	no IPv6 decoder, hence no IPv6 reassembly – disabled with an AttributeWarning, and ipv6_reassembly() raises; PCAP savefiles only, otherwise FormatError

Availability

A third-party engine is only usable where its backing package is, and three of them are harder to obtain than a plain pip install suggests. Asking for an engine whose package is missing is not fatal – pcapkit emits an EngineWarning and falls back to its own parser – but the extraction then has nothing to do with the engine requested, so it is worth knowing in advance.

Engine	What it needs beyond pip install
PCAP, PCAPNG	nothing – built in
DPKT, Scapy	nothing – both ship pure-Python wheels, and both were measured reading a capture on Python 3.14, so neither overrides unsupported_reason()
PyShark	Wireshark’s tshark binary – on PATH, or wherever pyshark’s config.ini points – and Python 3.13 or older: pyshark 0.6 builds its event loop with asyncio.get_event_loop(), which raises from 3.14
PyPCAP	libpcap(3) headers and library, a C compiler, and Python 3.11 or older – pypcap 1.3.0 publishes no wheel and its pre-generated pcap.c does not compile on 3.12+
PCAP_CT	a system libpcap.so.1 at run time – nothing to build, since pcap-ct and libpcap ship pure-Python wheels, but see the warning below: the vendored library those wheels carry is not what gets loaded by default
PyPCAPFile	Python 3.11 or older – pypcapfile 0.12.0 imports the imp module, removed in Python 3.12

Every one of these constraints is also enforced in code rather than only documented: each engine overrides unsupported_reason(), which Extractor.run consults before the import test, so asking for an engine that cannot run here produces one warning naming the actual cause and a clean fall back to the built-in parser.

See also

PyPCAPKit - Comprehensive Network Packet Analysis Library covers the installation prerequisites in full, including how pypcap’s setup.py looks for pcap.h and why a Homebrew libpcap is not always found.

Two engines, one interface: choosing between PyPCAP and PCAP_CT

PyPCAP and PCAP_CT read the same libpcap(3) interface from two independent distributions, both of which install a top-level pcap module. They are separate engines because they are separate projects, with different authors, different install requirements and different interpreter coverage:

Engine (engine=)	Distribution	Python	What it needs
PyPCAP ('pypcap')	PyPCAP 1.3.0 (stable)	3.10, 3.11 only	to build: a C compiler, pcap.h and a system libpcap – no wheels are published
PCAP_CT ('pcap_ct')	pcap-ct 1.3.0b3 + libpcap 1.11.0b29 (beta)	3.10 and newer	to install: nothing – py3-none-any wheels. At run time: a system libpcap.so.1 all the same – see the warning below

Neither engine dissects anything, so the capability gaps in the table above are identical for both and are a property of libpcap(3), not of the distribution: reassembly and flow tracing are unavailable whichever one is installed. Pick on availability, not on features.

Warning

PCAP_CT needs no toolchain to install, but it does need a system libpcap(3) to run. libpcap ships a vendored libpcap.so and, as published, does not use it: its libpcap.cfg says LIBPCAP = None, which sends the loader to ctypes.util.find_library(). With no system library at all import pcap raises OSError rather than ImportError; PCAP_CT.unsupported_reason catches that and turns it into an ordinary fall back to the default engine, instead of the hard error it would otherwise be.

The two distributions collide, so install exactly one. Both own the top-level pcap module, and pip will happily install both – measured on Python 3.10, the pcap-ct package then wins the import and upstream’s extension module is shadowed and unreachable. Each engine therefore detects which distribution it actually got, via pcapkit.foundation.engines._pcap_backend.probe(), and:

• reports it – PyPCAP.backend and PCAP_CT.backend name the distribution, version and file actually in use, so a bug report about “the pypcap engine” says which one ran;

• declines to run on the other one, through unsupported_reason(), with a message naming the engine= string that does want it; and

• warns with an EngineWarning when it finds both installed, since that state makes one of the two engines permanently unselectable and nothing else would explain why.