Engine Support

pcapkit.foundation.engines is a collection of engines support for pcapkit, including but not limited to the built-in PCAP and PCAP-NG file support, Scapy, PyShark, DPKT, PyPCAP, pcap-ct and PyPCAPFile 3rd party engine support.

See also

For more information on customisation and extension, please refer to Customisation & Extensions.

• Base Engine
  • Internal Definitions
  • Type Variables
• Built-in Engines
  • PCAP Support
  • PCAP-NG Support
• 3rd-Party Engines
  • Scapy Support
  • DPKT Support
  • PyShark Support
  • PyPCAP Support
  • pcap-ct Support
  • PyPCAPFile Support
  • Backend Detection

All engines are implemented as EngineBase subclasses, which are responsible for parsing the input files and extracting the network packets for further processing. Below is a brief diagram of the class hierarchy of pcapkit.foundation.engines:

        flowchart LR
    A{{EngineMeta}} -.->|metaclass| B(EngineBase)

    subgraph built-in [Built-in Engines]
        direction LR

        PCAP
        PCAPNG
    end
    B --> built-in

    subgraph third-party [3rd Party Engines]
        direction LR

        Scapy
        DPKT
        PyShark
        PyPCAP
        PCAP_CT
        PyPCAPFile
    end
    B --> third-party

    B --> C(Engine)
    C --> D([user customisation ...])

    click A "engine.html#pcapkit.foundation.engines.engine.EngineMeta"
    click B "engine.html#pcapkit.foundation.engines.engine.EngineBase"
    click C "engine.html#pcapkit.foundation.engines.engine.Engine"
    click D "../../../ext.html#extractor-engines"

    click PCAP "builtin.html#pcapkit.foundation.engines.pcap.PCAP"
    click PCAPNG "builtin.html#pcapkit.foundation.engines.pcapng.PCAPNG"

    click Scapy "3rdparty.html#pcapkit.foundation.engines.scapy.Scapy"
    click DPKT "3rdparty.html#pcapkit.foundation.engines.dpkt.DPKT"
    click PyShark "3rdparty.html#pcapkit.foundation.engines.pyshark.PyShark"
    click PyPCAP "3rdparty.html#pcapkit.foundation.engines.pypcap.PyPCAP"
    click PCAP_CT "3rdparty.html#pcapkit.foundation.engines.pcap_ct.PCAP_CT"
    click PyPCAPFile "3rdparty.html#pcapkit.foundation.engines.pypcapfile.PyPCAPFile"
    

Not every engine can do everything Extractor offers, and the ones that cannot say so rather than quietly doing less, unless their row says otherwise:

Engine	Gap, and how it is surfaced
PyShark	no reassembly – disabled with an AttributeWarning
PyPCAP	no protocol dissection at all, hence no reassembly and no flow tracing – both disabled with an AttributeWarning; PCAP savefiles on disk only, otherwise FormatError / UnsupportedCall
PCAP_CT	the same gaps as PyPCAP above – it reads the same interface, so no dissection, no reassembly and no flow tracing, and PCAP savefiles on disk only
PyPCAPFile	no IPv6 decoder, hence no IPv6 reassembly – disabled with an AttributeWarning, and ipv6_reassembly() raises; TCP over IPv6 straight over Ethernet is left out of TCP reassembly and flow tracing, with an AttributeWarning once per capture, and TCP behind a VLAN tag, in a tunnel or behind an extension header (an IPv6 one, or AH in IPv4) without one; PCAP savefiles only, otherwise FormatError
DPKT	TCP in an IP-in-IP tunnel, or behind an AH or IPv6 extension header carried in IPv4, is left out of TCP reassembly and flow tracing, and IPv4 carried in IPv6, or IPv6 in IPv4, out of IP reassembly, all without a warning; a frame DPKT cannot decode is kept raw, with a DPKTWarning, and is neither reassembled nor traced
Scapy	reads IPv6 only as far as Scapy dissects it, without a warning: behind a BSD loopback header only at the family Scapy binds it to on the host it runs on, and no TCP or Fragment header behind an AH, Mobility, HIP or Shim6 header
DPKT, Scapy, PyShark, PyPCAP, PCAP_CT and PyPCAPFile	no PCAP output, since each frame reaches the writer as a plain dict – json is written instead, with a FormatWarning
PCAPNG	no PCAP output for a PCAP-NG input, which declares its link type and timestamp resolution per interface – FormatError

Availability

A third-party engine is usable only where its backing package is, and three are harder to obtain than a plain pip install suggests. A missing package is not fatal – pcapkit emits an EngineWarning and falls back to its own parser – but the extraction then does not use the engine requested.

Engine	What it needs beyond pip install
PCAP, PCAPNG	nothing – built in
DPKT, Scapy	nothing – both are pure Python, and neither overrides unsupported_reason()
PyShark	Wireshark’s tshark binary – on PATH, or wherever pyshark’s config.ini points – and Python 3.13 or older: pyshark 0.6 builds its event loop with asyncio.get_event_loop(), which raises from 3.14
PyPCAP	libpcap(3) headers and library, a C compiler, and Python 3.11 or older – pypcap 1.3.0 publishes no wheel and its pre-generated pcap.c does not compile on 3.12+
PCAP_CT	a system libpcap.so.1 at run time – nothing to build, since pcap-ct and libpcap ship pure-Python wheels, but see the warning below: the vendored library those wheels carry is not what gets loaded by default
PyPCAPFile	Python 3.11 or older – pypcapfile 0.12.0 imports the imp module, removed in Python 3.12

A constraint that an import cannot reveal is enforced by overriding unsupported_reason(), which Extractor.run consults before the import test, so asking for an engine that cannot run here produces one warning naming the actual cause and a fall back to the built-in parser. PyShark checks its Python ceiling and for tshark, PyPCAPFile its Python ceiling, PCAP_CT for a system libpcap, and PyPCAP and PCAP_CT which distribution owns pcap. PyPCAP deliberately has no Python ceiling check, even though upstream cannot be installed on 3.12 or newer: what matters is which distribution is present, and a version check would refuse a working build.

See also

PyPCAPKit - Comprehensive Network Packet Analysis Library covers the installation prerequisites in full, including how pypcap’s setup.py looks for pcap.h and why a Homebrew libpcap is not always found.

Choosing Between PyPCAP and PCAP_CT

PyPCAP and PCAP_CT read the same libpcap(3) interface from two independent distributions, both of which install a top-level pcap module. They are separate engines because they are separate projects, with different authors, different install requirements and different interpreter coverage:

Engine (engine=)	Distribution	Python	What it needs
PyPCAP ('pypcap')	PyPCAP 1.3.0 (stable)	3.10, 3.11 only	to build: a C compiler, pcap.h and a system libpcap – no wheels are published
PCAP_CT ('pcap_ct')	pcap-ct 1.3.0b3 + libpcap 1.11.0b29 (beta)	3.10 and newer	to install: nothing – py3-none-any wheels. At run time: a system libpcap.so.1 all the same – see the warning below

Neither engine dissects anything, so the capability gaps in the table above are identical for both and are a property of libpcap(3), not of the distribution: reassembly and flow tracing are unavailable whichever one is installed. Pick on availability, not on features.

Warning

PCAP_CT needs no toolchain to install, but it does need a system libpcap(3) to run. libpcap ships a vendored libpcap.so and, as published, does not use it: its libpcap.cfg says LIBPCAP = None, which sends the loader to ctypes.util.find_library(). With no system library at all import pcap raises OSError rather than ImportError; PCAP_CT.unsupported_reason catches that and turns it into an ordinary fall back to the default engine, instead of the hard error it would otherwise be.

The two distributions collide, so install exactly one. Both own the top-level pcap module, and pip will happily install both – on Python 3.10, the pcap-ct package then wins the import and upstream’s extension module is shadowed and unreachable. Each engine therefore detects which distribution it actually got, via pcapkit.foundation.engines._pcap_backend.probe(), and:

• reports it – PyPCAP.backend and PCAP_CT.backend name the distribution, version and file actually in use, so a bug report about “the pypcap engine” says which one ran;

• declines to run on the other one, through unsupported_reason(), with a message naming the engine= string that does want it; and

• warns with an EngineWarning when it finds both installed, since that state makes one of the two engines permanently unselectable and nothing else would explain why.