Engine Comparison¶
Being a comprehensive packet processor costs speed: pcapkit’s own
engine is slower than every third-party engine below except PyShark, as
Test Results measures. engine= selects the engine:
|
Engine |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
PyPCAP and pcap-ct are independent distributions of the same
libpcap(3) interface, and both install a top-level pcap module:
upstream PyPCAP stops at Python 3.11, pcap-ct covers 3.10 and newer.
Install exactly one of them – with both present, pcap-ct wins the
import and the other becomes unselectable.
Every third-party engine supports less than the default one.
PyPCAP and
PCAP_CT perform no protocol
dissection at all, so they offer neither reassembly nor flow tracing, and
PyPCAPFile has no IPv6 decoder,
so IPv6 reassembly is unavailable. Each gap is announced with a warning or an
exception, except for these, which pass without a warning:
PyPCAPFileleaves TCP behind a VLAN tag, in a tunnel or behind an extension header (an IPv6 one, or AH in IPv4) out of TCP reassembly and flow tracing;DPKTleaves TCP in an IP-in-IP tunnel or behind an AH or IPv6 extension header in IPv4 out of TCP reassembly and flow tracing, and IPv4 in IPv6 or IPv6 in IPv4 out of IP reassembly;Scapyreads IPv6 only as far as Scapy dissects it, so IPv6 behind some BSD loopback headers, and TCP and IPv6 fragments behind some IPv6 extension headers, go undissected.
Engine Support tabulates them.
Every engine also answers a preflight check,
unsupported_reason(), which
the Extractor constructor consults
before importing the engine. An engine that cannot run here therefore gives one
warning naming the cause (a Python version, a missing tshark, a
missing libpcap(3), the wrong pcap distribution) and falls back
to the default engine, rather than raising from inside the third-party
package. The engine selection graph draws each
fallback in order.
Note
The figures under Test Environment, Test Results and
Test Results (Relative) are copied here by hand from a benchmark report;
nothing regenerates them on this page. The Engine Benchmark workflow,
.github/workflows/benchmark.yml, measures every engine on every
interpreter it is tested on, weekly and on dispatch, and posts its
environment and ratio tables to the benchmark: weekly engine benchmark
report thread in the CI reports Discussions category (see
CI Reports (#2030)); the per-version table is in the run’s job summary and its
benchmark-out artifact. examples/benchmark/run.sh produces the same
tables on any machine with Docker, and examples/benchmark/README.md
explains what each one measures.
Engine Support by Python Version¶
Which engines can run, by interpreter, verified by installing each engine and extracting a capture on 3.10 through 3.15.
Engine |
3.10 |
3.11 |
3.12 |
3.13 |
3.14 |
3.15 |
|---|---|---|---|---|---|---|
|
yes |
yes |
yes |
yes |
yes |
yes |
|
yes |
yes |
yes |
yes |
yes |
yes |
|
yes |
yes |
yes |
yes |
yes |
yes |
|
yes |
yes |
yes |
yes |
yes |
yes |
|
yes |
yes |
no |
no |
no |
no |
|
yes |
yes |
no |
no |
no |
no |
|
yes |
yes |
yes |
yes |
no |
no |
pypcap and pypcapfile stop at 3.11, and pyshark at 3.13, for the
reasons under Engine Prerequisites. Python 3.11 is the last version on which
every engine can run – though pypcap and pcap_ct are mutually
exclusive there too, so no single environment has every engine at once.
Engine Prerequisites¶
Some engines need more than a pip install. The
unsupported_reason()
preflight above enforces each constraint below in code.
PySharkNone of its requirements is visible to an import: the package imports cleanly and then fails when used.
Drives Wireshark’s tshark binary, which need not be on
PATH:pysharklooks attshark_pathin itsconfig.inifirst, thenPATHon POSIX, both Program Files directories on Windows, and/Applications/Wireshark.appon macOS. Install Wireshark (or just tshark) from your platform’s package manager.Requires Python 3.13 or older.
pyshark0.6 builds its event loop withasyncio.get_event_loop_policy().get_event_loop(), and from Python 3.14asyncio.get_event_loop()raisesRuntimeErrorwhen there is no current event loop instead of creating one. Measured: a loop is returned silently on 3.10 and 3.11, with aDeprecationWarningon 3.12, and refused on 3.14;pysharkextracted a capture on 3.13.
PyPCAPPyPCAP ships no wheels – only an sdist – so pip compiles it, and the build needs both libpcap’s headers (
pcap.h) and its shared or static library:# Debian/Ubuntu sudo apt-get install libpcap-dev # RHEL/Fedora/Amazon Linux sudo dnf install libpcap-devel # macOS brew install libpcap
Caveats, each an upstream packaging problem:
pypcap1.3.0 ships a pre-generatedpcap.cproduced by Cython 0.29.x, which does not compile against the Python 3.12+ C API, so on 3.12 or newer the build fails whatever else is present. Use Python 3.11 or older for this engine, or regeneratepcap.cwith Cython 3 yourself.Its
setup.pydoes not consultCFLAGS/LDFLAGSor pkg-config. It searches a fixed list of prefixes –/usr,sys.prefix,/opt/libpcap*,../libpcap*,../wpdpack*and the macOS SDKs – so a libpcap installed anywhere else, notably Homebrew’s keg-only prefix on Apple Silicon (/opt/homebrew/opt/libpcap), is not found. Install it intosys.prefixor/opt/libpcapinstead.
PCAP_CTThe way to drive the same libpcap interface on Python 3.12 and newer, where
pypcapcannot be built. Nothing to compile and nopcap.hneeded: pcap-ct is actypesreimplementation, and both it and itslibpcapdependency shippy3-none-anywheels. Verified reading a capture on Python 3.10 and 3.14.Caveats:
A system libpcap(3) is still required at run time. The
libpcapdistribution ships a vendoredlibpcap.soand, as published, does not use it: itslibpcap.cfgsaysLIBPCAP = None, which sends its loader toctypes.util.find_library(). So the library actually loaded is the host’slibpcap.so.1, and with none presentimport pcapraisesOSErrorrather thanImportError. SetLIBPCAP = tcpdumpinlibpcap.cfgto use the vendored copy instead.Both distributions are pre-releases, and
pcap-ctdocuments itself as tracking thepypcap1.2.3 interface. Every attribute the engine uses was measured behaving identically topypcap1.3.0, on the versions tested.
PyPCAPFilePyPCAPFile 0.12.0 imports the
impmodule, which was removed in Python 3.12, sopcapfile.savefile– the module needed to read a capture – cannot be imported on 3.12 or newer. Upstreammasterhas fixed this, but no release carries the fix, so this engine requires Python 3.11 or older.
Note
pcapkit itself and its default, dpkt and scapy engines
need nothing beyond their pip install – dpkt 1.9.8 and scapy
2.8.0 were both measured reading a capture on Python 3.14. The scapy
engine needs scapy 2.6.0 or newer, which the Scapy extra requires:
2.5.0 can drop a PCAP-NG interface’s timestamp resolution, reading
nanoseconds as microseconds (#1572). pcapkit does not check
the version at run time, so upgrade a scapy installed some other way.
Test Environment¶
These figures come from one run of the Engine Benchmark workflow on a GitHub-hosted
runner, measured natively rather than in the examples/benchmark/run.sh
container images, so the interpreters’ patch releases, tshark and
libpcap are the runner’s own, as recorded below.
Run |
Engine Benchmark run 40062653380, on
GitHub’s hosted |
Image |
|
|
|
Python |
CPython 3.10.22, 3.11.17, 3.12.15, 3.13.16, 3.14.8, 3.15.1 |
Architecture |
|
Capture |
|
Packets per extraction |
6 |
Iterations |
1000 timed extractions per engine per pass, the first discarded as a warm-up |
Passes |
3 over the whole engine set, in 8 environments: |
tshark |
|
libpcap |
|
Resolved package versions:
Package |
3.10-pcap_ct |
3.10-pypcap |
3.11-pcap_ct |
3.11-pypcap |
3.12-pcap_ct |
3.13-pcap_ct |
3.14-pcap_ct |
3.15-pcap_ct |
|---|---|---|---|---|---|---|---|---|
|
1.9.8 |
1.9.8 |
1.9.8 |
1.9.8 |
1.9.8 |
1.9.8 |
1.9.8 |
1.9.8 |
|
1.11.0b29 |
absent |
1.11.0b29 |
absent |
1.11.0b29 |
1.11.0b29 |
1.11.0b29 |
1.11.0b29 |
|
6.1.3 |
6.1.3 |
6.1.3 |
6.1.3 |
6.1.3 |
6.1.3 |
6.1.3 |
6.1.3 |
|
1.3.0b3 |
absent |
1.3.0b3 |
absent |
1.3.0b3 |
1.3.0b3 |
1.3.0b3 |
1.3.0b3 |
|
absent |
1.3.0 |
absent |
1.3.0 |
absent |
absent |
absent |
absent |
|
0.12.0 |
0.12.0 |
0.12.0 |
0.12.0 |
0.12.0 |
0.12.0 |
0.12.0 |
0.12.0 |
|
1.5.0b14 |
1.5.0b14 |
1.5.0b14 |
1.5.0b14 |
1.5.0b14 |
1.5.0b14 |
1.5.0b14 |
1.5.0b14 |
|
0.6 |
0.6 |
0.6 |
0.6 |
0.6 |
0.6 |
0.6 |
0.6 |
|
2.8.0 |
2.8.0 |
2.8.0 |
2.8.0 |
2.8.0 |
2.8.0 |
2.8.0 |
2.8.0 |
Test Results¶
Measured with examples/benchmark/native.sh, which the Engine Benchmark
workflow runs: 1000 timed extractions of examples/captures/in.pcap per
engine and Python version.
The first extraction is discarded as a warm-up. Values are milliseconds per
packet, the median of 3 passes.
All columns come from one run on one machine, differing in the interpreter and nothing else, so they may be compared with each other. They may not be compared with figures from another machine: absolute times are a property of the host as much as of the engine.
Engine |
3.10 |
3.11 |
3.12 |
3.13 |
3.14 |
3.15 |
|---|---|---|---|---|---|---|
|
0.0375 |
0.0262 |
– |
– |
– |
– |
|
0.0479 |
0.0342 |
– |
– |
– |
– |
|
0.0667 |
0.0489 |
0.0564 |
0.0565 |
0.0613 |
0.0534 |
|
0.0796 |
0.0599 |
0.0607 |
0.0616 |
0.0631 |
0.0858 |
|
0.2399 |
0.2028 |
0.2085 |
0.2021 |
0.2083 |
0.2208 |
|
0.6962 |
0.5709 |
0.5837 |
0.5660 |
0.6025 |
0.6128 |
|
17.0812 |
17.6570 |
17.2537 |
17.1367 |
– |
– |
The unavailable cells were attempted. They are not zeroes and must not be compared with a measured row.
Test Results (Relative)¶
Times relative to pcapkit, which is 1 by definition; lower is
faster. Each ratio is taken against the baseline measured in the same
environment on the same pass, so the machine cancels out. “Observed range” is
the smallest and largest ratio actually seen across the passes.
Ratios are pooled across every environment in the run, which means across
every Python version measured as well as both pcap providers. The
observed range therefore includes any disagreement between interpreters
about an engine; the per-version table is where that is broken out.
Engine |
Relative time |
Observed range |
Samples |
|---|---|---|---|
|
0.0496 |
0.0456-0.0542 |
12 |
|
0.0650 |
0.0600-0.0702 |
6 |
|
0.0967 |
0.0852-0.103 |
18 |
|
0.107 |
0.103-0.141 |
24 |
|
0.353 |
0.340-0.369 |
24 |
|
1 |
baseline |
24 |
|
29.7 |
23.4-31.5 |
18 |
Measured, but on less evidence than the rows beside it. What was collected is real; what was lost is named here rather than left to be inferred from the pass count: 3 individual extraction(s) failed and were discarded.
The single absolute figure, for converting the ratios back: pcapkit
itself ran at 0.5917 ms per packet on this run
(0.566-0.715 across 24 passes). Absolute times are a property of the
machine as much as of the engine, which is why every other figure here is a
ratio.
Run-to-run spread: the median row varied by 17% of its ratio across the passes and the widest by 35%. A gap between two rows that is smaller than that is noise, not a result.