TCP Datagram Reassembly

pcapkit.foundation.reassembly.tcp contains TCP only, which reconstructs fragmented TCP packets back to origin.

class pcapkit.foundation.reassembly.tcp.TCP[_PT: Info, _DT: Info, _IT: tuple, _BT: Info](*, strict=True, store=True, timeout=None)[source]

Bases: ReassemblyBase[Packet, Datagram, tuple[_AT, int, _AT, int], Buffer]

Reassembly for TCP payload.

Parameters:

  • strict (bool) – if True, report a datagram that is not completely reassembled as the tuple of its received runs; otherwise as one contiguous payload, its holes zero-filled

  • store (bool) – if store reassembled datagram in memory, i.e., self._dtgram (if not, datagram will be discarded after callback)

  • timeout (float | None) – reassembly timeout in seconds, on the capture’s own clock; None selects __timeout__, which for TCP disables expiry

Example

>>> from pcapkit.foundation.reassembly.tcp import TCP
# Initialise instance:
>>> tcp_reassembly = TCP()
# Call reassembly:
>>> tcp_reassembly(packet_dict)
# Fetch result:
>>> result = tcp_reassembly.datagram

Note

There are two coordinate systems in play here, and keeping them apart matters. Sequence numbers – in the hole descriptor list of RFC 815, and in each payload buffer’s own gap and conflict lists – are kept unwrapped, inclusive of both bounds: every arriving sequence number is read modulo 2 ** 32 relative to the data already buffered under its buffer ID, per RFC 9293 Section 3.4, and placed on one unbounded integer line. A connection that crosses 2 ** 32 therefore continues at 2 ** 32 rather than back at zero. A payload buffer, on the other hand, is indexed from zero, such that buffer.raw[n] holds the octet with sequence number buffer.isn + n. submit() converts between the two, subtracting that buffer’s initial sequence number from each gap bound, and reports conflict ranges reduced back to modulo 2 ** 32.

The hole descriptor list is held once per buffer ID, across every acknowledgement number’s payload buffer, so it describes the stream rather than any one payload buffer: it decides when a stream whose FIN has been seen is whole and may be submitted. Whether each payload buffer is complete is a question about that buffer alone, answered from its own gap list.

Warning

Limitation. Without the handshake the start of a stream is unknown, so a FIN is taken to complete the stream once every octet from the lowest one seen so far up to the FIN has arrived. Data that arrives later but lies below everything seen before that FIN therefore opens a buffer of its own, and the first buffer has already been reported COMPLETE although the stream was not: [seq 1000 A*5, FIN 1005, seq 990 B*10] gives B*10 and A*5 as two datagrams. This applies only to a buffer begun without a SYN: once the SYN is captured, the stream’s start is the SYN’s own sequence number, and a FIN completes the stream only when every octet from there up to the FIN has arrived.

Changed in version 1.5.0: Of two overlapping segments that disagree, the octets received first are kept, per RFC 9293 Section 3.10, and the contested range is recorded in Datagram.conflict; 1.4.x kept the later octets without a record (#443).

Removed in version 1.6.0: __window__, now TCP_GAP_LIMIT (#2546).

reassembly(info)[source]

Reassembly procedure.

A segment is merged into the payload buffer it is read against by zero-filling the gap between them, measured on whichever side the segment lands: from the buffer’s end to the segment’s start, or from the segment’s end to the buffer’s start. A gap wider than TCP_GAP_LIMIT is not padded to; the buffer held so far is submitted and a fresh one opened for the segment instead. What that means depends on whether the buffer’s SYN was captured:

  • No SYN – the segment is taken to start a new stream on a reused 4-tuple, and the two buffers are reported independently.

  • SYN captured – the SYN proves both are the same stream with the range in between lost, so that range is recorded in the gap of the payload buffer the segment was read against and of the new one, and both are reported PARTIAL (in strict mode, as a single run each). The new buffer keeps the SYN’s header and counts as having seen it.

Parameters:

info (Packet) – info dict of packets to be reassembled

submit(buf, *, bufid, timeout=False)[source]

Submit reassembled payload.

Parameters:

  • buf (Buffer) – buffer dict of reassembled packets

  • bufid (tuple[_AT, int, _AT, int]) – buffer identifier

  • timeout (bool) – whether this buffer is being submitted because expire() abandoned it under the reassembly timeout

Return type:

list[Datagram]

Returns:

Reassembled packets. Where the payload is one contiguous run, its packet is the parse tcp_datagram() runs, by this reassembly’s own protocol, deferred to its first read.

Important

The (source, destination) pair handed to Deferred is a pair of TCP members rather than of the bare port numbers BufferID holds (#2417), so a deferred analysis dispatches on the named service rather than on a number. The resolution is Transport._make_port, so it is the one the maker path uses and cannot drift from it. See _make_ports() for why it is reached through TCP rather than through self.protocol.

For a registered port that is the very member TCP.src reads off a parsed header. For an unassigned one the two are equal but not identical, and not even equally named: the parse path builds its member through EnumField._unregistered_member, whose .name is '<unassigned>', where get builds its own, named 'unknown' – measured on port 50000. Both are real members of the same registry carrying the same AppType.port, which is what every reader of this pair goes on to use, so the distinction costs nothing here; it is recorded because “the same member the header carries” would be an overstatement of it.

static _make_ports(bufid)[source]

The flow’s (source, destination) ports, as application types.

Parameters:

bufid (tuple[_AT, int, _AT, int]) – buffer identifier, whose positions 1 and 3 are the source and destination ports.

Return type:

tuple[TCP, TCP]

Returns:

The pair submit() hands Deferred, i.e. what Transport.analyze dispatches the reassembled payload on (#2417). Typed as the per-transport registry TCP rather than as the base AppType, because that is what it really is: proto=tcp sends AppType.get through AppType._dispatch to the TCP registry, and type(...) is TCP holds of the member it answers with – measured, for a registered port and an unassigned one alike. Transport.analyze keeps the base in its own signature, since it is the one method UDP and SCTP share, and each of them would name a different registry.

Important

Every port a 16-bit header field can hold resolves to a member, so this raises for nothing that can come off the wire: AppType.get answers an assignment with its canonical service, a port inside a documented IANA span with that span’s label (reserved for 0), and one in no span at all with an unregistered unknown member – never None, and without growing the registry (#860). It rejects a port outside 0-65535 and a non-int.

A bool is rejected by Transport._make_port rather than by get, which is the reason to go through it here and not call get directly: bool is an int subclass, so AppType.get(True, proto=tcp) resolves True as port 1 and answers <TCP.tcpmux: 1 [tcp]> – measured – where _make_port raises, which is what closed #2287. Calling get straight would reopen it. None of the three can come out of a TCP header either way; what _make_port also buys is idempotence, so a bufid already holding a member passes through untouched.

Note

The resolution is reached through TCP rather than through self.protocol, which is what submit() reads the analyser itself off. That is deliberate: __protocol_type__ is an extension point, and an analyser plugged into it need only define analyze – it is not required to be a Transport subclass and so need not have _make_port at all, which reaching it through self.protocol would turn into an AttributeError. What a TCP port means belongs to the TCP protocol either way, not to whatever consumes the payload, which is also why the transport below is named outright rather than read off the analyser.

__protocol_name__: str = 'TCP'

Protocol name of current reassembly object.

__protocol_type__: Type[ProtocolBase] = <class 'pcapkit.protocols.transport.tcp.protocol.TCP'>

Protocol of current reassembly object.

__callback_fn__: list[Callable[[List[_DT]], None]]

List of callback functions upon reassembled datagram.

__timeout__: float = inf

Default reassembly timeout – disabled, unlike IPv4 and IPv6.

No specification gives TCP stream reassembly a deadline the way RFC 1122 Section 3.3.2 and RFC 8200 Section 4.5 give IP fragmentation one, and the numbers that look like candidates are not reassembly timeouts: the Maximum Segment Lifetime of RFC 9293 Section 3.4.1 bounds how long a segment may linger in the network, and the user timeout of RFC 9293 Section 3.10.8 aborts a connection whose data goes unacknowledged. Picking either as a default would silently discard buffered stream data on captures that are merely idle – a long-lived connection with a two-minute lull is ordinary, while a 60-second gap between fragments of one IP datagram is pathological.

So the mechanism is available and the default is off: pass timeout to ask for one, e.g. 2 * 120 for 2·MSL if that is the policy wanted.

Type:

float

Algorithm

See also

This algorithm is an adaptation of the algorithm described in RFC 815.

Attribute

Definition

DSN

Data Sequence Number

ACK

TCP Acknowledgement

SYN

TCP Synchronisation Flag

FIN

TCP Finish Flag

RST

TCP Reset Connection Flag

BUFID

Buffer Identifier

HDL

Hole Descriptor List

ISN

Initial Sequence Number

PSN

Payload Sequence Number

src

source IP

dst

destination IP

srcport

source TCP port

dstport

destination TCP port

 
DO {
   BUFID <- src|srcport|dst|dstport;

   /* a SYN occupies a sequence number of its own, so payload sent by
      or after it starts one octet later than the segment's DSN */
   PSN <- DSN + 1 IF (SYN is true) ELSE DSN;

   IF (SYN is true) {
      IF (buffer with BUFID is allocated) {
         flush all reassembly for this BUFID;
         submit datagram to next step;
      }
   }

   IF (no buffer with BUFID is allocated) {
      allocate reassembly resources with BUFID;
      ISN <- PSN;
      put data from fragment into data buffer with BUFID
         [from octet fragment.first to octet fragment.last];
      HDL <- [one hole from PSN + fragment.len to infinity,
              i.e. HOLE_OPEN_BOUND];
      IF (SYN is false) {
         /* the start of the stream is unknown: this hole is the one
            before the capture */
         insert one hole from -HOLE_OPEN_BOUND to PSN - 1 into HDL;
      }
   } ELSE {
      put data from fragment into data buffer with BUFID
         [from octet fragment.first to octet fragment.last];

      /* a segment with no payload fills no hole, and its "last" lies
         one below its "first", so it is not run through the algorithm */
      IF (fragment.len > 0) {
         update HDL;
      }
   }

   /* a FIN fixes the end of the stream; data sent before it may still
      arrive after it */
   IF (FIN is true) {
      cut every hole in HDL back to end below the FIN;
   }

   IF (RST is true or HDL holds no hole but the one before the capture) {
      submit datagram to next step;
      free all reassembly resources for this BUFID;
      BREAK.
   }
} give up until (next fragment);

update HDL: {
   DO {
      select the next hole descriptor from HDL;

      IF (fragment.first > hole.last) CONTINUE.
      IF (fragment.last < hole.first) CONTINUE.

      delete the current entry from HDL;

      IF (fragment.first > hole.first) {
         create new entry "new_hole" in HDL;
         new_hole.first <- hole.first;
         new_hole.last <- fragment.first - 1;
      }

      IF (fragment.last < hole.last) {
         create new entry "new_hole" in HDL;
         new_hole.first <- fragment.last + 1;
         new_hole.last <- hole.last;
      }

      /* one segment may span several holes */
      CONTINUE.
   } give up until (no entry from HDL)
}

The following algorithm implementation is based on the IP Datagram Reassembly Algorithm introduced in RFC 815, which presents hole descriptors as an alternative to the RCVBT (fragment received bit table) approach used by the reassembly procedure in RFC 791. And here is the process:

  1. Select the next hole descriptor from the hole descriptor list. If there are no more entries, go to step eight.

  2. If fragment.first is greater than hole.last, go to step one.

  3. If fragment.last is less than hole.first, go to step one.

  4. Delete the current entry from the hole descriptor list.

  5. If fragment.first is greater than hole.first, then create a new hole descriptor new_hole with new_hole.first equal to hole.first, and new_hole.last equal to fragment.first minus one (-1).

  6. If fragment.last is less than hole.last, then create a new hole descriptor new_hole, with new_hole.first equal to fragment.last plus one (+1) and new_hole.last equal to hole.last.

  7. Go to step one.

  8. If the hole descriptor list holds no hole but the one below a capture that missed the handshake, the stream is complete. Pass it on to the higher level protocol processor for further handling. Otherwise, return.

TCP has no more fragments flag, so the hole above the last octet received stays open until a FIN cuts it back to end at the sequence number the FIN occupies. A RST submits the stream as it stands, holes and all.

Terminology

reasm.tcp.packet

Data structure for TCP datagram reassembly (TCP.reassembly) is as follows:

ip_info = ip.info
tcp_info = tcp.info

raw_len = len(tcp.packet.payload)
packet_dict = Info(
  bufid = (
      ip_info.src,                    # source IP address
      tcp_info.srcport.port,          # source port
      ip_info.dst,                    # destination IP address
      tcp_info.dstport.port,          # destination port
  ),
  num = frame.info.number,            # original packet range number
  ack = tcp_info.ack,                 # acknowledgement
  dsn = tcp_info.seq,                 # data sequence number
  syn = tcp_info.flags.syn,           # synchronise flag
  fin = tcp_info.flags.fin,           # finish flag
  rst = tcp_info.flags.rst,           # reset connection flag
  header = tcp.packet.header,         # raw bytes type header
  payload = bytearray(
      tcp.packet.payload),            # raw bytearray type payload
  first = tcp_info.seq,               # first sequence number of payload
  last = tcp_info.seq + raw_len - 1,  # last sequence number of payload
  len = raw_len,                      # payload length, header excludes
  timestamp = float(
      frame.info.time_epoch),         # capture timestamp
)

Both first and last are absolute TCP sequence numbers and both are inclusive, so a segment carrying no payload at all has last one below first.

reasm.tcp.datagram

Data structure for reassembled TCP datagram (element from TCP.datagram tuple) is as follows:

(tuple) datagram
 |--> (Info) data
 |     |--> 'completed' : (Completion) COMPLETE --> reassembled in whole
 |     |--> 'id' : (Info) original packet identifier
 |     |            |--> 'src' --> (tuple)
 |     |            |               |--> (IPv4Address) ip.src
 |     |            |               |--> (int) tcp.srcport
 |     |            |--> 'dst' --> (tuple)
 |     |            |               |--> (IPv4Address) ip.dst
 |     |            |               |--> (int) tcp.dstport
 |     |            |--> 'ack' --> (int) original packet ACK number
 |     |--> 'index' : (tuple) packet numbers
 |     |               |--> (int) original packet range number
 |     |               |--> ...
 |     |--> 'header' : (bytes) initial TCP header
 |     |--> 'payload' : (bytes) reassembled payload
 |     |--> 'packet' : (Protocol) parsed reassembled payload
 |     |--> 'conflict' : (tuple) sequence ranges on which two segments disagreed
 |     |                  |--> (tuple) (first, last), absolute and inclusive
 |     |                  |--> ...
 |--> (Info) data
 |     |--> 'completed' : (Completion) PARTIAL or TIMEOUT --> incomplete
 |     |--> 'id' : (Info) original packet identifier
 |     |            |--> 'src' --> (tuple)
 |     |            |               |--> (IPv4Address) ip.src
 |     |            |               |--> (int) tcp.srcport
 |     |            |--> 'dst' --> (tuple)
 |     |            |               |--> (IPv4Address) ip.dst
 |     |            |               |--> (int) tcp.dstport
 |     |            |--> 'ack' --> (int) original packet ACK number
 |     |--> 'index' : (tuple) packet numbers
 |     |               |--> (int) original packet range number
 |     |               |--> ...
 |     |--> 'header' : (bytes) initial TCP header
 |     |--> 'payload' : (tuple) partially reassembled payload
 |     |                 |--> (bytes) payload fragment
 |     |                 |--> ...
 |     |--> 'packet' : (None) not parsed
 |     |--> 'conflict' : (tuple) sequence ranges on which two segments disagreed
 |     |                  |--> (tuple) (first, last), absolute and inclusive
 |     |                  |--> ...
 |--> (Info) data ...

packet is what tcp_datagram() parses payload to, on its first read, unless __protocol_type__ names another protocol.

completed and conflict are independent signals: a datagram can be COMPLETE and still carry a non-empty conflict – the resolution of a conflicting overlap is first-write-wins (RFC 9293 Section 3.10), so it never leaves a hole, and a contested range that was later filled in around does not stop the datagram from completing. conflict is what lets a caller tell a clean stream from a contested one, which completed alone cannot.

reasm.tcp.buffer

Data structure for internal buffering when performing reassembly algorithms (TCP._buffer) is as follows:

(dict) buffer --> memory buffer for reassembly
 |--> (tuple) BUFID : (dict)
 |       |--> ip.src      |
 |       |--> tcp.srcport |
 |       |--> ip.dst      |
 |       |--> tcp.dstport |
 |                        |--> 'hdl' : (list) hole descriptor list
 |                        |             |--> (Info) hole --> hole descriptor
 |                        |                   |--> "first" --> (int) sequence number of the
 |                        |                   |                     first missing octet
 |                        |                   |--> "last" --> (int) sequence number of the
 |                        |                                        last missing octet, inclusive
 |                        |--> 'hdr' : (bytes) initial TCP header
 |                        |--> 'ack' : (dict) ACK list
 |                                      |--> (int) ACK : (dict)
 |                                      |                 |--> 'ind' : (list) list of reassembled packets
 |                                      |                 |             |--> (int) packet range number
 |                                      |                 |--> 'isn' : (int) sequence number of the octet
 |                                      |                 |                  held in raw[0]
 |                                      |                 |--> 'len' : (int) length of payload buffer
 |                                      |                 |--> 'raw' : (bytearray) reassembled payload,
 |                                      |                                          holes set to b'\x00'
 |                                      |                 |--> 'gap' : (list) sequence ranges still
 |                                      |                 |                  zero-fill placeholder in 'raw'
 |                                      |                 |                  |--> (tuple) (first, last),
 |                                      |                 |                               absolute and
 |                                      |                 |                               inclusive
 |                                      |                 |                  |--> ...
 |                                      |                 |--> 'conflict' : (list) sequence ranges on which
 |                                      |                 |                  an arriving segment disagreed
 |                                      |                 |                  with bytes already in 'raw'
 |                                      |                 |                  |--> (tuple) (first, last),
 |                                      |                 |                               absolute and
 |                                      |                 |                               inclusive
 |                                      |                 |                  |--> ...
 |                                      |--> (int) ACK ...
 |                                      |--> ...
 |                        |--> 'timestamp' : (float) capture timestamp of the
 |                                                   first segment buffered
 |--> (tuple) BUFID ...

gap is deliberately not derived from hdl above. hdl is shared by every ACK in this dict, while each ACK’s own raw is private to it, so a different ACK’s segment closing a hole in hdl says nothing about whether this ACK has received anything at the same sequence numbers – consulting hdl for that question would discard a fragment’s own real bytes whenever a different ACK bucket under the same buffer ID happens to cover the same range first.

gap is kept in the same absolute, inclusive sequence number convention as conflict above (and as hdl’s own hole descriptors), rather than as a per-octet marker aligned with raw. That is what lets it survive isn being revised downwards by a reach-back segment: a per-octet marker aligned with raw has to be re-prefixed in lockstep with every such revision, while an absolute interval needs no shifting at all. It also means a fragment with no holes carries an empty list instead of a raw-sized marker.

Note

TCP reassembly has no timeout by default: no specification gives stream reassembly a deadline the way RFC 1122 Section 3.3.2 and RFC 8200 Section 4.5 give IP fragmentation one, and an idle connection is ordinary rather than pathological. timestamp is recorded regardless, so passing timeout to TCP enables the same eviction the IP reassemblers use – see TCP.__timeout__.

The hole descriptor list is kept in absolute TCP sequence numbers, once per BUFID, whereas each ACK’s payload buffer is indexed from its own isn – raw[n] holds the octet with sequence number isn + n, and isn is revised downwards whenever a segment turns up below the data already buffered, so it is not necessarily the connection’s own initial sequence number. Each ACK’s gap and conflict lists are kept in sequence numbers too; TCP.submit turns the gap entries into offsets into raw when it reports the buffer.

Data Models

class pcapkit.foundation.reassembly.data.tcp.Packet(bufid, dsn, ack, num, syn, fin, rst, len, first, last, header, payload, timestamp)[source]

Bases: Info

Data model for TCP packet representation.

bufid: tuple[_AT, int, _AT, int]

Buffer ID.

dsn: int

Data sequence number.

ack: int

Acknowledgment number.

num: int

Original packet range number.

syn: bool

Synchronise flag.

fin: bool

Finish flag.

rst: bool

Reset connection flag.

len: int

Payload length, header excluded.

first: int

Sequence number of the first octet of payload, i.e. the segment’s own sequence number. Absolute, not an offset into any payload buffer.

last: int

Sequence number of the last octet of payload, i.e. first + len - 1. Inclusive, so a segment carrying no payload at all has last one below first.

header: bytes

Raw bytes type header.

payload: bytearray

Raw bytearray type payload.

timestamp: float

Capture timestamp of the segment, in seconds since the Unix epoch, i.e. the capture’s clock rather than the host’s. It drives the reassembly timeout, which for TCP is off by default – see TCP.__timeout__.

class pcapkit.foundation.reassembly.data.tcp.DatagramID[_AT: (IPv4Address, IPv6Address)](src, dst, ack)[source]

Bases: Info, Generic[_AT]

Data model for TCP original packet identifier.

Note

The ports here stay bare int port numbers, where the pair TCP.submit hands Deferred is one of TCP members (#2417). The two are not the same thing to name: this is the flow’s identity, split out of BufferID, which every pcapkit.toolkit adapter builds from a plain port number – the native ones by reading AppType.port off the parsed header, the third-party ones from whatever their own engine exposes – and which is then used as a dict key and dumped as part of the report. The deferred pair is a dispatch key for Transport.analyze, which is what gains from carrying the service name.

src: tuple[_AT, int]

Source address.

dst: tuple[_AT, int]

Destination address.

ack: int

Original packet ACK number.

class pcapkit.foundation.reassembly.data.tcp.Datagram[_AT: (IPv4Address, IPv6Address)](completed, id, index, header, payload, packet, conflict)[source]

Bases: DeferredPacket, Info, Generic[_AT]

Data model for TCP reassembled datagram.

completed: Completion

How completely the datagram was reassembled, and why reassembly stopped; see Completion. Only Completion.COMPLETE is truthy, so if datagram.completed: reads as it would for a bool.

id: DatagramID[_AT]

Original packet identifier.

index: tuple[int, ...]

Packet numbers.

header: bytes

Initial TCP header.

payload: bytes | tuple[bytes, ...]

Reassembled payload (application layer data).

packet: ProtocolBase | None

Parsed TCP payload, as tcp_datagram() returns it unless the reassembly’s __protocol_type__ names another protocol, which then parses it. Analysed on first read rather than at construction; a Deferred may be passed in its place, and reading this attribute then runs it and keeps the result.

conflict: tuple[tuple[int, int], ...]

Sequence ranges on which two segments disagreed, i.e. where an arriving segment overlapped bytes already buffered but did not repeat them. Each entry is (first, last), absolute TCP sequence numbers and both inclusive – the same convention as Packet.first and Packet.last. Empty when the stream never saw a contested byte. A range that crosses 2 ** 32 therefore has last < first.

Resolution keeps the already-buffered bytes and discards the conflicting portion of whichever segment arrived later, per RFC 9293 Section 3.10 (“we reconstruct the segment to contain just the new data”); this field is what lets a caller tell a clean stream from a contested one, since a contested range does not, on its own, leave a hole for completed to report.

class pcapkit.foundation.reassembly.data.tcp.HoleDescriptor(first, last)[source]

Bases: Info

Data model for TCP hole descriptor.

Both bounds are unwrapped TCP sequence numbers – read modulo 2 ** 32 relative to the data already buffered, so a stream crossing 2 ** 32 continues past it – and both are inclusive, so a hole covers last - first + 1 octets. They are not offsets into Fragment.raw: the descriptor list is kept once per buffer ID, across every acknowledgement number’s payload buffer, and describes the stream as a whole. It decides when a stream whose FIN has been seen is whole. A hole whose first is -HOLE_OPEN_BOUND stands for whatever preceded a capture that missed the handshake, and does not count; one whose last is HOLE_OPEN_BOUND is still open above.

Added in version 1.5.0: Replaces the misspelt HoleDiscriptor.

Removed in version 1.5.0: HoleDiscriptor, and its re-export pcapkit.foundation.reassembly.data.TCP_HoleDiscriptor, with no alias; import HoleDescriptor and TCP_HoleDescriptor instead.

first: int

Sequence number of the first missing octet.

last: int

Sequence number of the last missing octet, inclusive.

pcapkit.foundation.reassembly.data.tcp.HOLE_OPEN_BOUND = 9223372036854775807

Open bound of the HoleDescriptor list. A hole whose last is this value is still open above, and one whose first is its negation stands for whatever preceded a capture that missed the handshake.

It is 2 ** 63 - 1 on every platform, not sys.maxsize, which is 2 ** 31 - 1 on GraalPy and 32-bit builds and so falls inside the 2 ** 32 sequence space. The unwrapped sequence line reaches it only after 2 ** 31 wraps. It stays an int, like the descriptor’s own fields, and in both signs fits a standard JSON number and a plistlib integer, which math.inf does not.

Added in version 1.5.0.

class pcapkit.foundation.reassembly.data.tcp.Fragment(ind, isn, len, raw, gap, conflict)[source]

Bases: Info

Data model for TCP ACK list fragment item.

ind: list[int]

Numbers of the packets merged into this fragment.

isn: int

Sequence number of the octet held in raw[0], i.e. the origin this buffer is indexed from: raw[n] holds the octet whose sequence number is isn + n. Revised downwards whenever a segment turns up below the data already buffered, so it is not necessarily the connection’s own initial sequence number. Like every sequence number held in this fragment – gap and conflict included – it is unwrapped, and so may exceed 2 ** 32 on a stream that crossed it.

len: int

Length of payload buffer.

raw: bytearray

Reassembled payload, holes set to b'\x00'.

gap: list[tuple[int, int]]

Sequence ranges, absolute and inclusive, still zero-fill placeholder in raw rather than an actually-received byte of this fragment. Only the gap-creating sites in TCP.reassembly – the forward append and the reach-back prepend, the only places that ever splice a bytearray(GAP) filler into raw – add an entry; an overlap merge only ever shrinks or removes one, filling it from the arriving segment. The one exception lies wholly outside raw: when a stream whose SYN was captured is split rather than padded across a gap wider than TCP_GAP_LIMIT, both halves record the range left between them. Any entry at all makes the fragment incomplete.

This is deliberately not derived from Buffer.hdl. hdl is one list shared by every acknowledgement number under the same buffer ID, so a segment landing in a different fragment can close a hole in hdl that this fragment’s own raw never filled – and consulting hdl to decide whether an overlapping position here was “already received” then answers a question about the wrong fragment. Tracking gaps on the fragment itself is what keeps the merge in TCP.reassembly from discarding this fragment’s own real bytes because some other fragment happened to have received something at the same absolute sequence numbers.

Absolute sequence numbers rather than offsets into raw – conflict below uses the same convention – because it is what TCP.reassembly already computes (GAP = PSN - (ISN + LEN) and its mirror), so this reuses an existing concept rather than adding a second one; and because a gap entry then never needs shifting when isn is revised downward by the reach-back path, unlike an offset-based or a per-octet representation. A typical fragment carries zero or a handful of entries, against a per-octet marker the length of the whole payload – the difference that matters on a clean stream, where the per-octet form pays a buffer-sized cost to record that nothing is missing at all.

conflict: list[tuple[int, int]]

Sequence ranges, absolute and inclusive, on which an arriving segment disagreed with bytes already held in raw. Accumulated across every merge into this fragment, in the order the conflicts were found; carried onto Datagram.conflict verbatim when the buffer is submitted.

pcapkit.foundation.reassembly.data.tcp.BufferID

Buffer ID.

alias of tuple[_AT, int, _AT, int]

class pcapkit.foundation.reassembly.data.tcp.Buffer(hdl, hdr, ack, timestamp)[source]

Bases: Info

Data model for TCP buffer entry.

hdl: list[HoleDescriptor]

Hole descriptor list.

hdr: bytes

Initial TCP header.

ack: dict[int, Fragment]

ACK list.

timestamp: float

Capture timestamp of the first segment buffered under this buffer ID, in seconds since the Unix epoch. Origin of the reassembly timer, and never revised: a later segment does not extend the deadline.

Type Variables

pcapkit.foundation.reassembly.data.tcp._AT: ipaddress.IPv4Address | ipaddress.IPv6Address

IP address type.