I/O Objects

pcapkit.corekit.io contains SeekableReader, a seekable customisation of io.BufferedReader, and the stream proxies: PeekableStream, which adds peek to a seekable stream, and NamedStream, which gives a stream a name; advance(), which moves a stream past more octets than one read or seek can take; and can_seek(), which tells whether a stream really seeks.

class pcapkit.corekit.io.SeekableReader(raw, buffer_size=io.DEFAULT_BUFFER_SIZE, buffer_save=False, buffer_path=None, *, stream_closing=True)

Bases: BufferedReader

Seekable buffered reader.

A buffered binary stream providing higher-level access to a readable, non seekable RawIOBase raw binary stream. It inherits BufferedIOBase.

When reading data from this object, a larger amount of data may be requested from the underlying raw stream, and kept in an internal buffer. The buffered data can then be returned directly on subsequent reads.

The constructor creates a BufferedReader for the given readable raw stream and buffer_size. If buffer_size is omitted, DEFAULT_BUFFER_SIZE is used.

Parameters:

• raw (IO[bytes]) – Underlying raw stream.

• buffer_size (int) – Buffer size.

• buffer_save (bool) – Whether to save buffer to file.

• buffer_path (str | None) – Path to save buffer.

• stream_closing (bool) – Whether the stream should be closed upon exiting.

Changed in version 1.5.0: truncate() raises UnsupportedOperation, as writable() returns False (#645). A refused seek() leaves the position where it was, and a position the lookback buffer has dropped raises instead of reading from it (#643, #644).

Removed in version 1.5.0: writeable(), a misspelling that never overrode io.IOBase.writable(); use writable() (#645).

property raw: RawIOBase

The underlying raw stream (a RawIOBase instance) that BufferedIOBase deals with. This is not part of the BufferedIOBase API and may not exist on some implementations.

property closed: bool

True if the stream is closed.

_stream: IO[bytes]

Underlying raw stream.

_closed: bool

Whether the stream is closed.

read(size=-1, /)

Read and return size bytes, or if size is not given or negative, until EOF or if the read call would block in non-blocking mode.

Return type:

bytes

read1(size=-1, /)

Read and return up to size bytes with only one call on the raw stream. If at least one byte is buffered, only buffered bytes are returned. Otherwise, one raw stream read call is made.

Return type:

bytes

readinto(b, /)

Read bytes into a pre-allocated, writable bytes-like object b and return the number of bytes read. For example, b might be a bytearray.

Like read(), multiple reads may be issued to the underlying raw stream, unless the latter is interactive.

A BlockingIOError is raised if the underlying raw stream is in non blocking-mode, and has no data available at the moment.

Return type:

int

readinto1(b, /)

Read bytes into a pre-allocated, writable bytes-like object b, using at most one call to the underlying raw stream’s read() (or readinto()) method. Return the number of bytes read.

A BlockingIOError is raised if the underlying raw stream is in non blocking-mode, and has no data available at the moment.

Return type:

int

readable()

Return True if the stream can be read from. If False, read() will raise OSError.

Return type:

bool

readline(size=-1, /)

Read and return one line from the stream. If size is specified, at most size bytes will be read.

The line terminator is always b'\n' for binary files; for text files, the newline argument to open() can be used to select the line terminator(s) recognized.

Return type:

bytes

readlines(hint=-1, /)

Read and return a list of lines from the stream. hint can be specified to control the number of lines read: no more lines will be read if the total size (in bytes/characters) of all lines so far exceeds hint.

hint values of 0 or less, as well as None, are treated as no hint.

Note that it’s already possible to iterate on file objects using for line in file: ... without calling file.readlines().

Return type:

list[bytes]

writable()

Return True if the stream supports writing. If False, write() and truncate() will raise OSError.

Return type:

bool

write(b, /)

Write the given bytes-like object, b, and return the number of bytes written (always equal to the length of b in bytes, since if the write fails an OSError will be raised). Depending on the actual implementation, these bytes may be readily written to the underlying stream, or held in a buffer for performance and latency reasons.

When in non-blocking mode, a BlockingIOError is raised if the data needed to be written to the raw stream but it couldn’t accept all the data without blocking.

The caller may release or mutate b after this method returns, so the implementation should only access b during the method call.

Return type:

int

seekable()

Return True if the stream supports random access. If False, seek(), tell() and truncate() will raise OSError.

Return type:

bool

seek(offset, whence=io.SEEK_SET, /)

Change the stream position to the given byte offset. offset is interpreted relative to the position indicated by whence. The default value for whence is SEEK_SET. Values for whence are:

• SEEK_SET or 0 - start of the stream (the default); offset should be zero or positive

• SEEK_CUR or 1 - current stream position; offset may be negative

• SEEK_END or 2 - end of the stream; offset is usually negative

Return the new absolute position.

Note

The target is computed, then validated, and only then written to _tell. A branch that assigned the position before deciding whether to accept it would leave a refused seek having moved it anyway, with the resync below – the one thing that puts the buffer’s cursor back in step – skipped on the way out. A caller that catches the error and reasonably takes the position to be unchanged would then read from the rejected offset instead, silently and without a second error.

The negative check applies to every whence, rather than only SEEK_SET looking at its offset. An absolute position below zero cannot exist under any of them, and it is not the same failure as a position that has merely slid out of the window – which is ordinary, and recoverable with buffer_save=True. Reporting the first as negative seek value keeps the two distinguishable, and, being a property of the position rather than of the window, it holds with a saved buffer as well, which is exempt from the window check and would otherwise return a negative position as if the seek had worked.

A non-seekable stream has no end until it is over, so SEEK_END is as far as _read_ahead() reads past the position: _read_ahead_size octets, a buffer’s worth by default, or to the real end if that comes first. The frame readers measure what is left this way, and the engines set that size to the next record, so a record is measured whole, however long. The buffer’s far edge will not do as the end: once the buffer has filled it is just how far the stream has been read, so the first record ending there would be taken for the end of the capture (#1586). The read slides the buffer, so a position before the one SEEK_END was sought from may then be refused, as _read_ahead() says.

Return type:

int

tell()

Return the current stream position.

Return type:

int

truncate(size=None, /)

Resize the stream to the given size in bytes (or the current position if size is not specified).

Raises:

UnsupportedOperation – Always, since the reader is not writable.

Return type:

int

Note

io.IOBase.writable() gates this method as well as write() – “If False, write() and truncate() will raise OSError” – and writable() here returns False, so refusing is what the contract asks for. It is also what CPython’s own read-only buffered readers do: both the accelerated io.BufferedReader and the pure-Python _pyio.BufferedReader raise io.UnsupportedOperation, the latter from _BufferedIOMixin.truncate’s _checkWritable() (#645).

Resizing the lookback buffer is reachable internally, as _truncate_buffer(). It never touches the underlying stream – it resizes a private lookback window – which is why it stays, under a private name.

close()

Flush and close this stream. This method has no effect if the file is already closed. Once the file is closed, any operation on the file (e.g. reading or writing) will raise a ValueError.

As a convenience, it is allowed to call this method more than once; only the first call, however, will have an effect.

flush()

Flush the write buffers of the stream if applicable. This does nothing for read-only and non-blocking streams.

peek(size=0)

Return bytes from the stream without advancing the position.

At most one single read on the raw stream is done to satisfy the call. The number of bytes returned may be less or more than requested.

Return type:

bytes

detach()

Separate the underlying raw stream from the buffer and return it.

After the raw stream has been detached, the buffer is in an unusable state.

Some buffers, like BytesIO, do not have the concept of a single raw stream to return from this method. They raise UnsupportedOperation.

Return type:

RawIOBase

fileno()

Return the underlying file descriptor (an integer) of the stream if it exists. An OSError is raised if the IO object does not use a file descriptor.

Return type:

int

isatty()

Return True if the stream is interactive (i.e., connected to a terminal/tty device).

Return type:

bool

_read_ahead_size: int

Octets seek() reads past the position to find the end of the stream; the buffer size by default.

_read_ahead()

Read the stream to _read_ahead_size octets past the position.

Returns:

that many octets past the position, or the end of the stream if it ended first. seek() takes this as the end.

Return type:

int

Note

The buffer slides forward to make the room, so after this only the last _buffer_size less _read_ahead_size octets before the position can still be sought back to – none, at the default – unless buffer_save keeps them. It grows instead of sliding past the position itself, so a record longer than the buffer is held whole, as its reader seeks back to its first octet; a bogus length grows it only as far as the stream goes. A live stream holds the reader here until the octets arrive, which is why the engines set _read_ahead_size to the record they are about to read.

_truncate_buffer(size=None, /)

Resize the lookback buffer to the given size in bytes (or the current position if size is not specified). The current stream position isn’t changed. This resizing can extend or reduce the current buffer size. In case of extension, the new area is zero-filled. The new buffer size is returned.

Note

Nothing here writes to the underlying stream – write() raises – so what this resizes is the buffer, not the stream behind it. That is why truncate() refuses (#645) while this remains: the operation is a private-window one, not an io.IOBase write. The buffer is a sliding window over a stream that cannot be seeked: its octet 0 sits at absolute offset _buffer_set, its content occupies [0:_buffer_cur], and everything past that is padding never read.

As for which octets survive: an extension appends its zero octets at the tail, the new area being by definition the region past the old end. A reduction below the content keeps the most recent size octets and advances _buffer_set past the ones it drops, because what this holds is lookback: the octets it can still answer for are the ones just read, and the stream is already beyond them. Padding is never kept in preference to content either way.

Advancing _buffer_set is what keeps _buffer_set + _buffer_cur equal to how far the stream has actually been consumed, which seek() relies on to decide whether it may read ahead to fill a gap. A reduction that shrank the window without moving its base would leave that sum short of the stream, and the next forward seek() would splice in octets from the wrong absolute offset without complaining.

Octets dropped by a reduction are gone for good, since the stream cannot be rewound to re-supply them. A position left among them is then before the window, which seek() refuses as it refuses any other.

Return type:

int

class pcapkit.corekit.io.PeekableStream(stream)

Bases: object

Give a seekable binary stream without peek one.

Extractor and the PCAP-NG engine read ahead with peek, which io.BufferedReader has and a plain typing.IO such as io.BytesIO does not (#1506). The read-ahead is a read and a seek back, and every other attribute is the stream’s own. It is not an io.IOBase, so it has no finaliser that could close the caller’s stream behind their back (#610).

Copying and pickling follow the stream (#1524). copy.copy() gives a second proxy over the same stream, so the two share one position. copy.deepcopy() and pickle copy the stream too, so they work exactly when the stream itself can be deep-copied or pickled at that protocol: an io.BytesIO pickles at protocol 2 and above, and an open file not at all.

Parameters:

stream (IO[bytes]) – seekable binary stream

peek(size=0)

Return up to size octets, at least one if any remain, without consuming them.

Parameters:

size (int) – number of octets wanted

Return type:

bytes

Returns:

The octets at the current position.

class pcapkit.corekit.io.NamedStream(stream, name)

Bases: object

Read-only proxy that gives a stream the name attribute.

pcapfile.savefile.load_savefile dereferences input_file.name unconditionally, on the way into its trace helper. That is fine for the BufferedReader Extractor normally holds, but not for the SeekableReader it substitutes when the caller supplied a non-seekable stream – that class exposes no name, so the load would fail with AttributeError before a single byte was read.

Parameters:

• stream (BinaryIO) – Underlying binary stream.

• name (str) – Name to report as name.

name

Name of the underlying stream.

read(size=-1)

Read from the underlying stream.

Parameters:

size (int) – Number of bytes to read; all remaining bytes if negative.

Return type:

bytes

pcapkit.corekit.io.advance(file, count, /)

Move file forward by count octets, however many one read or seek can take.

Parameters:

• file (IO[bytes]) – Stream to move.

• count (int) – Octets to move past, counted from the position.

A frame reader asks for a record no further than one read can go, sys.maxsize, so a record the file really holds past that is read only as far as the limit; the next record still starts where this one declares it ends (#1975). A seekable stream gets there by relative seeks, none longer than sys.maxsize, and may land past its end, as an absolute seek would. A SeekableReader, or a stream that cannot seek – including one that only says it can, as a Windows pipe does (#2108) – reads the octets and discards them, no more than READ_CHUNK at a time, and stops at the end of the stream: a SeekableReader seek reads at most a buffer’s worth past what it holds, so a longer one would stop short of the target.

pcapkit.corekit.io.can_seek(stream)

Whether stream can really seek, rather than only saying it can.

On Windows before CPython 3.16, a pipe reports seekable() and its seek silently does nothing (#2097), so the answer is trusted only for a regular file, or for a stream with no descriptor to ask.

Parameters:

stream (IO[bytes]) – The input stream.

Return type:

bool