Bro Script Composer

File location:

  • Bundled implementation: source/client/python/compose.py

  • Cluster implementation: cluster/core/source/python/compose.py

Note

This file works as a standalone script for generating Bro scripts. It is NOT meant to be an importable module of the BroAPT system.

Introduction

As we can config what MIME types to extract through the BROAPT_LOAD_MIME environment variable, the BroAPT-Core framework will automatically generate the Bro scripts based on this environment variable and many others.

For MIME types with a shell-like pattern, we will use fnmatch.translate() to convert the pattern into a regular expression.

A generated Bro script for hook function extracting files with MIME type example/test-* would be as following:

@load ../__load__.bro

module FileExtraction;

hook FileExtraction::extract(f: fa_file, meta: fa_metadata) &priority=5 {
    if ( meta?$mime_type && /example\/test\-.*/ == meta$mime_type )
        break;
}

Besides this, the Bro script composer will also generate/rewrite the Bro configurations to customise several metrics and to load the scripts as specified in the environment variables.

Note

The full list of supported environment variables is as following:

Functions

compose.file_salt(uid: str)

Update the config.bro (Configurations) with provided uid as file_salt.

compose.compose()

Compose Bro scripts with environment variables defined.

Note

This function is the module entry.

compose.escape(mime_type: str)

Escape shell-like mime_type pattern to regular expression.

Caution

The underlying implementation of fnmatch.translate() calls re.escape() to escape special characters. However, in Python 3.6, the function will escape all characters other than ASCIIs, numbers and underlines (_); whilst in Python 3.7, it will only escape characters defined in re._special_chars_map.

Constants

Auxiliaries

compose.ROOT
Type:

str

Path to the BroAPT-Core framework source codes (absolute path at runtime).

compose.BOOLEAN_STATES = {'1': True,    '0': False, 'yes': True,  'no': False, 'true': True, 'false': False, 'on': True,   'off': False}

Mapping of boolean states, c.f. configparser.

Bro Configs

compose.LOGS_PATH
Type:

str (path)

Environ:

BROAPT_LOGS_PATH

Path to system logs.

compose.PCAP_PATH
Type:

str (path)

Environ:

BROAPT_PCAP_PATH

Path to source PCAP files.

compose.MIME_MODE
Type:

bool

Environ:

BROAPT_MIME_MODE

If group extracted files by MIME type.

compose.HASH_MODE_MD5
Type:

bool

Environ:

BROAPT_HASH_MD5

Calculate MD5 hash of extracted files.

compose.HASH_MODE_SHA1
Type:

bool

Environ:

BROAPT_HASH_SHA1

Calculate SHA1 hash of extracted files.

compose.HASH_MODE_SHA256
Type:

bool

Environ:

BROAPT_HASH_SHA256

Calculate SHA256 hash of extracted files.

compose.X509_MODE
Type:

bool

Environ:

BROAPT_X509_MODE

Include X509 information when running Bro.

compose.ENTROPY_MODE
Type:

bool

Environ:

BROAPT_ENTROPY_MODE

Include file entropy information when running Bro.

compose.DUMP_PATH
Type:

str (path)

Environ:

BROAPT_DUMP_PATH

Path to extracted files.

. data:: compose.FILE_BUFFER

type:

int (uint64)

environ:

BROAPT_FILE_BUFFER

Reassembly buffer size for file extraction.

compose.SIZE_LIMIT
Type:

int (uint64)

Environ:

BROAPT_SIZE_LIMIT

Size limit of extracted files.

compose.JSON_MODE
Type:

bool

Environ:

BROAPT_JSON_MODE

Toggle Bro logs in JSON or ASCII format.

compose.LOAD_MIME
Type:

List[str] (case-insensitive)

Environ:

BROAPT_LOAD_MIME

A , or ; separated string of MIME types to be extracted.

compose.LOAD_PROTOCOL
Type:

List[str] (case-insensitive)

Environ:

BROAPT_LOAD_PROTOCOL

A , or ; separated string of application layer protocols to be extracted, can be any of dtls, ftp, http, irc and smtp.

Subsitute Patterns

compose.FILE_TEMP
Type:

Tuple[str]

Template for MIME type extraction Bro scripts.

compose.MIME_REGEX
Type:

re.Pattern

Pattern for mime (MIME_MODE).

compose.LOGS_REGEX
Type:

re.Pattern

Pattern for logs (LOGS_PATH).

compose.HASH_REGEX_MD5
Type:

re.Pattern

Pattern for md5 (HASH_MODE_MD5).

compose.HASH_REGEX_SHA1
Type:

re.Pattern

Pattern for sha1 (HASH_MODE_SHA1).

compose.HASH_REGEX_SHA256
Type:

re.Pattern

Pattern for sha256 (HASH_MODE_SHA256).

compose.X509_REGEX
Type:

re.Pattern

Pattern for x509 (X509_MODE).

compose.ENTR_REGEX
Type:

re.Pattern

Pattern for entropy (ENTROPY_MODE).

compose.JSON_REGEX
Type:

re.Pattern

Pattern for use_json (JSON_MODE).

compose.SALT_REGEX
Type:

re.Pattern

Pattern for file_salt (file_salt()).

compose.FILE_REGEX
Type:

re.Pattern

Pattern for file_buffer (FILE_BUFFER).

compose.PATH_REGEX
Type:

re.Pattern

Pattern for path_prefix (DUMP_PATH).

compose.SIZE_REGEX
Type:

re.Pattern

Pattern for size_limit (SIZE_LIMIT).

compose.LOAD_REGEX
Type:

re.Pattern

Pattern for @load loading scripts.