netFound-large

Description

netFound is a network traffic encoder model that uses transformer architecture and includes a pretraining phase on unlabeled data to achieve high results.

Key features:

• netFound takes raw PCAP data as input
• netFound can (and need) be pretrained on the unlabeled dataset
• netFound uses Hierarchical Transformer architecture to take into account packet burst and flow behavior
• netFound uses burst metadata (inter arrival time, number of bytes per burst, etc)

Details

• Model config: large (24 layers, 16 attention heads, 1024 hidden size)
• Pretraining details: 128 GPUs, 15k steps, ~7bln tokens seen

Source code

https://github.com/SNL-UCSB/netFound