Anomaly Detection

Python scripts which can help to analyze anomalies in pcap file.

Source code is located in https://github.com/H21lab/Anomaly-Detection.


Scripts use as input json generated from pcap by the following command:

./tshark -T ek -x -r input.pcap > input.pcap.json

ad_tf_autoencoder.ipynb

Unsupervised learning by using autoencoder neural network by using tensorflow.


ad_tf.py

Script using tensorflow classifier neural network and tshark input to detect anomalies. Neural network should be first trained by normal json pcap and anomaly json pcap with defined fields identifying relevant protocol fields.


Run script:

cat input.pcap.json | python ad_tf.py -i normal.pcap.json -a anomaly.pcap.json -f field_1 field_2 .... field_n

ad_simple.py

Simple script simulating SQL GROUP BY and ORDER BY in pcap. Input is only single pcap and defining the fields in pcap which should be used for grouping and ordering.


Run script:

cat input.pcap.json | python ad_simple.py field_1 field_2 .... field_n

UPDATES:

2.6.2019: Tensorflow IO project: issue on github has been opened to enable direct pcap data ingestion

19.6.2019: Tensorflow IO project: support to pcap dataset has been added