JSON to PCAP

Script which can be used to reconstruct pcap and perform packet modifications from tshark json output.

Source code is located on github https://github.com/H21lab/json2pcap


This repository contains more recent and experimental changes compared to Wireshark. https://github.com/wireshark/wireshark/tree/master/tools/json2pcap


Command tshark -T json -x and new tshark -T jsonraw output adds into hex-data output in JSON also the information on which position each field is dissected in the original frame, what is the field length, the bitmask (for not byte aligned fields) and the type. This information can be used for latter processing. One use-case is the json2pcap script included in wireshark, which assembles the protocol layers back together from upper to lowers layers. This allows revers json to pcap conversion and also the packet modification/editing/rewriting.


/wireshark/tools/json2pcap/json2pcap.py

usage: json2pcap.py [-h] [-p] infile [infile ...]
Utility to generate pcap from json format.
Packet modification:In input json it is possible to modify the raw values of decoded fields.The output pcap will include the modified values. The algorithm ofgenerating the output pcap is to get all raw hex fields from input json andthen assembling them by layering from longest (less decoded fields) toshortest (more decoded fields). It means if the modified raw field isshorter field (more decoded field) it takes precedence against modificationin longer field (less decoded field). If the json includes duplicated rawfields with same position and length, the behavior is not deterministic.For manual packet editing it is always possible to remove any not requiredraw fields from json, only frame_raw is field mandatory for reconstruction.
Packet modification with -p switch:The python script is generated instead of pcap. This python script whenexecuted will generate the pcap of 1st packet from input json. Thegenerated code includes the decoded fields and the function to assembly thepacket. This enables to modify the script and programmatically edit orencode the packet variables. The assembling algorithm is different, becausethe decoded packet fields are relative and points to parent node with theirposition (compared to input json which has absolute positions).
positional arguments: infile json generated by tshark -T jsonraw or by tshark -T json -x
optional arguments: -h, --help show this help message and exit -p, --python generate python payload instead of pcap (only 1st packet)

Example

1. Original dns.cap from wireshark example

2. Create json from pcap

tshark -T jsonraw -J "dns" -r dns.cap > dns.cap.json
{ "_index": "packets-2017-02-27", "_type": "pcap_file", "_score": null, "_source": { "layers": { "frame_raw": ["00c09f32418c00e018b10cad0800450000380000400040116547c0a8aa08c0a8aa14801b0035002485ed10320100000100000000000006676f6f676c6503636f6d0000100001", 0, 70, 0, 1], "frame": { "filtered": "frame" }, "eth_raw": ["00c09f32418c00e018b10cad0800", 0, 14, 0, 1], "eth": { "filtered": "eth" }, "ip_raw": ["450000380000400040116547c0a8aa08c0a8aa14", 14, 20, 0, 1], "ip": { "filtered": "ip" }, "udp_raw": ["801b0035002485ed", 34, 8, 0, 1], "udp": { "filtered": "udp" }, "dns_raw": ["10320100000100000000000006676f6f676c6503636f6d0000100001", 42, 28, 0, 1], "dns": { "dns.id_raw": ["1032", 42, 2, 0, 5], "dns.flags_raw": ["0100", 44, 2, 0, 5], "dns.flags_tree": { "dns.flags.response_raw": ["0", 44, 2, 32768, 2], "dns.flags.opcode_raw": ["0", 44, 2, 30720, 5], "dns.flags.truncated_raw": ["0", 44, 2, 512, 2], "dns.flags.recdesired_raw": ["1", 44, 2, 256, 2], "dns.flags.z_raw": ["0", 44, 2, 64, 2], "dns.flags.checkdisable_raw": ["0", 44, 2, 16, 2] }, "dns.count.queries_raw": ["0001", 46, 2, 0, 5], "dns.count.answers_raw": ["0000", 48, 2, 0, 5], "dns.count.auth_rr_raw": ["0000", 50, 2, 0, 5], "dns.count.add_rr_raw": ["0000", 52, 2, 0, 5], "Queries": { "google.com: type TXT, class IN": { "dns.qry.name_raw": ["06676f6f676c6503636f6d00", 54, 12, 0, 26], "dns.qry.name.len_raw": ["06676f6f676c6503636f", 54, 10, 0, 5], "dns.count.labels_raw": ["06676f6f676c6503636f", 54, 10, 0, 5], "dns.qry.type_raw": ["0010", 66, 2, 0, 5], "dns.qry.class_raw": ["0001", 68, 2, 0, 5] } }, } } }}

3. Modify dns.cap.json

vi dns.cap.json

{ "_index": "packets-2017-02-27", "_type": "pcap_file", "_score": null, "_source": { "layers": { "frame_raw": ["00c09f32418c00e018b10cad0800450000380000400040116547c0a8aa08c0a8aa14801b0035002485ed10320100000100000000000006676f6f676c6503636f6d0000100001", 0, 70, 0, 1], "frame": { "filtered": "frame" }, "eth_raw": ["00c09f32418c00e018b10cad0800", 0, 14, 0, 1], "eth": { "filtered": "eth" }, "ip_raw": ["450000380000400040116547c0a8aa08c0a8aa14", 14, 20, 0, 1], "ip": { "filtered": "ip" }, "udp_raw": ["801b0035002485ed", 34, 8, 0, 1], "udp": { "filtered": "udp" }, "dns_raw": ["10320100000100000000000006676f6f676c6503636f6d0000100001", 42, 28, 0, 1], "dns": { "dns.id_raw": ["1032", 42, 2, 0, 5], "dns.flags_raw": ["0100", 44, 2, 0, 5], "dns.flags_tree": { "dns.flags.response_raw": ["0", 44, 2, 32768, 2], "dns.flags.opcode_raw": ["0", 44, 2, 30720, 5], "dns.flags.truncated_raw": ["0", 44, 2, 512, 2], "dns.flags.recdesired_raw": ["1", 44, 2, 256, 2], "dns.flags.z_raw": ["0", 44, 2, 64, 2], "dns.flags.checkdisable_raw": ["0", 44, 2, 16, 2] }, "dns.count.queries_raw": ["0001", 46, 2, 0, 5], "dns.count.answers_raw": ["0000", 48, 2, 0, 5], "dns.count.auth_rr_raw": ["0000", 50, 2, 0, 5], "dns.count.add_rr_raw": ["0000", 52, 2, 0, 5], "Queries": { "google.com: type TXT, class IN": { "dns.qry.name_raw": ["0667676767676703636f6d00", 54, 12, 0, 26], "dns.qry.name.len_raw": ["06676f6f676c6503636f", 54, 10, 0, 5], "dns.count.labels_raw": ["06676f6f676c6503636f", 54, 10, 0, 5], "dns.qry.type_raw": ["0010", 66, 2, 0, 5], "dns.qry.class_raw": ["0001", 68, 2, 0, 5] } } } } }}



4. json2pcap.py to generate pcap

The raw fields are flatten and by this the frame is created. The shortest raw fields overwrite the longer fields.

./wireshark/tools/json2pcap/json2pcap.py -p dns.cap.json

5. (OPTIONAL) json2pcap.py to generate python payload instead of pcap.

This is with relative positions to the parents. And the pcap reconstruct algorithm is different by using the parents and child hierarchy. This can be useful for possible latter more complex encoding (e.g. using scapy, dpkt, pycrate/libmich or other libraries).

./wireshark/tools/json2pcap/json2pcap.py -p dns.cap.json

vi dns.cap.json.py

...def main(): d = OrderedDict() d['frame_raw'] = [None, u'00c09f32418c00e018b10cad0800450000380000400040116547c0a8aa08c0a8aa14801b0035002485ed10320100000100000000000006676f6f676c6503636f6d0000100001', 0, 140, 0, 1] d['eth_raw'] = ['frame_raw', u'00c09f32418c00e018b10cad0800', 0, 28, 0, 1] d['ip_raw'] = ['frame_raw', u'450000380000400040116547c0a8aa08c0a8aa14', 14, 40, 0, 1] d['udp_raw'] = ['frame_raw', u'801b0035002485ed', 34, 16, 0, 1] d['dns_raw'] = ['frame_raw', u'10320100000100000000000006676f6f676c6503636f6d0000100001', 42, 56, 0, 1] d['dns_id_raw'] = [u'dns_raw', u'1032', 0, 4, 0, 5] d['dns_flags_raw'] = [u'dns_raw', u'0100', 2, 4, 0, 5] d['dns_flags_response_raw'] = [u'dns_flags_raw', u'0', 0, 1, 32768, 2] d['dns_flags_opcode_raw'] = [u'dns_flags_raw', u'0', 0, 1, 30720, 5] d['dns_flags_truncated_raw'] = [u'dns_flags_raw', u'0', 0, 1, 512, 2] d['dns_flags_recdesired_raw'] = [u'dns_flags_raw', u'1', 0, 1, 256, 2] d['dns_flags_z_raw'] = [u'dns_flags_raw', u'0', 0, 1, 64, 2] d['dns_flags_checkdisable_raw'] = [u'dns_flags_raw', u'0', 0, 1, 16, 2] d['dns_count_queries_raw'] = [u'dns_raw', u'0001', 4, 4, 0, 5] d['dns_count_answers_raw'] = [u'dns_raw', u'0000', 6, 4, 0, 5] d['dns_count_auth_rr_raw'] = [u'dns_raw', u'0000', 8, 4, 0, 5] d['dns_count_add_rr_raw'] = [u'dns_raw', u'0000', 10, 4, 0, 5] d['dns_qry_name_raw'] = [u'dns_raw', u'0667676767676703636f6d00', 12, 24, 0, 26] d['dns_qry_name_len_raw'] = [u'dns_raw', u'06676f6f676c6503636f', 12, 20, 0, 5] d['dns_count_labels_raw'] = [u'dns_raw', u'06676f6f676c6503636f', 12, 20, 0, 5] d['dns_qry_type_raw'] = [u'dns_raw', u'0010', 24, 4, 0, 5] d['dns_qry_class_raw'] = [u'dns_raw', u'0001', 26, 4, 0, 5] generate_pcap(d)...
python dns.cap.json.py

6. New pcap dns.cap.json.pcap