JSON to PCAP

Script which can be used to reconstruct pcap and perform packet modifications from tshark json output.

Source code is located on github https://github.com/H21lab/json2pcap


This repository contains more recent and experimental changes compared to Wireshark. https://github.com/wireshark/wireshark/tree/master/tools/json2pcap


Command tshark -T json -x and new tshark -T jsonraw output adds into hex-data output in JSON also the information on which position each field is dissected in the original frame, what is the field length, the bitmask (for not byte aligned fields) and the type. This information can be used for latter processing. One use-case is the json2pcap script included in wireshark, which assembles the protocol layers back together from upper to lowers layers. This allows revers json to pcap conversion and also the packet modification/editing/rewriting.


/wireshark/tools/json2pcap/json2pcap.py

usage: json2pcap.py [-h] [-p] infile [infile ...]

Utility to generate pcap from json format.

Packet modification:
In input json  it is possible to  modify the raw values  of decoded fields.
The  output  pcap  will  include  the modified  values.  The  algorithm  of
generating the output pcap is to get all raw hex fields from input json and
then  assembling them  by layering  from longest  (less decoded  fields) to
shortest  (more decoded  fields). It  means if  the modified  raw field  is
shorter field (more decoded field) it takes precedence against modification
in longer field  (less decoded field). If the json  includes duplicated raw
fields with  same position and  length, the behavior is  not deterministic.
For manual packet editing it is  always possible to remove any not required
raw fields from json, only frame_raw is field mandatory for reconstruction.

Packet modification with -p switch:
The python  script is generated  instead of  pcap. This python  script when
executed  will  generate the  pcap  of  1st  packet  from input  json.  The
generated code includes the decoded fields and the function to assembly the
packet.  This enables  to modify  the script  and programmatically  edit or
encode the packet variables. The assembling algorithm is different, because
the decoded packet fields are relative and points to parent node with their
position (compared to input json which has absolute positions).

positional arguments:
  infile        json generated by tshark -T jsonraw or by tshark -T json -x

optional arguments:
  -h, --help    show this help message and exit
  -p, --python  generate python payload instead of pcap (only 1st packet)


Example

1. Original dns.cap from wireshark example

2. Create json from pcap

tshark -T jsonraw -J "dns" -r dns.cap > dns.cap.json


{
    "_index": "packets-2017-02-27",
    "_type": "pcap_file",
    "_score": null,
    "_source": {
      "layers": {
        "frame_raw": ["00c09f32418c00e018b10cad0800450000380000400040116547c0a8aa08c0a8aa14801b0035002485ed1032010000010
0000000000006676f6f676c6503636f6d0000100001", 0, 70, 0, 1],
        "frame": {
          "filtered": "frame"
        },
        "eth_raw": ["00c09f32418c00e018b10cad0800", 0, 14, 0, 1],
        "eth": {
          "filtered": "eth"
        },
        "ip_raw": ["450000380000400040116547c0a8aa08c0a8aa14", 14, 20, 0, 1],
        "ip": {
          "filtered": "ip"
        },
        "udp_raw": ["801b0035002485ed", 34, 8, 0, 1],
        "udp": {
          "filtered": "udp"
        },
        "dns_raw": ["10320100000100000000000006676f6f676c6503636f6d0000100001", 42, 28, 0, 1],
        "dns": {
          "dns.id_raw": ["1032", 42, 2, 0, 5],
          "dns.flags_raw": ["0100", 44, 2, 0, 5],
          "dns.flags_tree": {
            "dns.flags.response_raw": ["0", 44, 2, 32768, 2],
            "dns.flags.opcode_raw": ["0", 44, 2, 30720, 5],
            "dns.flags.truncated_raw": ["0", 44, 2, 512, 2],
            "dns.flags.recdesired_raw": ["1", 44, 2, 256, 2],
            "dns.flags.z_raw": ["0", 44, 2, 64, 2],
            "dns.flags.checkdisable_raw": ["0", 44, 2, 16, 2]
          },
          "dns.count.queries_raw": ["0001", 46, 2, 0, 5],
          "dns.count.answers_raw": ["0000", 48, 2, 0, 5],
          "dns.count.auth_rr_raw": ["0000", 50, 2, 0, 5],
          "dns.count.add_rr_raw": ["0000", 52, 2, 0, 5],
          "Queries": {
            "google.com: type TXT, class IN": {
              "dns.qry.name_raw": ["06676f6f676c6503636f6d00", 54, 12, 0, 26],
              "dns.qry.name.len_raw": ["06676f6f676c6503636f", 54, 10, 0, 5],
              "dns.count.labels_raw": ["06676f6f676c6503636f", 54, 10, 0, 5],
              "dns.qry.type_raw": ["0010", 66, 2, 0, 5],
              "dns.qry.class_raw": ["0001", 68, 2, 0, 5]
            }
          },
        }
      }
    }
}


3. Modify dns.cap.json

vi dns.cap.json

{
    "_index": "packets-2017-02-27",
    "_type": "pcap_file",
    "_score": null,
    "_source": {
      "layers": {
        "frame_raw": ["00c09f32418c00e018b10cad0800450000380000400040116547c0a8aa08c0a8aa14801b0035002485ed1032010000010
0000000000006676f6f676c6503636f6d0000100001", 0, 70, 0, 1],
        "frame": {
          "filtered": "frame"
        },
        "eth_raw": ["00c09f32418c00e018b10cad0800", 0, 14, 0, 1],
        "eth": {
          "filtered": "eth"
        },
        "ip_raw": ["450000380000400040116547c0a8aa08c0a8aa14", 14, 20, 0, 1],
        "ip": {
          "filtered": "ip"
        },
        "udp_raw": ["801b0035002485ed", 34, 8, 0, 1],
        "udp": {
          "filtered": "udp"
        },
        "dns_raw": ["10320100000100000000000006676f6f676c6503636f6d0000100001", 42, 28, 0, 1],
        "dns": {
          "dns.id_raw": ["1032", 42, 2, 0, 5],
          "dns.flags_raw": ["0100", 44, 2, 0, 5],
          "dns.flags_tree": {
            "dns.flags.response_raw": ["0", 44, 2, 32768, 2],
            "dns.flags.opcode_raw": ["0", 44, 2, 30720, 5],
            "dns.flags.truncated_raw": ["0", 44, 2, 512, 2],
            "dns.flags.recdesired_raw": ["1", 44, 2, 256, 2],
            "dns.flags.z_raw": ["0", 44, 2, 64, 2],
            "dns.flags.checkdisable_raw": ["0", 44, 2, 16, 2]
          },
          "dns.count.queries_raw": ["0001", 46, 2, 0, 5],
          "dns.count.answers_raw": ["0000", 48, 2, 0, 5],
          "dns.count.auth_rr_raw": ["0000", 50, 2, 0, 5],
          "dns.count.add_rr_raw": ["0000", 52, 2, 0, 5],
          "Queries": {
            "google.com: type TXT, class IN": {
              "dns.qry.name_raw": ["0667676767676703636f6d00", 54, 12, 0, 26],
              "dns.qry.name.len_raw": ["06676f6f676c6503636f", 54, 10, 0, 5],
              "dns.count.labels_raw": ["06676f6f676c6503636f", 54, 10, 0, 5],
              "dns.qry.type_raw": ["0010", 66, 2, 0, 5],
              "dns.qry.class_raw": ["0001", 68, 2, 0, 5]
            }
          }
        }
      }
    }
}


4. json2pcap.py to generate pcap

The raw fields are flatten and by this the frame is created. The shortest raw fields overwrite the longer fields.

./wireshark/tools/json2pcap/json2pcap.py -p dns.cap.json


5. (OPTIONAL) json2pcap.py to generate python payload instead of pcap.

This is with relative positions to the parents. And the pcap reconstruct algorithm is different by using the parents and child hierarchy. This can be useful for possible latter more complex encoding (e.g. using scapy, dpkt, pycrate/libmich or other libraries).

./wireshark/tools/json2pcap/json2pcap.py -p dns.cap.json


vi dns.cap.json.py

...
def main():
    d = OrderedDict()
    d['frame_raw'] = [None, u'00c09f32418c00e018b10cad0800450000380000400040116547c0a8aa08c0a8aa14801b0035002485ed10320100000100000000000006676f6f676c6503636f6d0000100001', 0, 140, 0, 1]
    d['eth_raw'] = ['frame_raw', u'00c09f32418c00e018b10cad0800', 0, 28, 0, 1]
    d['ip_raw'] = ['frame_raw', u'450000380000400040116547c0a8aa08c0a8aa14', 14, 40, 0, 1]
    d['udp_raw'] = ['frame_raw', u'801b0035002485ed', 34, 16, 0, 1]
    d['dns_raw'] = ['frame_raw', u'10320100000100000000000006676f6f676c6503636f6d0000100001', 42, 56, 0, 1]
    d['dns_id_raw'] = [u'dns_raw', u'1032', 0, 4, 0, 5]
    d['dns_flags_raw'] = [u'dns_raw', u'0100', 2, 4, 0, 5]
    d['dns_flags_response_raw'] = [u'dns_flags_raw', u'0', 0, 1, 32768, 2]
    d['dns_flags_opcode_raw'] = [u'dns_flags_raw', u'0', 0, 1, 30720, 5]
    d['dns_flags_truncated_raw'] = [u'dns_flags_raw', u'0', 0, 1, 512, 2]
    d['dns_flags_recdesired_raw'] = [u'dns_flags_raw', u'1', 0, 1, 256, 2]
    d['dns_flags_z_raw'] = [u'dns_flags_raw', u'0', 0, 1, 64, 2]
    d['dns_flags_checkdisable_raw'] = [u'dns_flags_raw', u'0', 0, 1, 16, 2]
    d['dns_count_queries_raw'] = [u'dns_raw', u'0001', 4, 4, 0, 5]
    d['dns_count_answers_raw'] = [u'dns_raw', u'0000', 6, 4, 0, 5]
    d['dns_count_auth_rr_raw'] = [u'dns_raw', u'0000', 8, 4, 0, 5]
    d['dns_count_add_rr_raw'] = [u'dns_raw', u'0000', 10, 4, 0, 5]
    d['dns_qry_name_raw'] = [u'dns_raw', u'0667676767676703636f6d00', 12, 24, 0, 26]
    d['dns_qry_name_len_raw'] = [u'dns_raw', u'06676f6f676c6503636f', 12, 20, 0, 5]
    d['dns_count_labels_raw'] = [u'dns_raw', u'06676f6f676c6503636f', 12, 20, 0, 5]
    d['dns_qry_type_raw'] = [u'dns_raw', u'0010', 24, 4, 0, 5]
    d['dns_qry_class_raw'] = [u'dns_raw', u'0001', 26, 4, 0, 5]
    generate_pcap(d)
...


python dns.cap.json.py


5. New pcap dns.cap.json.pcap