{ "cells": [ { "cell_type": "code", "execution_count": 1, "metadata": { "nbsphinx": "hidden" }, "outputs": [], "source": [ "# Delete this cell to re-enable tracebacks\n", "import sys\n", "ipython = get_ipython()\n", "\n", "def hide_traceback(exc_tuple=None, filename=None, tb_offset=None,\n", " exception_only=False, running_compiled_code=False):\n", " etype, value, tb = sys.exc_info()\n", " value.__cause__ = None # suppress chained exceptions\n", " return ipython._showtraceback(etype, value, ipython.InteractiveTB.get_exception_only(etype, value))\n", "\n", "ipython.showtraceback = hide_traceback" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "nbsphinx": "hidden" }, "outputs": [], "source": [ "# JSON output syntax highlighting\n", "from __future__ import print_function\n", "from pygments import highlight\n", "from pygments.lexers import JsonLexer, TextLexer\n", "from pygments.formatters import HtmlFormatter\n", "from IPython.display import display, HTML\n", "from IPython.core.interactiveshell import InteractiveShell\n", "\n", "InteractiveShell.ast_node_interactivity = \"all\"\n", "\n", "def json_print(inpt):\n", " string = str(inpt)\n", " formatter = HtmlFormatter()\n", " if string[0] == '{':\n", " lexer = JsonLexer()\n", " else:\n", " lexer = TextLexer()\n", " return HTML('{}'.format(\n", " formatter.get_style_defs('.highlight'),\n", " highlight(string, lexer, formatter)))\n", "\n", "globals()['print'] = json_print" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Parsing STIX Content" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Parsing STIX content is as easy as calling the [parse()](../api/stix2.parsing.rst#stix2.parsing.parse) function on a JSON string, dictionary, or file-like object. It will automatically determine the type of the object. The STIX objects within `bundle` objects, and any cyber observables contained within `observed-data` objects will be parsed as well.\n", "\n", "**Parsing a string**" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
<class 'stix2.v21.sdo.ObservedData'>\n",
       "
\n" ], "text/plain": [ "" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" }, { "data": { "text/html": [ "
{\n",
       "    "type": "observed-data",\n",
       "    "spec_version": "2.1",\n",
       "    "id": "observed-data--b67d30ff-02ac-498a-92f9-32f845f448cf",\n",
       "    "created": "2016-04-06T19:58:16.000Z",\n",
       "    "modified": "2016-04-06T19:58:16.000Z",\n",
       "    "first_observed": "2015-12-21T19:00:00Z",\n",
       "    "last_observed": "2015-12-21T19:00:00Z",\n",
       "    "number_observed": 50,\n",
       "    "objects": {\n",
       "        "0": {\n",
       "            "type": "file",\n",
       "            "id": "file--5d0833b7-065e-571f-8bf2-657cb9569570",\n",
       "            "spec_version": "2.1",\n",
       "            "hashes": {\n",
       "                "SHA-256": "0969de02ecf8a5f003e3f6d063d848c8a193aada092623f8ce408c15bcb5f038"\n",
       "            }\n",
       "        }\n",
       "    }\n",
       "}\n",
       "
\n" ], "text/plain": [ "" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from stix2 import parse\n", "\n", "input_string = \"\"\"{\n", " \"type\": \"observed-data\",\n", " \"id\": \"observed-data--b67d30ff-02ac-498a-92f9-32f845f448cf\",\n", " \"spec_version\": \"2.1\",\n", " \"created\": \"2016-04-06T19:58:16.000Z\",\n", " \"modified\": \"2016-04-06T19:58:16.000Z\",\n", " \"first_observed\": \"2015-12-21T19:00:00Z\",\n", " \"last_observed\": \"2015-12-21T19:00:00Z\",\n", " \"number_observed\": 50,\n", " \"objects\": {\n", " \"0\": {\n", " \"type\": \"file\",\n", " \"hashes\": {\n", " \"SHA-256\": \"0969de02ecf8a5f003e3f6d063d848c8a193aada092623f8ce408c15bcb5f038\"\n", " }\n", " }\n", " }\n", "}\"\"\"\n", "\n", "obj = parse(input_string)\n", "print(type(obj))\n", "print(obj)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Parsing a dictionary**" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
<class 'stix2.v21.sdo.Identity'>\n",
       "
\n" ], "text/plain": [ "" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" }, { "data": { "text/html": [ "
{\n",
       "    "type": "identity",\n",
       "    "spec_version": "2.1",\n",
       "    "id": "identity--311b2d2d-f010-4473-83ec-1edf84858f4c",\n",
       "    "created": "2015-12-21T19:59:11.000Z",\n",
       "    "modified": "2015-12-21T19:59:11.000Z",\n",
       "    "name": "Cole Powers",\n",
       "    "identity_class": "individual"\n",
       "}\n",
       "
\n" ], "text/plain": [ "" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "input_dict = {\n", " \"type\": \"identity\",\n", " \"id\": \"identity--311b2d2d-f010-4473-83ec-1edf84858f4c\",\n", " \"spec_version\": \"2.1\",\n", " \"created\": \"2015-12-21T19:59:11Z\",\n", " \"modified\": \"2015-12-21T19:59:11Z\",\n", " \"name\": \"Cole Powers\",\n", " \"identity_class\": \"individual\"\n", "}\n", "\n", "obj = parse(input_dict)\n", "print(type(obj))\n", "print(obj)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Parsing a file-like object**" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
<class 'stix2.v21.sdo.CourseOfAction'>\n",
       "
\n" ], "text/plain": [ "" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" }, { "data": { "text/html": [ "
{\n",
       "    "type": "course-of-action",\n",
       "    "spec_version": "2.1",\n",
       "    "id": "course-of-action--d9727aee-48b8-4fdb-89e2-4c49746ba4dd",\n",
       "    "created_by_ref": "identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5",\n",
       "    "created": "2017-05-31T21:30:41.022Z",\n",
       "    "modified": "2017-05-31T21:30:41.022Z",\n",
       "    "name": "Data from Network Shared Drive Mitigation",\n",
       "    "description": "Identify unnecessary system utilities or potentially malicious software that may be used to collect data from a network share, and audit and/or block them by using whitelisting[[CiteRef::Beechey 2010]] tools, like AppLocker,[[CiteRef::Windows Commands JPCERT]][[CiteRef::NSA MS AppLocker]] or Software Restriction Policies[[CiteRef::Corio 2008]] where appropriate.[[CiteRef::TechNet Applocker vs SRP]]"\n",
       "}\n",
       "
\n" ], "text/plain": [ "" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "file_handle = open(\"/tmp/stix2_store/course-of-action/course-of-action--d9727aee-48b8-4fdb-89e2-4c49746ba4dd/20170531213041022744.json\")\n", "\n", "obj = parse(file_handle)\n", "print(type(obj))\n", "print(obj)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Parsing Custom STIX Content" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Parsing custom STIX objects and/or STIX objects with custom properties is also completed easily with [parse()](../api/stix2.parsing.rst#stix2.parsing.parse). Just supply the keyword argument ``allow_custom=True``. When ``allow_custom`` is specified, [parse()](../api/stix2.parsing.rst#stix2.parsing.parse) will attempt to convert the supplied STIX content to known STIX 2 domain objects and/or previously defined [custom STIX 2 objects](custom.ipynb). If the conversion cannot be completed (and ``allow_custom`` is specified), [parse()](../api/stix2.parsing.rst#stix2.parsing.parse) will treat the supplied STIX 2 content as valid STIX 2 objects and return them. This is an axiomatic possibility as the ``stix2`` library cannot guarantee proper processing of unknown custom STIX 2 objects that were explicitly flagged to be allowed, and thus may not be valid.\n", "
\n", "\n", "**Warning**\n", "\n", "Specifying allow_custom may lead to critical errors if further processing (searching, filtering, modifying etc...) of the custom content occurs where the custom content supplied is not valid STIX 2\n", "\n", "
\n", "\n", "For examples of parsing STIX 2 objects with custom STIX properties, see [Custom STIX Content: Custom Properties](custom.ipynb#Custom-Properties)\n", "\n", "For examples of parsing defined custom STIX 2 objects, see [Custom STIX Content: Custom STIX Object Types](custom.ipynb#Custom-STIX-Object-Types)\n", "\n", "For retrieving STIX 2 content from a source (e.g. file system, TAXII) that may possibly have custom STIX 2 content unknown to the user, the user can create a STIX 2 DataStore/Source with the flag ``allow_custom=True``. As mentioned, this will configure the DataStore/Source to allow for unknown STIX 2 content to be returned (albeit not converted to full STIX 2 domain objects and properties); the ``stix2`` library may preclude processing the unknown content, if the content is not valid or actual STIX 2 domain objects and properties." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "from taxii2client import Collection\n", "from stix2 import CompositeDataSource, FileSystemSource, TAXIICollectionSource\n", "\n", "# to allow for the retrieval of unknown custom STIX2 content,\n", "# just create *Stores/*Sources with the 'allow_custom' flag\n", "\n", "# create FileSystemStore\n", "fs = FileSystemSource(\"/path/to/stix2_data/\", allow_custom=True)\n", "\n", "# create TAXIICollectionSource\n", "colxn = Collection('http://taxii_url')\n", "ts = TAXIICollectionSource(colxn, allow_custom=True)" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.0a6" } }, "nbformat": 4, "nbformat_minor": 2 }