Welcome to stix2’s documentation!

Overview

Goals

High level goals/principles of the python-stix2 library:

1. It should be as easy as possible (but no easier!) to perform common tasks of producing, consuming, and processing STIX 2 content.
2. It should be hard, if not impossible, to emit invalid STIX 2.
3. The library should default to doing “the right thing”, complying with both the STIX 2.0 spec, as well as associated best practices. The library should make it hard to do “the wrong thing”.

Design Decisions

To accomplish these goals, and to incorporate lessons learned while developing python-stix (for STIX 1.x), several decisions influenced the design of python-stix2:

1. All data structures are immutable by default. In contrast to python-stix, where users would create an object and then assign attributes to it, in python-stix2 all properties must be provided when creating the object.
2. Where necessary, library objects should act like dict’s. When treated as a str, the JSON reprentation of the object should be used.
3. Core Python data types (including numeric types, datetime) should be used when appropriate, and serialized to the correct format in JSON as specified in the STIX 2.0 spec.

Architecture

The stix2 library APIs are divided into three logical layers, representing different levels of abstraction useful in different types of scripts and larger applications. It is possible to combine multiple layers in the same program, and the higher levels build on the layers below.

Object Layer

The lowest layer, Object Layer, is where Python objects representing STIX 2 data types (such as SDOs, SROs, and Cyber Observable Objects, as well as non-top-level objects like External References, Kill Chain phases, and Cyber Observable extensions) are created, and can be serialized and deserialized to and from JSON representation.

This layer is appropriate for stand-alone scripts that produce or consume STIX 2 content, or can serve as a low-level data API for larger applications that need to represent STIX objects as Python classes.

At this level, non-embedded reference properties (those ending in _ref, such as the links from a Relationship object to its source and target objects) are not implemented as references between the Python objects themselves, but by simply having the same values in id and reference properties. There is no referential integrity maintained by the stix2 library.

This layer is mostly complete.

Environment Layer

The Environment Layer adds several components that make it easier to handle STIX 2 data as part of a larger application and as part of a larger cyber threat intelligence ecosystem.

• Data Sources represent locations from which STIX data can be retrieved, such as a TAXII server, database, or local filesystem. The Data Source API abstracts differences between these storage location, giving a common API to get objects by ID or query by various properties, as well as allowing federated operations over multiple data sources.
• Similarly, Data Sink objects represent destinations for sending STIX data.
• An Object Factory provides a way to add common properties to all created objects (such as the same created_by_ref, or a StatementMarking with copyright information or terms of use for the STIX data).

Each of these components can be used individually, or combined as part of an Environment. These Environment objects allow different settings to be used by different users of a multi-user application (such as a web application).

This layer is mostly complete.

Workbench Layer

The highest layer of the stix2 APIs is the Workbench Layer, designed for a single user in a highly-interactive analytical environment (such as a Jupyter Notebook). It builds on the lower layers of the API, while hiding most of their complexity. Unlike the other layers, this layer is designed to be used directly by end users. For users who are comfortable with, Python, the Workbench Layer makes it easy to quickly interact with STIX data from a variety of sources without needing to write and run one-off Python scripts.

This layer is currently being developed.

User’s Guide

Creating STIX Content

Creating STIX Domain Objects

To create a STIX object, provide keyword arguments to the type’s constructor:

from stix2 import Indicator

indicator = Indicator(name="File hash for malware variant",
                      labels=["malicious-activity"],
                      pattern="[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']")
print(indicator)

{
    "type": "indicator",
    "id": "indicator--dbcbd659-c927-4f9a-994f-0a2632274394",
    "created": "2017-09-26T23:33:39.829Z",
    "modified": "2017-09-26T23:33:39.829Z",
    "labels": [
        "malicious-activity"
    ],
    "name": "File hash for malware variant",
    "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",
    "valid_from": "2017-09-26T23:33:39.829952Z"
}

Certain required attributes of all objects will be set automatically if not provided as keyword arguments:

• If not provided, type will be set automatically to the correct type. You can also provide the type explicitly, but this is not necessary:

indicator2 = Indicator(type='indicator',
                       labels=["malicious-activity"],
                       pattern="[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']")

Passing a value for type that does not match the class being constructed will cause an error:

indicator3 = Indicator(type='xxx',
                       labels=["malicious-activity"],
                       pattern="[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']")

InvalidValueError: Invalid value for Indicator 'type': must equal 'indicator'.

• If not provided, id will be generated randomly. If you provide an id argument, it must begin with the correct prefix:

indicator4 = Indicator(id="campaign--63ce9068-b5ab-47fa-a2cf-a602ea01f21a",
                       labels=["malicious-activity"],
                       pattern="[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']")

InvalidValueError: Invalid value for Indicator 'id': must start with 'indicator--'.

For indicators, labels and pattern are required and cannot be set automatically. Trying to create an indicator that is missing one of these properties will result in an error:

indicator = Indicator()

MissingPropertiesError: No values for required properties for Indicator: (labels, pattern).

However, the required valid_from attribute on Indicators will be set to the current time if not provided as a keyword argument.

Once created, the object acts like a frozen dictionary. Properties can be accessed using the standard Python dictionary syntax:

indicator['name']

u'File hash for malware variant'

Or access properties using the standard Python attribute syntax:

indicator.name

u'File hash for malware variant'

Attempting to modify any attributes will raise an error:

indicator['name'] = "This is a revised name"

TypeError: 'Indicator' object does not support item assignment

indicator.name = "This is a revised name"

ImmutableError: Cannot modify 'name' property in 'Indicator' after creation.

To update the properties of an object, see the Versioning section.

Creating a Malware object follows the same pattern:

from stix2 import Malware

malware = Malware(name="Poison Ivy",
                  labels=['remote-access-trojan'])
print(malware)

{
    "type": "malware",
    "id": "malware--d7fd675d-94eb-4d95-b0bc-b3c5e28e8ed2",
    "created": "2017-09-26T23:33:56.908Z",
    "modified": "2017-09-26T23:33:56.908Z",
    "name": "Poison Ivy",
    "labels": [
        "remote-access-trojan"
    ]
}

As with indicators, the type, id, created, and modified properties will be set automatically if not provided. For Malware objects, the labels and name properties must be provided.

You can see the full list of SDO classes here.

Creating Relationships

STIX 2 Relationships are separate objects, not properties of the object on either side of the relationship. They are constructed similarly to other STIX objects. The type, id, created, and modified properties are added automatically if not provided. Callers must provide the relationship_type, source_ref, and target_ref properties.

from stix2 import Relationship

relationship = Relationship(relationship_type='indicates',
                            source_ref=indicator.id,
                            target_ref=malware.id)
print(relationship)

{
    "type": "relationship",
    "id": "relationship--637aa3b1-d4b8-4bc4-85e7-77cc82b198a3",
    "created": "2017-09-26T23:34:01.765Z",
    "modified": "2017-09-26T23:34:01.765Z",
    "relationship_type": "indicates",
    "source_ref": "indicator--dbcbd659-c927-4f9a-994f-0a2632274394",
    "target_ref": "malware--d7fd675d-94eb-4d95-b0bc-b3c5e28e8ed2"
}

The source_ref and target_ref properties can be either the ID’s of other STIX objects, or the STIX objects themselves. For readability, Relationship objects can also be constructed with the source_ref, relationship_type, and target_ref as positional (non-keyword) arguments:

relationship2 = Relationship(indicator, 'indicates', malware)
print(relationship2)

{
    "type": "relationship",
    "id": "relationship--70fe77c2-ab00-4181-a2dc-fe5567d971ca",
    "created": "2017-09-26T23:34:03.923Z",
    "modified": "2017-09-26T23:34:03.923Z",
    "relationship_type": "indicates",
    "source_ref": "indicator--dbcbd659-c927-4f9a-994f-0a2632274394",
    "target_ref": "malware--d7fd675d-94eb-4d95-b0bc-b3c5e28e8ed2"
}

Creating Bundles

STIX Bundles can be created by passing objects as arguments to the Bundle constructor. All required properties (type, id, and spec_version) will be set automatically if not provided, or can be provided as keyword arguments:

from stix2 import Bundle

bundle = Bundle(indicator, malware, relationship)
print(bundle)

{
    "type": "bundle",
    "id": "bundle--2536c43d-c874-418e-886c-20a22120d8cb",
    "spec_version": "2.0",
    "objects": [
        {
            "type": "indicator",
            "id": "indicator--dbcbd659-c927-4f9a-994f-0a2632274394",
            "created": "2017-09-26T23:33:39.829Z",
            "modified": "2017-09-26T23:33:39.829Z",
            "labels": [
                "malicious-activity"
            ],
            "name": "File hash for malware variant",
            "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",
            "valid_from": "2017-09-26T23:33:39.829952Z"
        },
        {
            "type": "malware",
            "id": "malware--d7fd675d-94eb-4d95-b0bc-b3c5e28e8ed2",
            "created": "2017-09-26T23:33:56.908Z",
            "modified": "2017-09-26T23:33:56.908Z",
            "name": "Poison Ivy",
            "labels": [
                "remote-access-trojan"
            ]
        },
        {
            "type": "relationship",
            "id": "relationship--637aa3b1-d4b8-4bc4-85e7-77cc82b198a3",
            "created": "2017-09-26T23:34:01.765Z",
            "modified": "2017-09-26T23:34:01.765Z",
            "relationship_type": "indicates",
            "source_ref": "indicator--dbcbd659-c927-4f9a-994f-0a2632274394",
            "target_ref": "malware--d7fd675d-94eb-4d95-b0bc-b3c5e28e8ed2"
        }
    ]
}

Custom STIX Content

Custom Properties

Attempting to create a STIX object with properties not defined by the specification will result in an error. Try creating an Identity object with a custom x_foo property:

from stix2 import Identity

Identity(name="John Smith",
         identity_class="individual",
         x_foo="bar")

ExtraPropertiesError: Unexpected properties for Identity: (x_foo).

To create a STIX object with one or more custom properties, pass them in as a dictionary parameter called custom_properties:

from stix2 import Identity

identity = Identity(name="John Smith",
                    identity_class="individual",
                    custom_properties={
                        "x_foo": "bar"
                    })
print(identity)

{
    "type": "identity",
    "id": "identity--00c5743f-2d5e-4d66-88f1-1842584f4519",
    "created": "2017-11-09T16:17:44.596Z",
    "modified": "2017-11-09T16:17:44.596Z",
    "name": "John Smith",
    "identity_class": "individual",
    "x_foo": "bar"
}

Alternatively, setting allow_custom to True will allow custom properties without requiring a custom_properties dictionary.

identity2 = Identity(name="John Smith",
                     identity_class="individual",
                     x_foo="bar",
                     allow_custom=True)
print(identity2)

{
    "x_foo": "bar",
    "type": "identity",
    "id": "identity--1e8188eb-245f-400b-839d-7f612169c514",
    "created": "2017-09-26T21:02:22.708Z",
    "modified": "2017-09-26T21:02:22.708Z",
    "name": "John Smith",
    "identity_class": "individual"
}

Likewise, when parsing STIX content with custom properties, pass allow_custom=True to parse():

from stix2 import parse

input_string = """{
    "type": "identity",
    "id": "identity--311b2d2d-f010-5473-83ec-1edf84858f4c",
    "created": "2015-12-21T19:59:11Z",
    "modified": "2015-12-21T19:59:11Z",
    "name": "John Smith",
    "identity_class": "individual",
    "x_foo": "bar"
}"""
identity3 = parse(input_string, allow_custom=True)
print(identity3.x_foo)

bar

Custom STIX Object Types

To create a custom STIX object type, define a class with the @CustomObject decorator. It takes the type name and a list of property tuples, each tuple consisting of the property name and a property instance. Any special validation of the properties can be added by supplying an __init__ function.

Let’s say zoo animals have become a serious cyber threat and we want to model them in STIX using a custom object type. Let’s use a species property to store the kind of animal, and make that property required. We also want a property to store the class of animal, such as “mammal” or “bird” but only want to allow specific values in it. We can add some logic to validate this property in __init__.

from stix2 import CustomObject, properties

@CustomObject('x-animal', [
    ('species', properties.StringProperty(required=True)),
    ('animal_class', properties.StringProperty()),
])
class Animal(object):
    def __init__(self, animal_class=None, **kwargs):
        if animal_class and animal_class not in ['mammal', 'bird', 'fish', 'reptile']:
            raise ValueError("'%s' is not a recognized class of animal." % animal_class)

Now we can create an instance of our custom Animal type.

animal = Animal(species="lion",
                animal_class="mammal")
print(animal)

{
    "type": "x-animal",
    "id": "x-animal--caebdf17-9d2a-4c84-8864-7406326618f0",
    "created": "2017-09-26T21:02:34.724Z",
    "modified": "2017-09-26T21:02:34.724Z",
    "species": "lion",
    "animal_class": "mammal"
}

Trying to create an Animal instance with an animal_class that’s not in the list will result in an error:

Animal(species="xenomorph",
       animal_class="alien")

ValueError: 'alien' is not a recognized class of animal.

Parsing custom object types that you have already defined is simple and no different from parsing any other STIX object.

input_string2 = """{
    "type": "x-animal",
    "id": "x-animal--941f1471-6815-456b-89b8-7051ddf13e4b",
    "created": "2015-12-21T19:59:11Z",
    "modified": "2015-12-21T19:59:11Z",
    "species": "shark",
    "animal_class": "fish"
}"""
animal2 = parse(input_string2)
print(animal2.species)

shark

However, parsing custom object types which you have not defined will result in an error:

input_string3 = """{
    "type": "x-foobar",
    "id": "x-foobar--d362beb5-a04e-4e6b-a030-b6935122c3f9",
    "created": "2015-12-21T19:59:11Z",
    "modified": "2015-12-21T19:59:11Z",
    "bar": 1,
    "baz": "frob"
}"""
parse(input_string3)

ParseError: Can't parse unknown object type 'x-foobar'! For custom types, use the CustomObject decorator.

Custom Cyber Observable Types

Similar to custom STIX object types, use a decorator to create custom Cyber Observable types. Just as before, __init__() can hold additional validation, but it is not necessary.

from stix2 import CustomObservable

@CustomObservable('x-new-observable', [
    ('a_property', properties.StringProperty(required=True)),
    ('property_2', properties.IntegerProperty()),
])
class NewObservable():
    pass

new_observable = NewObservable(a_property="something",
                               property_2=10)
print(new_observable)

{
    "type": "x-new-observable",
    "a_property": "something",
    "property_2": 10
}

Likewise, after the custom Cyber Observable type has been defined, it can be parsed.

from stix2 import ObservedData

input_string4 = """{
    "type": "observed-data",
    "id": "observed-data--b67d30ff-02ac-498a-92f9-32f845f448cf",
    "created_by_ref": "identity--f431f809-377b-45e0-aa1c-6a4751cae5ff",
    "created": "2016-04-06T19:58:16.000Z",
    "modified": "2016-04-06T19:58:16.000Z",
    "first_observed": "2015-12-21T19:00:00Z",
    "last_observed": "2015-12-21T19:00:00Z",
    "number_observed": 50,
    "objects": {
        "0": {
            "type": "x-new-observable",
            "a_property": "foobaz",
            "property_2": 5
        }
    }
}"""
obs_data = parse(input_string4)
print(obs_data.objects["0"].a_property)
print(obs_data.objects["0"].property_2)

foobaz
5

Custom Cyber Observable Extensions

Finally, custom extensions to existing Cyber Observable types can also be created. Just use the @CustomExtension decorator. Note that you must provide the Cyber Observable class to which the extension applies. Again, any extra validation of the properties can be implemented by providing an __init__() but it is not required. Let’s say we want to make an extension to the File Cyber Observable Object:

from stix2 import File, CustomExtension

@CustomExtension(File, 'x-new-ext', [
    ('property1', properties.StringProperty(required=True)),
    ('property2', properties.IntegerProperty()),
])
class NewExtension():
    pass

new_ext = NewExtension(property1="something",
                       property2=10)
print(new_ext)

{
    "property1": "something",
    "property2": 10
}

Once the custom Cyber Observable extension has been defined, it can be parsed.

input_string5 = """{
    "type": "observed-data",
    "id": "observed-data--b67d30ff-02ac-498a-92f9-32f845f448cf",
    "created_by_ref": "identity--f431f809-377b-45e0-aa1c-6a4751cae5ff",
    "created": "2016-04-06T19:58:16.000Z",
    "modified": "2016-04-06T19:58:16.000Z",
    "first_observed": "2015-12-21T19:00:00Z",
    "last_observed": "2015-12-21T19:00:00Z",
    "number_observed": 50,
    "objects": {
        "0": {
            "type": "file",
            "name": "foo.bar",
            "hashes": {
                "SHA-256": "35a01331e9ad96f751278b891b6ea09699806faedfa237d40513d92ad1b7100f"
            },
            "extensions": {
                "x-new-ext": {
                    "property1": "bla",
                    "property2": 50
                }
            }
        }
    }
}"""
obs_data2 = parse(input_string5)
print(obs_data2.objects["0"].extensions["x-new-ext"].property1)
print(obs_data2.objects["0"].extensions["x-new-ext"].property2)

bla
50

# without this configuration, only last print() call is outputted in cells
from IPython.core.interactiveshell import InteractiveShell
InteractiveShell.ast_node_interactivity = "all"

DataStore API

CTI Python STIX2 features a new interface for pulling and pushing STIX2 content. The new interface consists of DataStore, DataSource and DataSink constructs: a DataSource for pulling STIX2 content, a DataSink for pushing STIX2 content, and a DataStore for both pulling and pushing.

The DataStore, DataSource, DataSink (collectively referred to as the “DataStore suite”) APIs are not referenced directly by a user but are used as base classes, which are then subclassed by real DataStore suites. CTI Python STIX2 provides the DataStore suites of FileSystem, Memory, and TAXII. Users are also encouraged to subclass the base classes and create their own custom DataStore suites.

CompositeDataSource

CompositeDataSource is an available controller that can be used as a single interface to a set of defined DataSources. The purpose of this controller is allow for the grouping of DataSources and making get()/query() calls to a set of DataSources in one API call. CompositeDataSources can be used to organize/group DataSources, federate get()/all_versions()/query() calls, and reduce user code.

CompositeDataSource is just a wrapper around a set of defined DataSources (e.g. FileSystemSource) that federates get()/all_versions()/query() calls individually to each of the attached DataSources , collects the results from each DataSource and returns them.

Filters can be attached to CompositeDataSources just as they can be done to DataStores and DataSources. When get()/all_versions()/query() calls are made to the CompositeDataSource, it will pass along any query filters from the call and any of its own filters to the attached DataSources. In addition, those DataSources may have their own attached filters as well. The effect is that all the filters are eventually combined when the get()/all_versions()/query() call is actually executed within a DataSource.

A CompositeDataSource can also be attached to a CompositeDataSource for multiple layers of grouped DataSources.

CompositeDataSource API

CompositeDataSource Examples

from taxii2client import Collection
from stix2 import CompositeDataSource, FileSystemSource, TAXIICollectionSource

# create FileSystemStore
fs = FileSystemSource("/home/michael/cti-python-stix2/stix2/test/stix2_data/")

# create TAXIICollectionSource
colxn = Collection('https://test.freetaxii.com:8000/osint/collections/a9c22eaf-0f3e-482c-8bb4-45ae09e75d9b/')
ts = TAXIICollectionSource(colxn)

# add them both to the CompositeDataSource
cs = CompositeDataSource()
cs.add_data_sources([fs,ts])

# get an object that is only in the filesystem
intrusion_set = cs.get('intrusion-set--f3bdec95-3d62-42d9-a840-29630f6cdc1a')
print(intrusion_set)

# get an object that is only in the TAXII collection
ind = cs.get('indicator--02b90f02-a96a-43ee-88f1-1e87297941f2')
print(ind)

{
    "type": "intrusion-set",
    "id": "intrusion-set--f3bdec95-3d62-42d9-a840-29630f6cdc1a",
    "created_by_ref": "identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5",
    "created": "2017-05-31T21:31:53.197Z",
    "modified": "2017-05-31T21:31:53.197Z",
    "name": "DragonOK",
    "description": "DragonOK is a threat group that has targeted Japanese organizations with phishing emails. Due to overlapping TTPs, including similar custom tools, DragonOK is thought to have a direct or indirect relationship with the threat group Moafee. [[Citation: Operation Quantum Entanglement]][[Citation: Symbiotic APT Groups]] It is known to use a variety of malware, including Sysget/HelloBridge, PlugX, PoisonIvy, FormerFirstRat, NFlog, and NewCT. [[Citation: New DragonOK]]",
    "aliases": [
        "DragonOK"
    ],
    "external_references": [
        {
            "source_name": "mitre-attack",
            "url": "https://attack.mitre.org/wiki/Group/G0017",
            "external_id": "G0017"
        },
        {
            "source_name": "Operation Quantum Entanglement",
            "description": "Haq, T., Moran, N., Vashisht, S., Scott, M. (2014, September). OPERATION QUANTUM ENTANGLEMENT. Retrieved November 4, 2015.",
            "url": "https://www.fireeye.com/content/dam/fireeye-www/global/en/current-threats/pdfs/wp-operation-quantum-entanglement.pdf"
        },
        {
            "source_name": "Symbiotic APT Groups",
            "description": "Haq, T. (2014, October). An Insight into Symbiotic APT Groups. Retrieved November 4, 2015.",
            "url": "https://dl.mandiant.com/EE/library/MIRcon2014/MIRcon%202014%20R&D%20Track%20Insight%20into%20Symbiotic%20APT.pdf"
        },
        {
            "source_name": "New DragonOK",
            "description": "Miller-Osborn, J., Grunzweig, J.. (2015, April). Unit 42 Identifies New DragonOK Backdoor Malware Deployed Against Japanese Targets. Retrieved November 4, 2015.",
            "url": "http://researchcenter.paloaltonetworks.com/2015/04/unit-42-identifies-new-dragonok-backdoor-malware-deployed-against-japanese-targets/"
        }
    ],
    "object_marking_refs": [
        "marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168"
    ]
}

{
    "type": "indicator",
    "id": "indicator--02b90f02-a96a-43ee-88f1-1e87297941f2",
    "created": "2017-11-13T07:00:24.000Z",
    "modified": "2017-11-13T07:00:24.000Z",
    "name": "Ransomware IP Blocklist",
    "description": "IP Blocklist address from abuse.ch",
    "pattern": "[ ipv4-addr:value = '91.237.247.24' ]",
    "valid_from": "2017-11-13T07:00:24Z",
    "labels": [
        "malicious-activity",
        "Ransomware",
        "Botnet",
        "C&C"
    ],
    "external_references": [
        {
            "source_name": "abuse.ch",
            "url": "https://ransomwaretracker.abuse.ch/blocklist/"
        }
    ]
}

Filters

The CTI Python STIX2 DataStore suites - FileSystem, Memory, and TAXII - all use the Filters module to allow for the querying of STIX content. The basic functionality is that filters can be created and supplied everytime to calls to query(), and/or attached to a DataStore so that every future query placed to that DataStore is evaluated against the attached filters, supplemented with any further filters supplied with the query call. Attached filters can also be removed from DataStores.

Filters are very simple, as they consist of a field name, comparison operator and an object property value (i.e. value to compare to). All properties of STIX2 objects can be filtered on. In addition, TAXII2 Filtering parameters for fields can also be used in filters.

TAXII2 filter fields:

• added_after
• match[id]
• match[type]
• match[version]

Supported operators:

• =
• !=
• in
• <
• >=
• <=

Value types of the property values must be one of these (Python) types:

• bool
• dict
• float
• int
• list
• str
• tuple

Filter Examples

import sys
from stix2 import Filter

# create filter for STIX objects that have external references to MITRE ATT&CK framework
f = Filter("external_references.source_name", "=", "mitre-attack")

# create filter for STIX objects that are not of SDO type Attack-Pattnern
f1 = Filter("type", "!=", "attack-pattern")

# create filter for STIX objects that have the "threat-report" label
f2 = Filter("labels", "in", "threat-report")

# create filter for STIX objects that have been modified past the timestamp
f3 = Filter("modified", ">=", "2017-01-28T21:33:10.772474Z")

# create filter for STIX objects that have been revoked
f4 = Filter("revoked", "=", True)

For Filters to be applied to a query, they must be either supplied with the query call or attached to a DataStore, more specifically to a DataSource whether that DataSource is a part of a DataStore or stands by itself.

from stix2 import MemoryStore, FileSystemStore, FileSystemSource

fs = FileSystemStore("/home/michael/Desktop/sample_stix2_data")
fs_source = FileSystemSource("/home/michael/Desktop/sample_stix2_data")

# attach filter to FileSystemStore
fs.source.filters.add(f)

# attach multiple filters to FileSystemStore
fs.source.filters.update([f1,f2])

# can also attach filters to a Source
# attach multiple filters to FileSystemSource
fs_source.filters.update([f3, f4])


mem = MemoryStore()

# As it is impractical to only use MemorySink or MemorySource,
# attach a filter to a MemoryStore
mem.source.filters.add(f)

# attach multiple filters to a MemoryStore
mem.source.filters.update([f1,f2])

De-Referencing Relationships

Given a STIX object, there are several ways to find other STIX objects related to it. To illustrate this, let’s first create a DataStore and add some objects and relationships.

from stix2 import Campaign, Identity, Indicator, Malware, Relationship

mem = MemoryStore()
cam = Campaign(name='Charge', description='Attack!')
idy = Identity(name='John Doe', identity_class="individual")
ind = Indicator(labels=['malicious-activity'], pattern="[file:hashes.MD5 = 'd41d8cd98f00b204e9800998ecf8427e']")
mal = Malware(labels=['ransomware'], name="Cryptolocker", created_by_ref=idy)
rel1 = Relationship(ind, 'indicates', mal,)
rel2 = Relationship(mal, 'targets', idy)
rel3 = Relationship(cam, 'uses', mal)
mem.add([cam, idy, ind, mal, rel1, rel2, rel3])

If a STIX object has a created_by_ref property, you can use the creator_of() method to retrieve the Identity object that created it.

print(mem.creator_of(mal))

{
    "type": "identity",
    "id": "identity--be3baac0-9aba-48a8-81e4-4408b1c379a8",
    "created": "2017-11-21T22:14:45.213Z",
    "modified": "2017-11-21T22:14:45.213Z",
    "name": "John Doe",
    "identity_class": "individual"
}

Use the relationships() method to retrieve all the relationship objects that reference a STIX object.

rels = mem.relationships(mal)
len(rels)

3

You can limit it to only specific relationship types:

mem.relationships(mal, relationship_type='indicates')

[Relationship(type='relationship', id='relationship--bd6fd399-c907-4feb-b1da-b90f15942f1d', created='2017-11-21T22:14:45.214Z', modified='2017-11-21T22:14:45.214Z', relationship_type=u'indicates', source_ref='indicator--5ee33ff0-c50d-456b-a8dd-b5d1b69a66e8', target_ref='malware--66c0bc78-4e27-4d80-a565-a07e6eb6fba4')]

You can limit it to only relationships where the given object is the source:

mem.relationships(mal, source_only=True)

[Relationship(type='relationship', id='relationship--7eb7f5cd-8bf2-4f7c-8756-84c0b5693b9a', created='2017-11-21T22:14:45.215Z', modified='2017-11-21T22:14:45.215Z', relationship_type=u'targets', source_ref='malware--66c0bc78-4e27-4d80-a565-a07e6eb6fba4', target_ref='identity--be3baac0-9aba-48a8-81e4-4408b1c379a8')]

And you can limit it to only relationships where the given object is the target:

mem.relationships(mal, target_only=True)

[Relationship(type='relationship', id='relationship--bd6fd399-c907-4feb-b1da-b90f15942f1d', created='2017-11-21T22:14:45.214Z', modified='2017-11-21T22:14:45.214Z', relationship_type=u'indicates', source_ref='indicator--5ee33ff0-c50d-456b-a8dd-b5d1b69a66e8', target_ref='malware--66c0bc78-4e27-4d80-a565-a07e6eb6fba4'),
 Relationship(type='relationship', id='relationship--3c759d40-c92a-430e-aab6-77d5c5763302', created='2017-11-21T22:14:45.215Z', modified='2017-11-21T22:14:45.215Z', relationship_type=u'uses', source_ref='campaign--82ab7aa4-d13b-4e99-8a09-ebcba30668a7', target_ref='malware--66c0bc78-4e27-4d80-a565-a07e6eb6fba4')]

Finally, you can retrieve all STIX objects related to a given STIX object using related_to(). This calls relationships() but then performs the extra step of getting the objects that these Relationships point to. related_to() takes all the same arguments that relationships() does.

mem.related_to(mal, target_only=True, relationship_type='uses')

[Campaign(type='campaign', id='campaign--82ab7aa4-d13b-4e99-8a09-ebcba30668a7', created='2017-11-21T22:14:45.213Z', modified='2017-11-21T22:14:45.213Z', name=u'Charge', description=u'Attack!')]

Using Environments

An Environment object makes it easier to use STIX 2 content as part of a larger application or ecosystem. It allows you to abstract away the nasty details of sending and receiving STIX data, and to create STIX objects with default values for common properties.

Storing and Retrieving STIX Content

An Environment can be set up with a DataStore if you want to store and retrieve STIX content from the same place.

from stix2 import Environment, MemoryStore

env = Environment(store=MemoryStore())

If desired, you can instead set up an Environment with different data sources and sinks. In the following example we set up an environment that retrieves objects from memory and a directory on the filesystem, and stores objects in a different directory on the filesystem.

from stix2 import CompositeDataSource, FileSystemSink, FileSystemSource, MemorySource

src = CompositeDataSource()
src.add_data_sources([MemorySource(), FileSystemSource("/tmp/stix_source")])
env2 = Environment(source=src,
                   sink=FileSystemSink("/tmp/stix_sink"))

Once you have an Environment you can store some STIX content in its DataSinks with add():

from stix2 import Indicator

indicator = Indicator(id="indicator--01234567-89ab-cdef-0123-456789abcdef",
                      labels=["malicious-activity"],
                      pattern="[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']")
env.add(indicator)

You can retrieve STIX objects from the DataSources in the Environment with get(), query(), all_versions(), creator_of(), related_to(), and relationships() just as you would for a DataSource.

print(env.get("indicator--01234567-89ab-cdef-0123-456789abcdef"))

{
    "type": "indicator",
    "id": "indicator--01234567-89ab-cdef-0123-456789abcdef",
    "created": "2017-10-02T13:20:39.373Z",
    "modified": "2017-10-02T13:20:39.373Z",
    "labels": [
        "malicious-activity"
    ],
    "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",
    "valid_from": "2017-10-02T13:20:39.3737Z"
}

Creating STIX Objects With Defaults

To create STIX objects with default values for certain properties, use an ObjectFactory. For instance, say we want all objects we create to have a created_by_ref property pointing to the Identity object representing our organization.

from stix2 import Indicator, ObjectFactory

factory = ObjectFactory(created_by_ref="identity--311b2d2d-f010-5473-83ec-1edf84858f4c")

Once you’ve set up the ObjectFactory, use its create() method, passing in the class for the type of object you wish to create, followed by the other properties and their values for the object.

ind = factory.create(Indicator,
                     labels=["malicious-activity"],
                     pattern="[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']")
print(ind)

{
    "type": "indicator",
    "id": "indicator--c92ad60d-449d-4adf-86b3-4e5951a8f480",
    "created_by_ref": "identity--311b2d2d-f010-5473-83ec-1edf84858f4c",
    "created": "2017-10-02T13:23:00.607Z",
    "modified": "2017-10-02T13:23:00.607Z",
    "labels": [
        "malicious-activity"
    ],
    "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",
    "valid_from": "2017-10-02T13:23:00.607216Z"
}

All objects we create with that ObjectFactory will automatically get the default value for created_by_ref. These are the properties for which defaults can be set:

• created_by_ref
• created
• external_references
• object_marking_refs

These defaults can be bypassed. For example, say you have an Environment with multiple default values but want to create an object with a different value for created_by_ref, or none at all.

factory2 = ObjectFactory(created_by_ref="identity--311b2d2d-f010-5473-83ec-1edf84858f4c",
                         created="2017-09-25T18:07:46.255472Z")
env2 = Environment(factory=factory2)

ind2 = env2.create(Indicator,
                   created_by_ref=None,
                   labels=["malicious-activity"],
                   pattern="[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']")
print(ind2)

{
    "type": "indicator",
    "id": "indicator--ae206b9f-8723-4fcf-beb7-8b1b9a2570ab",
    "created": "2017-09-25T18:07:46.255Z",
    "modified": "2017-09-25T18:07:46.255Z",
    "labels": [
        "malicious-activity"
    ],
    "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",
    "valid_from": "2017-10-02T13:23:05.790562Z"
}

ind3 = env2.create(Indicator,
                       created_by_ref="identity--962cabe5-f7f3-438a-9169-585a8c971d12",
                       labels=["malicious-activity"],
                       pattern="[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']")
print(ind3)

{
    "type": "indicator",
    "id": "indicator--a8e2be68-b496-463f-9ff4-f620046e7cf2",
    "created_by_ref": "identity--962cabe5-f7f3-438a-9169-585a8c971d12",
    "created": "2017-09-25T18:07:46.255Z",
    "modified": "2017-09-25T18:07:46.255Z",
    "labels": [
        "malicious-activity"
    ],
    "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",
    "valid_from": "2017-10-02T13:23:08.32424Z"
}

For the full power of the Environment layer, create an Environment with both a DataStore/Source/Sink and an ObjectFactory:

environ = Environment(ObjectFactory(created_by_ref="identity--311b2d2d-f010-5473-83ec-1edf84858f4c"),
                      MemoryStore())

i = environ.create(Indicator,
                   labels=["malicious-activity"],
                   pattern="[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']")
environ.add(i)
print(environ.get(i.id))

{
    "type": "indicator",
    "id": "indicator--89ba04ea-cce9-47a3-acd3-b6379ce51581",
    "created_by_ref": "identity--311b2d2d-f010-5473-83ec-1edf84858f4c",
    "created": "2017-10-02T13:23:29.629Z",
    "modified": "2017-10-02T13:23:29.629Z",
    "labels": [
        "malicious-activity"
    ],
    "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",
    "valid_from": "2017-10-02T13:23:29.629857Z"
}

FileSystem

The FileSystem suite contains FileSystemStore, FileSystemSource and FileSystemSink. Under the hood, all FileSystem objects point to a file directory (on disk) that contains STIX2 content.

The directory and file structure of the intended STIX2 content should be:

stix2_content/
    /STIX2 Domain Object type
        STIX2 Domain Object
        STIX2 Domain Object
            .
            .
            .
    /STIX2 Domain Object type
        STIX2 Domain Object
        STIX2 Domain Object
            .
            .
            .
        .
        .
        .
    /STIX2 Domain Object type

The master STIX2 content directory contains subdirectories, each of which aligns to a STIX2 domain object type (i.e. “attack-pattern”, “campaign”, “malware”, etc.). Within each STIX2 domain object subdirectory are JSON files that are STIX2 domain objects of the specified type. The name of the json files correspond to the ID of the STIX2 domain object found within that file. A real example of the FileSystem directory structure:

stix2_content/
    /attack-pattern
        attack-pattern--00d0b012-8a03-410e-95de-5826bf542de6.json
        attack-pattern--0a3ead4e-6d47-4ccb-854c-a6a4f9d96b22.json
        attack-pattern--1b7ba276-eedc-4951-a762-0ceea2c030ec.json
    /campaign
    /course-of-action
        course-of-action--2a8de25c-f743-4348-b101-3ee33ab5871b.json
        course-of-action--2c3ce852-06a2-40ee-8fe6-086f6402a739.json
    /identity
        identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5.json
    /indicator
    /intrusion-set
    /malware
        malware--1d808f62-cf63-4063-9727-ff6132514c22.json
        malware--2eb9b131-d333-4a48-9eb4-d8dec46c19ee.json
    /observed-data
    /report
    /threat-actor
    /vulnerability

FileSystemStore is intended for use cases where STIX2 content is retrieved and pushed to the same file directory. As FileSystemStore is just a wrapper around a paired FileSystemSource and FileSystemSink that point the same file directory.

For use cases where STIX2 content will only be retrieved or pushed, then a FileSystemSource and FileSystemSink can be used individually. They can also be used individually when STIX2 content will be retrieved from one distinct file directory and pushed to another.

FileSystem API

A note on get(), all_versions(), and query(): The format of the STIX2 content targeted by the FileSystem suite is JSON files. When the FileSystemStore retrieves STIX2 content (in JSON) from disk, it will attempt to parse the content into full-featured python-stix2 objects and returned as such.

A note on add(): When STIX content is added (pushed) to the file system, the STIX content can be supplied in the following forms: Python STIX objects, Python dictionaries (of valid STIX objects or Bundles), JSON-encoded strings (of valid STIX objects or Bundles), or a (Python) list of any of the previously listed types. Any of the previous STIX content forms will be converted to a STIX JSON object (in a STIX Bundle) and written to disk.

FileSystem Examples

FileSystemStore

from stix2 import FileSystemStore

"""
Working with the FileSystemStore, where STIX content can be retrieved and pushed to a file system.
"""

# create FileSystemStore
fs = FileSystemStore("/home/michael/Desktop/sample_stix2_data")

# retrieve STIX2 content from FileSystemStore
ap = fs.get("attack-pattern--00d0b012-8a03-410e-95de-5826bf542de6")
mal = fs.get("malware--00c3bfcb-99bd-4767-8c03-b08f585f5c8a")

# for visual purposes
print(mal)

{
    "type": "malware",
    "id": "malware--00c3bfcb-99bd-4767-8c03-b08f585f5c8a",
    "created_by_ref": "identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5",
    "created": "2017-05-31T21:33:19.746Z",
    "modified": "2017-05-31T21:33:19.746Z",
    "name": "PowerDuke",
    "description": "PowerDuke is a backdoor that was used by APT29 in 2016. It has primarily been delivered through Microsoft Word or Excel attachments containing malicious macros.[[Citation: Volexity PowerDuke November 2016]]",
    "labels": [
        "malware"
    ],
    "external_references": [
        {
            "source_name": "mitre-attack",
            "url": "https://attack.mitre.org/wiki/Software/S0139",
            "external_id": "S0139"
        },
        {
            "source_name": "Volexity PowerDuke November 2016",
            "description": "Adair, S.. (2016, November 9). PowerDuke: Widespread Post-Election Spear Phishing Campaigns Targeting Think Tanks and NGOs. Retrieved January 11, 2017.",
            "url": "https://www.volexity.com/blog/2016/11/09/powerduke-post-election-spear-phishing-campaigns-targeting-think-tanks-and-ngos/"
        }
    ],
    "object_marking_refs": [
        "marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168"
    ]
}

from stix2 import ThreatActor, Indicator

# create new STIX threat-actor
ta = ThreatActor(name="Adjective Bear",
                labels=["nation-state"],
                sophistication="innovator",
                resource_level="government",
                goals=[
                    "compromising media outlets",
                    "water-hole attacks geared towards political, military targets",
                    "intelligence collection"
                ])

# create new indicators
ind = Indicator(description="Crusades C2 implant",
                labels=["malicious-activity"],
                pattern="[file:hashes.'SHA-256' = '54b7e05e39a59428743635242e4a867c932140a999f52a1e54fa7ee6a440c73b']")

ind1 = Indicator(description="Crusades C2 implant 2",
                 labels=["malicious-activity"],
                 pattern="[file:hashes.'SHA-256' = '64c7e05e40a59511743635242e4a867c932140a999f52a1e54fa7ee6a440c73b']")

# add STIX object (threat-actor) to FileSystemStore
fs.add(ta)

# can also add multiple STIX objects to FileSystemStore in one call
fs.add([ind, ind1])

FileSystemSource - (if STIX content is only to be retrieved from FileSystem)

from stix2 import FileSystemSource
"""
Working with FileSystemSource for retrieving STIX content.
"""

# create FileSystemSource
fs_source = FileSystemSource("/home/michael/Desktop/sample_stix2_data")

# retrieve STIX 2 objects
ap = fs_source.get("attack-pattern--00d0b012-8a03-410e-95de-5826bf542de6")

# for visual purposes
print(ap)

{
    "type": "attack-pattern",
    "id": "attack-pattern--00d0b012-8a03-410e-95de-5826bf542de6",
    "created_by_ref": "identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5",
    "created": "2017-05-31T21:30:54.176Z",
    "modified": "2017-05-31T21:30:54.176Z",
    "name": "Indicator Removal from Tools",
    "description": "If a malicious...command-line parameters, Process monitoring",
    "kill_chain_phases": [
        {
            "kill_chain_name": "mitre-attack",
            "phase_name": "defense-evasion"
        }
    ],
    "external_references": [
        {
            "source_name": "mitre-attack",
            "url": "https://attack.mitre.org/wiki/Technique/T1066",
            "external_id": "T1066"
        }
    ],
    "object_marking_refs": [
        "marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168"
    ]
}

from stix2 import Filter

# create filter for type=malware
query = [Filter("type", "=", "malware")]

# query on the filter
mals = fs_source.query(query)

for mal in mals:
    print(mal)

{
    "type": "malware",
    "id": "malware--0f862b01-99da-47cc-9bdb-db4a86a95bb1",
    "created_by_ref": "identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5",
    "created": "2017-05-31T21:32:54.772Z",
    "modified": "2017-05-31T21:32:54.772Z",
    "name": "Emissary",
    "description": "Emissary is a Trojan that has been used by Lotus Blossom. It shares code with Elise, with both Trojans being part of a malware group referred to as LStudio.[[Citation: Lotus Blossom Dec 2015]]",
    "labels": [
        "malware"
    ],
    "external_references": [
        {
            "source_name": "mitre-attack",
            "url": "https://attack.mitre.org/wiki/Software/S0082",
            "external_id": "S0082"
        },
        {
            "source_name": "Lotus Blossom Dec 2015",
            "description": "Falcone, R. and Miller-Osborn, J.. (2015, December 18). Attack on French Diplomat Linked to Operation Lotus Blossom. Retrieved February 15, 2016.",
            "url": "http://researchcenter.paloaltonetworks.com/2015/12/attack-on-french-diplomat-linked-to-operation-lotus-blossom/"
        }
    ],
    "object_marking_refs": [
        "marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168"
    ]
}
{
    "type": "malware",
    "id": "malware--2a6f4c7b-e690-4cc7-ab6b-1f821fb6b80b",
    "created_by_ref": "identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5",
    "created": "2017-05-31T21:32:33.348Z",
    "modified": "2017-05-31T21:32:33.348Z",
    "name": "LOWBALL",
    "description": "LOWBALL is malware used by admin@338. It was used in August 2015 in email messages targeting Hong Kong-based media organizations.[[Citation: FireEye admin@338]]",
    "labels": [
        "malware"
    ],
    "external_references": [
        {
            "source_name": "mitre-attack",
            "url": "https://attack.mitre.org/wiki/Software/S0042",
            "external_id": "S0042"
        },
        {
            "source_name": "FireEye admin@338",
            "description": "FireEye Threat Intelligence. (2015, December 1). China-based Cyber Threat Group Uses Dropbox for Malware Communications and Targets Hong Kong Media Outlets. Retrieved December 4, 2015.",
            "url": "https://www.fireeye.com/blog/threat-research/2015/11/china-based-threat.html"
        }
    ],
    "object_marking_refs": [
        "marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168"
    ]
}
{
    "type": "malware",
    "id": "malware--00c3bfcb-99bd-4767-8c03-b08f585f5c8a",
    "created_by_ref": "identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5",
    "created": "2017-05-31T21:33:19.746Z",
    "modified": "2017-05-31T21:33:19.746Z",
    "name": "PowerDuke",
    "description": "PowerDuke is a backdoor that was used by APT29 in 2016. It has primarily been delivered through Microsoft Word or Excel attachments containing malicious macros.[[Citation: Volexity PowerDuke November 2016]]",
    "labels": [
        "malware"
    ],
    "external_references": [
        {
            "source_name": "mitre-attack",
            "url": "https://attack.mitre.org/wiki/Software/S0139",
            "external_id": "S0139"
        },
        {
            "source_name": "Volexity PowerDuke November 2016",
            "description": "Adair, S.. (2016, November 9). PowerDuke: Widespread Post-Election Spear Phishing Campaigns Targeting Think Tanks and NGOs. Retrieved January 11, 2017.",
            "url": "https://www.volexity.com/blog/2016/11/09/powerduke-post-election-spear-phishing-campaigns-targeting-think-tanks-and-ngos/"
        }
    ],
    "object_marking_refs": [
        "marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168"
    ]
}
{
    "type": "malware",
    "id": "malware--0db09158-6e48-4e7c-8ce7-2b10b9c0c039",
    "created_by_ref": "identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5",
    "created": "2017-05-31T21:32:55.126Z",
    "modified": "2017-05-31T21:32:55.126Z",
    "name": "Misdat",
    "description": "Misdat is a backdoor that was used by Dust Storm from 2010 to 2011.[[Citation: Cylance Dust Storm]]",
    "labels": [
        "malware"
    ],
    "external_references": [
        {
            "source_name": "mitre-attack",
            "url": "https://attack.mitre.org/wiki/Software/S0083",
            "external_id": "S0083"
        },
        {
            "source_name": "Cylance Dust Storm",
            "description": "Gross, J. (2016, February 23). Operation Dust Storm. Retrieved February 25, 2016.",
            "url": "https://www.cylance.com/hubfs/2015%20cylance%20website/assets/operation-dust-storm/Op%20Dust%20Storm%20Report.pdf?t=1456259131512"
        }
    ],
    "object_marking_refs": [
        "marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168"
    ]
}
{
    "type": "malware",
    "id": "malware--1d808f62-cf63-4063-9727-ff6132514c22",
    "created_by_ref": "identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5",
    "created": "2017-05-31T21:33:06.433Z",
    "modified": "2017-05-31T21:33:06.433Z",
    "name": "WEBC2",
    "description": "WEBC2 is a backdoor used by APT1 to retrieve a Web page from a predetermined C2 server.[[Citation: Mandiant APT1 Appendix]]",
    "labels": [
        "malware"
    ],
    "external_references": [
        {
            "source_name": "mitre-attack",
            "url": "https://attack.mitre.org/wiki/Software/S0109",
            "external_id": "S0109"
        },
        {
            "source_name": "Mandiant APT1 Appendix",
            "description": "Mandiant. (n.d.). Appendix C (Digital) - The Malware Arsenal. Retrieved July 18, 2016.",
            "url": "https://www.fireeye.com/content/dam/fireeye-www/services/pdfs/mandiant-apt1-report-appendix.zip"
        }
    ],
    "object_marking_refs": [
        "marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168"
    ]
}

# add more filters to the query
query.append(Filter("modified", ">" , "2017-05-31T21:33:10.772474Z"))

mals = fs_source.query(query)

# for visual purposes
for mal in mals:
    print(mal)

{
    "type": "malware",
    "id": "malware--00c3bfcb-99bd-4767-8c03-b08f585f5c8a",
    "created_by_ref": "identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5",
    "created": "2017-05-31T21:33:19.746Z",
    "modified": "2017-05-31T21:33:19.746Z",
    "name": "PowerDuke",
    "description": "PowerDuke is a backdoor that was used by APT29 in 2016. It has primarily been delivered through Microsoft Word or Excel attachments containing malicious macros.[[Citation: Volexity PowerDuke November 2016]]",
    "labels": [
        "malware"
    ],
    "external_references": [
        {
            "source_name": "mitre-attack",
            "url": "https://attack.mitre.org/wiki/Software/S0139",
            "external_id": "S0139"
        },
        {
            "source_name": "Volexity PowerDuke November 2016",
            "description": "Adair, S.. (2016, November 9). PowerDuke: Widespread Post-Election Spear Phishing Campaigns Targeting Think Tanks and NGOs. Retrieved January 11, 2017.",
            "url": "https://www.volexity.com/blog/2016/11/09/powerduke-post-election-spear-phishing-campaigns-targeting-think-tanks-and-ngos/"
        }
    ],
    "object_marking_refs": [
        "marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168"
    ]
}

FileSystemSink - (if STIX content is only to be pushed to FileSystem)

from stix2 import FileSystemSink, Campaign
"""
Working with FileSystemSink for pushing STIX content.
"""
# create FileSystemSink
fs_sink = FileSystemSink("/home/michael/Desktop/sample_stix2_data")

# create STIX objects and add to sink
camp = Campaign(name="The Crusades",
               objective="Infiltrating Israeli, Iranian and Palestinian digital infrastructure and government systems.",
               aliases=["Desert Moon"])

ind = Indicator(description="Crusades C2 implant",
                labels=["malicious-activity"],
                pattern="[file:hashes.'SHA-256' = '54b7e05e39a59428743635242e4a867c932140a999f52a1e54fa7ee6a440c73b']")

ind1 = Indicator(description="Crusades C2 implant",
                 labels=["malicious-activity"],
                 pattern="[file:hashes.'SHA-256' = '54b7e05e39a59428743635242e4a867c932140a999f52a1e54fa7ee6a440c73b']")

# add Campaign object to FileSystemSink
fs_sink.add(camp)

# can also add STIX objects to FileSystemSink in on call
fs_sink.add([ind, ind1])

Data Markings

Creating Objects With Data Markings

To create an object with a (predefined) TLP marking to an object, just provide it as a keyword argument to the constructor. The TLP markings can easily be imported from python-stix2.

from stix2 import Indicator, TLP_AMBER

indicator = Indicator(labels=["malicious-activity"],
                      pattern="[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",
                      object_marking_refs=TLP_AMBER)
print(indicator)

{
    "type": "indicator",
    "id": "indicator--65ff0082-bb92-4812-9b74-b144b858297f",
    "created": "2017-11-13T14:42:14.641Z",
    "modified": "2017-11-13T14:42:14.641Z",
    "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",
    "valid_from": "2017-11-13T14:42:14.641818Z",
    "labels": [
        "malicious-activity"
    ],
    "object_marking_refs": [
        "marking-definition--f88d31f6-486f-44da-b317-01333bde0b82"
    ]
}

If you’re creating your own marking (for example, a Statement marking), first create the statement marking:

from stix2 import MarkingDefinition, StatementMarking

marking_definition = MarkingDefinition(
    definition_type="statement",
    definition=StatementMarking(statement="Copyright 2017, Example Corp")
)
print(marking_definition)

{
    "type": "marking-definition",
    "id": "marking-definition--d16f0975-c5dd-4b25-a41d-af4afcc5da92",
    "created": "2017-11-13T14:43:30.558058Z",
    "definition_type": "statement",
    "definition": {
        "statement": "Copyright 2017, Example Corp"
    }
}

Then you can add it to an object as it’s being created (passing either full object or the the ID as a keyword argument, like with relationships).

indicator2 = Indicator(labels=["malicious-activity"],
                      pattern="[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",
                      object_marking_refs=marking_definition)
print(indicator2)

{
    "type": "indicator",
    "id": "indicator--526cda4e-6745-4cd6-852f-0750c6a79784",
    "created": "2017-10-04T14:43:09.586Z",
    "modified": "2017-10-04T14:43:09.586Z",
    "labels": [
        "malicious-activity"
    ],
    "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",
    "valid_from": "2017-10-04T14:43:09.586133Z",
    "object_marking_refs": [
        "marking-definition--030bb5c6-c5eb-4e9c-8e7a-b9aab08ded53"
    ]
}

indicator3 = Indicator(labels=["malicious-activity"],
                      pattern="[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",
                      object_marking_refs="marking-definition--f88d31f6-486f-44da-b317-01333bde0b82")
print(indicator3)

{
    "type": "indicator",
    "id": "indicator--1505b789-fcd2-48ee-bea9-3b20627a4abd",
    "created": "2017-10-04T14:43:20.049Z",
    "modified": "2017-10-04T14:43:20.049Z",
    "labels": [
        "malicious-activity"
    ],
    "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",
    "valid_from": "2017-10-04T14:43:20.049166Z",
    "object_marking_refs": [
        "marking-definition--f88d31f6-486f-44da-b317-01333bde0b82"
    ]
}

Granular markings work in the same way, except you also need to provide a full granular-marking object (including the selector).

from stix2 import Malware, TLP_WHITE

malware = Malware(name="Poison Ivy",
                  labels=['remote-access-trojan'],
                  description="A ransomware related to ...",
                  granular_markings=[
                      {
                          "selectors": ["description"],
                          "marking_ref": marking_definition
                      },
                      {
                          "selectors": ["name"],
                          "marking_ref": TLP_WHITE
                      }
                  ])
print(malware)

{
    "type": "malware",
    "id": "malware--f7128008-f6ab-4d43-a8a2-a681651268f8",
    "created": "2017-11-13T14:43:34.857Z",
    "modified": "2017-11-13T14:43:34.857Z",
    "name": "Poison Ivy",
    "description": "A ransomware related to ...",
    "labels": [
        "remote-access-trojan"
    ],
    "granular_markings": [
        {
            "marking_ref": "marking-definition--d16f0975-c5dd-4b25-a41d-af4afcc5da92",
            "selectors": [
                "description"
            ]
        },
        {
            "marking_ref": "marking-definition--613f2e26-407d-48c7-9eca-b8e91df99dc9",
            "selectors": [
                "name"
            ]
        }
    ]
}

Make sure that the selector is a field that exists and is populated on the object, otherwise this will cause an error:

Malware(name="Poison Ivy",
        labels=['remote-access-trojan'],
        description="A ransomware related to ...",
        granular_markings=[
            {
                "selectors": ["title"],
                "marking_ref": marking_definition
            }
        ])

InvalidSelectorError: Selector title in Malware is not valid!

Adding Data Markings To Existing Objects

Several functions exist to support working with data markings.

Both object markings and granular markings can be added to STIX objects which have already been created.

Note: Doing so will create a new version of the object (note the updated modified time).

indicator4 = indicator.add_markings(marking_definition)
print(indicator4)

{
    "type": "indicator",
    "id": "indicator--409a0b15-1108-4251-8aee-a08995976561",
    "created": "2017-10-04T14:42:54.685Z",
    "modified": "2017-10-04T15:03:46.599Z",
    "labels": [
        "malicious-activity"
    ],
    "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",
    "valid_from": "2017-10-04T14:42:54.685184Z",
    "object_marking_refs": [
        "marking-definition--f88d31f6-486f-44da-b317-01333bde0b82",
        "marking-definition--030bb5c6-c5eb-4e9c-8e7a-b9aab08ded53"
    ]
}

You can also remove specific markings from STIX objects. This will also create a new version of the object.

indicator5 = indicator4.remove_markings(marking_definition)
print(indicator5)

{
    "type": "indicator",
    "id": "indicator--409a0b15-1108-4251-8aee-a08995976561",
    "created": "2017-10-04T14:42:54.685Z",
    "modified": "2017-10-04T15:03:54.290Z",
    "labels": [
        "malicious-activity"
    ],
    "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",
    "valid_from": "2017-10-04T14:42:54.685184Z",
    "object_marking_refs": [
        "marking-definition--f88d31f6-486f-44da-b317-01333bde0b82"
    ]
}

The markings on an object can be replaced with a different set of markings:

from stix2 import TLP_GREEN

indicator6 = indicator5.set_markings([TLP_GREEN, marking_definition])
print(indicator6)

{
    "type": "indicator",
    "id": "indicator--409a0b15-1108-4251-8aee-a08995976561",
    "created": "2017-10-04T14:42:54.685Z",
    "modified": "2017-10-04T15:04:04.218Z",
    "labels": [
        "malicious-activity"
    ],
    "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",
    "valid_from": "2017-10-04T14:42:54.685184Z",
    "object_marking_refs": [
        "marking-definition--34098fce-860f-48ae-8e50-ebd3cc5e41da",
        "marking-definition--030bb5c6-c5eb-4e9c-8e7a-b9aab08ded53"
    ]
}

STIX objects can also be cleared of all markings with clear_markings():

indicator7 = indicator5.clear_markings()
print(indicator7)

{
    "type": "indicator",
    "id": "indicator--409a0b15-1108-4251-8aee-a08995976561",
    "created": "2017-10-04T14:42:54.685Z",
    "modified": "2017-10-04T14:54:39.331Z",
    "labels": [
        "malicious-activity"
    ],
    "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",
    "valid_from": "2017-10-04T14:42:54.685184Z"
}

All of these functions can be used for granular markings by passing in a list of selectors. Note that they will create new versions of the objects.

Evaluating Data Markings

You can get a list of the object markings on a STIX object:

indicator6.get_markings()

['marking-definition--34098fce-860f-48ae-8e50-ebd3cc5e41da',
 'marking-definition--030bb5c6-c5eb-4e9c-8e7a-b9aab08ded53']

To get a list of the granular markings on an object, pass the object and a list of selectors to get_markings():

from stix2 import get_markings

get_markings(malware, 'name')

['marking-definition--613f2e26-407d-48c7-9eca-b8e91df99dc9']

You can also call get_markings() as a method on the STIX object.

malware.get_markings('name')

['marking-definition--613f2e26-407d-48c7-9eca-b8e91df99dc9']

Finally, you may also check if an object is marked by a specific markings. Again, for granular markings, pass in the selector or list of selectors.

indicator.is_marked(TLP_AMBER.id)

True

malware.is_marked(TLP_WHITE.id, 'name')

True

malware.is_marked(TLP_WHITE.id, 'description')

False

Memory

The Memory suite consists of MemoryStore, MemorySource, and MemorySink. Under the hood, the Memory suite points to an in-memory dictionary. Similarly, the MemoryStore is a just a wrapper around a paired MemorySource and MemorySink; as there is quite limited uses for just a MemorySource or a MemorySink, it is recommended to always use MemoryStore. The MemoryStore is intended for retrieving/searching and pushing STIX content to memory. It is important to note that all STIX content in memory is not backed up on the file system (disk), as that functionality is encompassed within the FileSystemStore. However, the Memory suite does provide some utility methods for saving and loading STIX content to disk. MemoryStore.save_to_file() allows for saving all the STIX content that is in memory to a json file. MemoryStore.load_from_file() allows for loading STIX content from a JSON-formatted file.

Memory API

A note on adding and retreiving STIX content to the Memory suite: As mentioned, under the hood the Memory suite is an internal, in-memory dictionary. STIX content that is to be added can be in the following forms: python-stix2 objects, (Python) dictionaries (of valid STIX objects or Bundles), JSON-encoded strings (of valid STIX objects or Bundles), or a (Python) list of any of the previously listed types. MemoryStore actually stores STIX content either as python-stix2 objects or as (Python) dictionaries, reducing and converting any of the aforementioned types to one of those. Additionally, whatever form the STIX object is stored as, is how it will be returned when retrieved. python-stix2 objects, and json-encoded strings (of STIX content) are stored as python-stix2 objects, while (Python) dictionaries (of STIX objects) are stored as (Python) dictionaries.

A note on load_from_file(): For load_from_file(), STIX content is assumed to be in JSON form within the file, as an individual STIX object or in a Bundle. When the JSON is loaded, the STIX objects are parsed into python-stix2 objects before being stored in the in-memory dictionary.

A note on save_to_file(): This method dumps all STIX content that is in the MemoryStore to the specified file. The file format will be JSON, and the STIX content will be within a STIX Bundle. Note also that the output form will be a JSON STIX Bundle regardless of the form that the individual STIX objects are stored in (i.e. supplied to) the MemoryStore.

Memory Examples

MemoryStore

from stix2 import MemoryStore, Indicator

# create default MemoryStore
mem = MemoryStore()

# insert newly created indicator into memory
ind = Indicator(description="Crusades C2 implant",
                labels=["malicious-activity"],
                pattern="[file:hashes.'SHA-256' = '54b7e05e39a59428743635242e4a867c932140a999f52a1e54fa7ee6a440c73b']")

mem.add(ind)

# for visual purposes
print(mem.get(ind.id))

{
    "type": "indicator",
    "id": "indicator--2f61e4e7-0891-4e09-b79a-66f5e594fec0",
    "created": "2017-11-17T17:01:31.590Z",
    "modified": "2017-11-17T17:01:31.590Z",
    "description": "Crusades C2 implant",
    "pattern": "[file:hashes.'SHA-256' = '54b7e05e39a59428743635242e4a867c932140a999f52a1e54fa7ee6a440c73b']",
    "valid_from": "2017-11-17T17:01:31.590939Z",
    "labels": [
        "malicious-activity"
    ]
}

from stix2 import Malware

# add multiple STIX objects into memory
ind2 = Indicator(description="Crusades stage 2 implant",
                 labels=["malicious-activity"],
                 pattern="[file:hashes.'SHA-256' = '70fa62fb218dd9d936ee570dbe531dfa4e7c128ff37e6af7a6a6b2485487e50a']")
ind3 = Indicator(description="Crusades stage 2 implant variant",
                 labels=["malicious-activity"],
                 pattern="[file:hashes.'SHA-256' = '31a45e777e4d58b97f4c43e38006f8cd6580ddabc4037905b2fad734712b582c']")
mal = Malware(labels=["rootkit"], name= "Alexios")

mem.add([ind2,ind3, mal])

# for visual purposes
print(mem.get(ind3.id))

{
    "type": "indicator",
    "id": "indicator--ddb765ba-ff1e-4285-bf33-1f6d08f583d6",
    "created": "2017-11-17T17:01:31.799Z",
    "modified": "2017-11-17T17:01:31.799Z",
    "description": "Crusades stage 2 implant variant",
    "pattern": "[file:hashes.'SHA-256' = '31a45e777e4d58b97f4c43e38006f8cd6580ddabc4037905b2fad734712b582c']",
    "valid_from": "2017-11-17T17:01:31.799228Z",
    "labels": [
        "malicious-activity"
    ]
}

from stix2 import Filter

mal = mem.query([Filter("labels","=", "rootkit")])[0]
print(mal)

{
    "type": "malware",
    "id": "malware--e8170e70-522f-4ec3-aa22-afb55bfad0b0",
    "created": "2017-11-17T17:01:31.806Z",
    "modified": "2017-11-17T17:01:31.806Z",
    "name": "Alexios",
    "labels": [
        "rootkit"
    ]
}

from stix2 import Filter

# add json formatted string to MemoryStore
# Again, would NOT manually create json-formatted string
# but taken as an output form from another source
report = '{"type": "report","id": "report--2add14d6-bbf3-4308-bb8e-226d314a08e4","labels": ["threat-report"], "name": "The Crusades: Looking into the relentless infiltration of Israels digital infrastructure.", "published": "2017-05-08T10:24:11.011Z", "object_refs":["malware--2daa14d6-cbf3-4308-bb8e-226d324a08e4"], "created": "2017-05-08T18:34:08.042Z", "modified": "2017-05-08T18:34:08.042Z"}'

mem.add(report)

print(mem.get("report--2add14d6-bbf3-4308-bb8e-226d314a08e4"))

{
    "type": "report",
    "id": "report--2add14d6-bbf3-4308-bb8e-226d314a08e4",
    "created": "2017-05-08T18:34:08.042Z",
    "modified": "2017-05-08T18:34:08.042Z",
    "name": "The Crusades: Looking into the relentless infiltration of Israels digital infrastructure.",
    "published": "2017-05-08T10:24:11.011Z",
    "object_refs": [
        "malware--2daa14d6-cbf3-4308-bb8e-226d324a08e4"
    ],
    "labels": [
        "threat-report"
    ]
}

load_from_file() and save_to_file()

mem_2 = MemoryStore()

# save (dump) all STIX content in MemoryStore to json file
mem.save_to_file("path_to_target_file.json")

# load(add) STIX content from json file into MemoryStore
mem_2.load_from_file("path_to_target_file.json")

report = mem_2.get("report--2add14d6-bbf3-4308-bb8e-226d314a08e4")

# for visualpurposes
print(report)

{
    "type": "report",
    "id": "report--2add14d6-bbf3-4308-bb8e-226d314a08e4",
    "created": "2017-05-08T18:34:08.042Z",
    "modified": "2017-05-08T18:34:08.042Z",
    "name": "The Crusades: Looking into the relentless infiltration of Israels digital infrastructure.",
    "published": "2017-05-08T10:24:11.011Z",
    "object_refs": [
        "malware--2daa14d6-cbf3-4308-bb8e-226d324a08e4"
    ],
    "labels": [
        "threat-report"
    ]
}

Parsing STIX Content

Parsing STIX content is as easy as calling the parse() function on a JSON string. It will automatically determine the type of the object. The STIX objects within bundle objects, and the cyber observables contained within observed-data objects will be parsed as well.

from stix2 import parse

input_string = """{
    "type": "observed-data",
    "id": "observed-data--b67d30ff-02ac-498a-92f9-32f845f448cf",
    "created": "2016-04-06T19:58:16.000Z",
    "modified": "2016-04-06T19:58:16.000Z",
    "first_observed": "2015-12-21T19:00:00Z",
    "last_observed": "2015-12-21T19:00:00Z",
    "number_observed": 50,
    "objects": {
        "0": {
            "type": "file",
            "hashes": {
                "SHA-256": "0969de02ecf8a5f003e3f6d063d848c8a193aada092623f8ce408c15bcb5f038"
            }
        }
    }
}"""

obj = parse(input_string)
print(obj.type)
print(obj.objects["0"].hashes['SHA-256'])

observed-data
0969de02ecf8a5f003e3f6d063d848c8a193aada092623f8ce408c15bcb5f038

Serializing STIX Objects

The string representation of all STIX classes is a valid STIX JSON object.

from stix2 import Indicator

indicator = Indicator(name="File hash for malware variant",
                      labels=["malicious-activity"],
                      pattern="[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']")

print(str(indicator))

{
    "type": "indicator",
    "id": "indicator--5eac4517-6539-4e48-ab51-7d499f599674",
    "created": "2017-11-09T19:21:06.285Z",
    "modified": "2017-11-09T19:21:06.285Z",
    "name": "File hash for malware variant",
    "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",
    "valid_from": "2017-11-09T19:21:06.285451Z",
    "labels": [
        "malicious-activity"
    ]
}

However, the string representation can be slow, as it sorts properties to be in a more readable order. If you need performance and don’t care about the human-readability of the output, use the object’s serialize() function:

print(indicator.serialize())

{"valid_from": "2017-11-09T19:21:06.285451Z", "name": "File hash for malware variant", "created": "2017-11-09T19:21:06.285Z", "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']", "labels": ["malicious-activity"], "modified": "2017-11-09T19:21:06.285Z", "type": "indicator", "id": "indicator--5eac4517-6539-4e48-ab51-7d499f599674"}

TAXIICollection

The TAXIICollection suite contains TAXIICollectionStore, TAXIICollectionSource, and TAXIICollectionSink. TAXIICollectionStore pushes and retrieves STIX content to local/remote TAXII Collection(s). TAXIICollectionSource retrieves STIX content from local/remote TAXII Collection(s). TAXIICollectionSink pushes STIX content to local/remote TAXII Collection(s). Each of the interfaces is designed to be bound to a Collection from the taxii2client library (taxii2client.Collection), where all TAXIICollection API calls will be executed through that Collection instance.

A note on TAXII2 searching/filtering of STIX content: TAXII2 server implementations natively support searching on the STIX2 object properties: id, type and version; API requests made to TAXII2 can contain filter arguments for those 3 properties. However, the TAXIICollection suite supports searching on all STIX2 common object properties (see Filters documentation for full listing). This works simply by augmenting the filtering that is done remotely at the TAXII2 server instance. TAXIICollection will seperate any supplied queries into TAXII supported filters and non-supported filters. During a TAXIICollection API call, TAXII2 supported filters get inserted into the TAXII2 server request (to be evaluated at the server). The rest of the filters are kept locally and then applied to the STIX2 content that is returned from the TAXII2 server, before being returned from the TAXIICollection API call.

TAXIICollection API

TAXIICollection Examples

TAXIICollectionSource

from stix2 import TAXIICollectionSource
from taxii2client import Collection

# establish TAXII2 Collection instance
collection = Collection("http://127.0.0.1:5000/trustgroup1/collections/91a7b528-80eb-42ed-a74d-c6fbd5a26116/", user="admin", password="Password0")
# supply the TAXII2 collection to TAXIICollection
tc_source = TAXIICollectionSource(collection)

#retrieve STIX objects by id
stix_obj = tc_source.get("malware--fdd60b30-b67c-11e3-b0b9-f01faf20d111")
stix_obj_versions = tc_source.all_versions("indicator--a932fcc6-e032-176c-126f-cb970a5a1ade")

#for visual purposes
print(stix_obj)
print("-------")
for so in stix_obj_versions:
    print(so)

{
    "type": "malware",
    "id": "malware--fdd60b30-b67c-11e3-b0b9-f01faf20d111",
    "created": "2017-01-27T13:49:53.997Z",
    "modified": "2017-01-27T13:49:53.997Z",
    "name": "Poison Ivy",
    "description": "Poison Ivy",
    "labels": [
        "remote-access-trojan"
    ]
}
-------
{
    "type": "indicator",
    "id": "indicator--a932fcc6-e032-176c-126f-cb970a5a1ade",
    "created": "2014-05-08T09:00:00.000Z",
    "modified": "2014-05-08T09:00:00.000Z",
    "name": "File hash for Poison Ivy variant",
    "pattern": "[file:hashes.'SHA-256' = 'ef537f25c895bfa782526529a9b63d97aa631564d5d789c2b765448c8635fb6c']",
    "valid_from": "2014-05-08T09:00:00Z",
    "labels": [
        "file-hash-watchlist"
    ]
}

from stix2 import Filter

# retrieve multiple object from TAXIICollectionSource
# by using filters
f1 = Filter("type","=", "indicator")

indicators = tc_source.query([f1])

#for visual purposes
for indicator in indicators:
    print(indicator)

{
    "type": "indicator",
    "id": "indicator--a932fcc6-e032-176c-126f-cb970a5a1ade",
    "created": "2014-05-08T09:00:00.000Z",
    "modified": "2014-05-08T09:00:00.000Z",
    "name": "File hash for Poison Ivy variant",
    "pattern": "[file:hashes.'SHA-256' = 'ef537f25c895bfa782526529a9b63d97aa631564d5d789c2b765448c8635fb6c']",
    "valid_from": "2014-05-08T09:00:00Z",
    "labels": [
        "file-hash-watchlist"
    ]
}

TAXIICollectionSink

from stix2 import TAXIICollectionSink, ThreatActor

#create TAXIICollectionSINK and push STIX content to it
tc_sink = TAXIICollectionSink(collection)

# create new STIX threat-actor
ta = ThreatActor(name="Teddy Bear",
                labels=["nation-state"],
                sophistication="innovator",
                resource_level="government",
                goals=[
                    "compromising environment NGOs",
                    "water-hole attacks geared towards energy sector",
                ])

tc_sink.add(ta)

TAXIICollectionStore

from stix2 import TAXIICollectionStore

# create TAXIICollectionStore - note the same collection instance can
# be used for the store
tc_store = TAXIICollectionStore(collection)

# retrieve STIX object by id from TAXII Collection through
# TAXIICollectionStore
stix_obj2 = tc_source.get("malware--fdd60b30-b67c-11e3-b0b9-f01faf20d111")

print(stix_obj2)

{
    "type": "malware",
    "id": "malware--fdd60b30-b67c-11e3-b0b9-f01faf20d111",
    "created": "2017-01-27T13:49:53.997Z",
    "modified": "2017-01-27T13:49:53.997Z",
    "name": "Poison Ivy",
    "description": "Poison Ivy",
    "labels": [
        "remote-access-trojan"
    ]
}

from stix2 import indicator

# add STIX object to TAXIICollectionStore
ind = Indicator(description="Smokey Bear implant",
                labels=["malicious-activity"],
                pattern="[file:hashes.'SHA-256' = '09c7e05a39a59428743635242e4a867c932140a909f12a1e54fa7ee6a440c73b']")

tc_store.add(ind)

Technical Specification Support

How imports will work

Imports can be used in different ways depending on the use case and support levels.

People who want to support the latest version of STIX 2.X without having to make changes, can implicitly use the latest version:

import stix2

stix2.Indicator()

or,

from stix2 import Indicator

Indicator()

People who want to use an explicit version:

import stix2.v20

stix2.v20.Indicator()

or,

from stix2.v20 import Indicator

Indicator()

or even,

import stix2.v20 as stix2

stix2.Indicator()

The last option makes it easy to update to a new version in one place per file, once you’ve made the deliberate action to do this.

People who want to use multiple versions in a single file:

import stix2

stix2.v20.Indicator()
stix2.v21.Indicator()

or,

from stix2 import v20, v21

v20.Indicator()
v21.Indicator()

or (less preferred):

from stix2.v20 import Indicator as Indicator_v20
from stix2.v21 import Indicator as Indicator_v21

Indicator_v20()
Indicator_v21()

How parsing will work

If the version positional argument is not provided. The data will be parsed using the latest version of STIX 2.X supported by the stix2 library.

You can lock your parse() method to a specific STIX version by:

from stix2 import parse

indicator = parse("""{
    "type": "indicator",
    "id": "indicator--dbcbd659-c927-4f9a-994f-0a2632274394",
    "created": "2017-09-26T23:33:39.829Z",
    "modified": "2017-09-26T23:33:39.829Z",
    "labels": [
        "malicious-activity"
    ],
    "name": "File hash for malware variant",
    "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",
    "valid_from": "2017-09-26T23:33:39.829952Z"
}""", version="2.0")
print(indicator)

{
    "type": "indicator",
    "id": "indicator--dbcbd659-c927-4f9a-994f-0a2632274394",
    "created": "2017-09-26T23:33:39.829Z",
    "modified": "2017-09-26T23:33:39.829Z",
    "name": "File hash for malware variant",
    "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",
    "valid_from": "2017-09-26T23:33:39.829952Z",
    "labels": [
        "malicious-activity"
    ]
}

Keep in mind that if a 2.1 or higher object is parsed, the operation will fail.

How will custom content work

CustomObject, CustomObservable, CustomMarking and CustomExtension must be registered explicitly by STIX version. This is a design decision since properties or requirements may change as the STIX Technical Specification advances.

You can perform this by:

import stix2

# Make my custom observable available in STIX 2.0
@stix2.v20.CustomObservable('x-new-object-type',
                            (("prop", stix2.properties.BooleanProperty())))
class NewObject2(object):
    pass


# Make my custom observable available in STIX 2.1
@stix2.v21.CustomObservable('x-new-object-type',
                            (("prop", stix2.properties.BooleanProperty())))
class NewObject2(object):
    pass

Versioning

To create a new version of an existing object, specify the property(ies) you want to change and their new values:

from stix2 import Indicator

indicator = Indicator(created="2016-01-01T08:00:00.000Z",
                      name="File hash for suspicious file",
                      labels=["anomalous-activity"],
                      pattern="[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']")

indicator2 = indicator.new_version(name="File hash for Foobar malware",
                                   labels=["malicious-activity"])
print(indicator2)

{
    "type": "indicator",
    "id": "indicator--92bb1ae4-db9c-4d6e-8ded-ef7280b4439a",
    "created": "2016-01-01T08:00:00.000Z",
    "modified": "2017-09-26T23:39:07.149Z",
    "labels": [
        "malicious-activity"
    ],
    "name": "File hash for Foobar malware",
    "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",
    "valid_from": "2017-09-26T23:39:07.132129Z"
}

The modified time will be updated to the current time unless you provide a specific value as a keyword argument. Note that you can’t change the type, id, or created properties.

indicator.new_version(id="indicator--cc42e358-8b9b-493c-9646-6ecd73b41c21")

UnmodifiablePropertyError: These properties cannot be changed when making a new version: id.

To revoke an object:

indicator2 = indicator2.revoke()
print(indicator2)

{
    "type": "indicator",
    "id": "indicator--92bb1ae4-db9c-4d6e-8ded-ef7280b4439a",
    "created": "2016-01-01T08:00:00.000Z",
    "modified": "2017-09-26T23:39:09.463Z",
    "labels": [
        "malicious-activity"
    ],
    "name": "File hash for Foobar malware",
    "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",
    "valid_from": "2017-09-26T23:39:07.132129Z",
    "revoked": true
}

API Reference

This section of documentation contains information on all of the classes and functions in the stix2 API, as given by the package’s docstrings.

Note

All the classes and functions detailed in the pages below are importable directly from stix2. See also: How imports will work.

Python APIs for STIX 2.

core	STIX 2.0 Objects that are neither SDOs nor SROs.
datastore	Python STIX 2.0 DataStore API
environment
exceptions	STIX 2 error classes.
markings	Functions for working with STIX 2 Data Markings.
patterns	Classes to aid in working with the STIX 2 patterning language.
properties	Classes for representing properties of STIX Objects and Cyber Observables.
utils	Utility functions and classes for the stix2 library.
v20.common	STIX 2 Common Data Types and Properties.
v20.observables	STIX 2.0 Cyber Observable Objects.
v20.sdo	STIX 2.0 Domain Objects
v20.sro	STIX 2.0 Relationship Objects.

DataStore API

Warning

The DataStore API is still in the planning stages and may be subject to major changes. We encourage anyone with feedback to contact the maintainers to help ensure the API meets a large variety of use cases.

One prominent feature of python-stix2 will be an interface for connecting different backend data stores containing STIX content. This will allow a uniform interface for querying and saving STIX content, and allow higher level code to be written without regard to the underlying data storage format. python-stix2 will define the API and contain some default implementations of this API, but developers are encouraged to write their own implementations.

Potential functions of the API include:

• get a STIX Object by ID (returns the most recent version).
• get all versions of a STIX object by ID.
• get all relationships involving a given object, and all related objects.
• save an object.
• query for objects that match certain criteria (query syntax TBD).

For all queries, the API will include a “filter” interface that can be used to either explicitly include or exclude results with certain criteria. For example,

• only trust content from a set of object creators.
• exclude content from certain (untrusted) object creators.
• only include content with a confidence above a certain threshold (once confidence is added to STIX).
• only return content that can be shared with external parties (in other words, that has TLP:GREEN markings).

Additionally, the python-stix2 library will contain a “composite” data store, which implements the DataStore API while delegating functionality to one or more “child” data stores.

Development Roadmap

Warning

Prior to version 1.0, all APIs are considered unstable and subject to change.

This is a list of (planned) features before version 1.0 is released.

• Serialization of all STIX and Cyber Observable objects to JSON.
• De-serialization (parsing) of all STIX and Cyber Observable objects.
• APIs for versioning (revising and revoking) STIX objects.
• APIs for marking STIX objects and interpreting markings of STIX objects.
• DataStore API, providing a common interface for querying sources of STIX content (such as objects in memory, on a filesystem, in a database, or via a TAXII feed).

Contributing

We’re thrilled that you’re interested in contributing to python-stix2! Here are some things you should know:

• contribution-guide.org has great ideas for contributing to any open-source project (not just python-stix2).
• All contributors must sign a Contributor License Agreement. See CONTRIBUTING.md in the project repository for specifics.
• If you are planning to implement a major feature (vs. fixing a bug), please discuss with a project maintainer first to ensure you aren’t duplicating the work of someone else, and that the feature is likely to be accepted.

Now, let’s get started!

Setting up a development environment

We recommend using a virtualenv.

1. Clone the repository. If you’re planning to make pull request, you should fork the repository on GitHub and clone your fork instead of the main repo:

git clone https://github.com/yourusername/cti-python-stix2.git

2. Install develoment-related dependencies:

cd cti-python-stix2
pip install -r requirements.txt

3. Install pre-commit git hooks:

pre-commit install

At this point you should be able to make changes to the code.

Code style

All code should follow PEP 8. We allow for line lengths up to 160 characters, but any lines over 80 characters should be the exception rather than the rule. PEP 8 conformance will be tested automatically by Tox and Travis-CI (see below).

Testing

Note

All of the tools mentioned in this section are installed when you run pip install -r requirements.txt.

python-stix2 uses pytest for testing. We encourage the use of test-driven development (TDD), where you write (failing) tests that demonstrate a bug or proposed new feature before writing code that fixes the bug or implements the features. Any code contributions to python-stix2 should come with new or updated tests.

To run the tests in your current Python environment, use the pytest command from the root project directory:

pytest

This should show all of the tests that ran, along with their status.

You can run a specific test file by passing it on the command line:

pytest stix2/test/test_<xxx>.py

To ensure that the test you wrote is running, you can deliberately add an assert False statement at the beginning of the test. This is another benefit of TDD, since you should be able to see the test failing (and ensure it’s being run) before making it pass.

tox allows you to test a package across multiple versions of Python. Setting up multiple Python environments is beyond the scope of this guide, but feel free to ask for help setting them up. Tox should be run from the root directory of the project:

tox

We aim for high test coverage, using the coverage.py library. Though it’s not an absolute requirement to maintain 100% coverage, all code contributions must be accompanied by tests. To run coverage and look for untested lines of code, run:

pytest --cov=stix2
coverage html

then look at the resulting report in htmlcov/index.html.

All commits pushed to the master branch or submitted as a pull request are tested with Travis-CI automatically.

Indices and tables

• Index
• Module Index
• Search Page