FileSystem¶

The FileSystem suite contains FileSystemStore, FileSystemSource and FileSystemSink. Under the hood, all FileSystem objects point to a file directory (on disk) that contains STIX2 content.

The directory and file structure of the intended STIX2 content should be:

stix2_content/
    /STIX2 Domain Object type
        STIX2 Domain Object
        STIX2 Domain Object
            .
            .
            .
    /STIX2 Domain Object type
        STIX2 Domain Object
        STIX2 Domain Object
            .
            .
            .
        .
        .
        .
    /STIX2 Domain Object type

The master STIX2 content directory contains subdirectories, each of which aligns to a STIX2 domain object type (i.e. “attack-pattern”, “campaign”, “malware”, etc.). Within each STIX2 domain object subdirectory are JSON files that are STIX2 domain objects of the specified type. The name of the json files correspond to the ID of the STIX2 domain object found within that file. A real example of the FileSystem directory structure:

stix2_content/
    /attack-pattern
        attack-pattern--00d0b012-8a03-410e-95de-5826bf542de6.json
        attack-pattern--0a3ead4e-6d47-4ccb-854c-a6a4f9d96b22.json
        attack-pattern--1b7ba276-eedc-4951-a762-0ceea2c030ec.json
    /campaign
    /course-of-action
        course-of-action--2a8de25c-f743-4348-b101-3ee33ab5871b.json
        course-of-action--2c3ce852-06a2-40ee-8fe6-086f6402a739.json
    /identity
        identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5.json
    /indicator
    /intrusion-set
    /malware
        malware--1d808f62-cf63-4063-9727-ff6132514c22.json
        malware--2eb9b131-d333-4a48-9eb4-d8dec46c19ee.json
    /observed-data
    /report
    /threat-actor
    /vulnerability

FileSystemStore is intended for use cases where STIX2 content is retrieved and pushed to the same file directory. As FileSystemStore is just a wrapper around a paired FileSystemSource and FileSystemSink that point the same file directory.

For use cases where STIX2 content will only be retrieved or pushed, then a FileSystemSource and FileSystemSink can be used individually. They can also be used individually when STIX2 content will be retrieved from one distinct file directory and pushed to another.

FileSystem API¶

A note on get(), all_versions(), and query(): The format of the STIX2 content targeted by the FileSystem suite is JSON files. When the FileSystemStore retrieves STIX2 content (in JSON) from disk, it will attempt to parse the content into full-featured python-stix2 objects and returned as such.

A note on add(): When STIX content is added (pushed) to the file system, the STIX content can be supplied in the following forms: Python STIX objects, Python dictionaries (of valid STIX objects or Bundles), JSON-encoded strings (of valid STIX objects or Bundles), or a (Python) list of any of the previously listed types. Any of the previous STIX content forms will be converted to a STIX JSON object (in a STIX Bundle) and written to disk.

FileSystem Examples¶

FileSystemStore¶

In [10]:

from stix2 import FileSystemStore

"""
Working with the FileSystemStore, where STIX content can be retrieved and pushed to a file system.
"""

# create FileSystemStore
fs = FileSystemStore("/home/michael/Desktop/sample_stix2_data")

# retrieve STIX2 content from FileSystemStore
ap = fs.get("attack-pattern--00d0b012-8a03-410e-95de-5826bf542de6")
mal = fs.get("malware--00c3bfcb-99bd-4767-8c03-b08f585f5c8a")

# for visual purposes
print(mal)

{
    "type": "malware",
    "id": "malware--00c3bfcb-99bd-4767-8c03-b08f585f5c8a",
    "created_by_ref": "identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5",
    "created": "2017-05-31T21:33:19.746Z",
    "modified": "2017-05-31T21:33:19.746Z",
    "name": "PowerDuke",
    "description": "PowerDuke is a backdoor that was used by APT29 in 2016. It has primarily been delivered through Microsoft Word or Excel attachments containing malicious macros.[[Citation: Volexity PowerDuke November 2016]]",
    "labels": [
        "malware"
    ],
    "external_references": [
        {
            "source_name": "mitre-attack",
            "url": "https://attack.mitre.org/wiki/Software/S0139",
            "external_id": "S0139"
        },
        {
            "source_name": "Volexity PowerDuke November 2016",
            "description": "Adair, S.. (2016, November 9). PowerDuke: Widespread Post-Election Spear Phishing Campaigns Targeting Think Tanks and NGOs. Retrieved January 11, 2017.",
            "url": "https://www.volexity.com/blog/2016/11/09/powerduke-post-election-spear-phishing-campaigns-targeting-think-tanks-and-ngos/"
        }
    ],
    "object_marking_refs": [
        "marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168"
    ]
}

In [2]:

from stix2 import ThreatActor, Indicator

# create new STIX threat-actor
ta = ThreatActor(name="Adjective Bear",
                labels=["nation-state"],
                sophistication="innovator",
                resource_level="government",
                goals=[
                    "compromising media outlets",
                    "water-hole attacks geared towards political, military targets",
                    "intelligence collection"
                ])

# create new indicators
ind = Indicator(description="Crusades C2 implant",
                labels=["malicious-activity"],
                pattern="[file:hashes.'SHA-256' = '54b7e05e39a59428743635242e4a867c932140a999f52a1e54fa7ee6a440c73b']")

ind1 = Indicator(description="Crusades C2 implant 2",
                 labels=["malicious-activity"],
                 pattern="[file:hashes.'SHA-256' = '64c7e05e40a59511743635242e4a867c932140a999f52a1e54fa7ee6a440c73b']")

# add STIX object (threat-actor) to FileSystemStore
fs.add(ta)

# can also add multiple STIX objects to FileSystemStore in one call
fs.add([ind, ind1])

FileSystemSource - (if STIX content is only to be retrieved from FileSystem)¶

In [4]:

from stix2 import FileSystemSource
"""
Working with FileSystemSource for retrieving STIX content.
"""

# create FileSystemSource
fs_source = FileSystemSource("/home/michael/Desktop/sample_stix2_data")

# retrieve STIX 2 objects
ap = fs_source.get("attack-pattern--00d0b012-8a03-410e-95de-5826bf542de6")

# for visual purposes
print(ap)

{
    "type": "attack-pattern",
    "id": "attack-pattern--00d0b012-8a03-410e-95de-5826bf542de6",
    "created_by_ref": "identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5",
    "created": "2017-05-31T21:30:54.176Z",
    "modified": "2017-05-31T21:30:54.176Z",
    "name": "Indicator Removal from Tools",
    "description": "If a malicious...command-line parameters, Process monitoring",
    "kill_chain_phases": [
        {
            "kill_chain_name": "mitre-attack",
            "phase_name": "defense-evasion"
        }
    ],
    "external_references": [
        {
            "source_name": "mitre-attack",
            "url": "https://attack.mitre.org/wiki/Technique/T1066",
            "external_id": "T1066"
        }
    ],
    "object_marking_refs": [
        "marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168"
    ]
}

In [5]:

from stix2 import Filter

# create filter for type=malware
query = [Filter("type", "=", "malware")]

# query on the filter
mals = fs_source.query(query)

for mal in mals:
    print(mal)

{
    "type": "malware",
    "id": "malware--0f862b01-99da-47cc-9bdb-db4a86a95bb1",
    "created_by_ref": "identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5",
    "created": "2017-05-31T21:32:54.772Z",
    "modified": "2017-05-31T21:32:54.772Z",
    "name": "Emissary",
    "description": "Emissary is a Trojan that has been used by Lotus Blossom. It shares code with Elise, with both Trojans being part of a malware group referred to as LStudio.[[Citation: Lotus Blossom Dec 2015]]",
    "labels": [
        "malware"
    ],
    "external_references": [
        {
            "source_name": "mitre-attack",
            "url": "https://attack.mitre.org/wiki/Software/S0082",
            "external_id": "S0082"
        },
        {
            "source_name": "Lotus Blossom Dec 2015",
            "description": "Falcone, R. and Miller-Osborn, J.. (2015, December 18). Attack on French Diplomat Linked to Operation Lotus Blossom. Retrieved February 15, 2016.",
            "url": "http://researchcenter.paloaltonetworks.com/2015/12/attack-on-french-diplomat-linked-to-operation-lotus-blossom/"
        }
    ],
    "object_marking_refs": [
        "marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168"
    ]
}
{
    "type": "malware",
    "id": "malware--2a6f4c7b-e690-4cc7-ab6b-1f821fb6b80b",
    "created_by_ref": "identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5",
    "created": "2017-05-31T21:32:33.348Z",
    "modified": "2017-05-31T21:32:33.348Z",
    "name": "LOWBALL",
    "description": "LOWBALL is malware used by admin@338. It was used in August 2015 in email messages targeting Hong Kong-based media organizations.[[Citation: FireEye admin@338]]",
    "labels": [
        "malware"
    ],
    "external_references": [
        {
            "source_name": "mitre-attack",
            "url": "https://attack.mitre.org/wiki/Software/S0042",
            "external_id": "S0042"
        },
        {
            "source_name": "FireEye admin@338",
            "description": "FireEye Threat Intelligence. (2015, December 1). China-based Cyber Threat Group Uses Dropbox for Malware Communications and Targets Hong Kong Media Outlets. Retrieved December 4, 2015.",
            "url": "https://www.fireeye.com/blog/threat-research/2015/11/china-based-threat.html"
        }
    ],
    "object_marking_refs": [
        "marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168"
    ]
}
{
    "type": "malware",
    "id": "malware--00c3bfcb-99bd-4767-8c03-b08f585f5c8a",
    "created_by_ref": "identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5",
    "created": "2017-05-31T21:33:19.746Z",
    "modified": "2017-05-31T21:33:19.746Z",
    "name": "PowerDuke",
    "description": "PowerDuke is a backdoor that was used by APT29 in 2016. It has primarily been delivered through Microsoft Word or Excel attachments containing malicious macros.[[Citation: Volexity PowerDuke November 2016]]",
    "labels": [
        "malware"
    ],
    "external_references": [
        {
            "source_name": "mitre-attack",
            "url": "https://attack.mitre.org/wiki/Software/S0139",
            "external_id": "S0139"
        },
        {
            "source_name": "Volexity PowerDuke November 2016",
            "description": "Adair, S.. (2016, November 9). PowerDuke: Widespread Post-Election Spear Phishing Campaigns Targeting Think Tanks and NGOs. Retrieved January 11, 2017.",
            "url": "https://www.volexity.com/blog/2016/11/09/powerduke-post-election-spear-phishing-campaigns-targeting-think-tanks-and-ngos/"
        }
    ],
    "object_marking_refs": [
        "marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168"
    ]
}
{
    "type": "malware",
    "id": "malware--0db09158-6e48-4e7c-8ce7-2b10b9c0c039",
    "created_by_ref": "identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5",
    "created": "2017-05-31T21:32:55.126Z",
    "modified": "2017-05-31T21:32:55.126Z",
    "name": "Misdat",
    "description": "Misdat is a backdoor that was used by Dust Storm from 2010 to 2011.[[Citation: Cylance Dust Storm]]",
    "labels": [
        "malware"
    ],
    "external_references": [
        {
            "source_name": "mitre-attack",
            "url": "https://attack.mitre.org/wiki/Software/S0083",
            "external_id": "S0083"
        },
        {
            "source_name": "Cylance Dust Storm",
            "description": "Gross, J. (2016, February 23). Operation Dust Storm. Retrieved February 25, 2016.",
            "url": "https://www.cylance.com/hubfs/2015%20cylance%20website/assets/operation-dust-storm/Op%20Dust%20Storm%20Report.pdf?t=1456259131512"
        }
    ],
    "object_marking_refs": [
        "marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168"
    ]
}
{
    "type": "malware",
    "id": "malware--1d808f62-cf63-4063-9727-ff6132514c22",
    "created_by_ref": "identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5",
    "created": "2017-05-31T21:33:06.433Z",
    "modified": "2017-05-31T21:33:06.433Z",
    "name": "WEBC2",
    "description": "WEBC2 is a backdoor used by APT1 to retrieve a Web page from a predetermined C2 server.[[Citation: Mandiant APT1 Appendix]]",
    "labels": [
        "malware"
    ],
    "external_references": [
        {
            "source_name": "mitre-attack",
            "url": "https://attack.mitre.org/wiki/Software/S0109",
            "external_id": "S0109"
        },
        {
            "source_name": "Mandiant APT1 Appendix",
            "description": "Mandiant. (n.d.). Appendix C (Digital) - The Malware Arsenal. Retrieved July 18, 2016.",
            "url": "https://www.fireeye.com/content/dam/fireeye-www/services/pdfs/mandiant-apt1-report-appendix.zip"
        }
    ],
    "object_marking_refs": [
        "marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168"
    ]
}

In [6]:

# add more filters to the query
query.append(Filter("modified", ">" , "2017-05-31T21:33:10.772474Z"))

mals = fs_source.query(query)

# for visual purposes
for mal in mals:
    print(mal)

{
    "type": "malware",
    "id": "malware--00c3bfcb-99bd-4767-8c03-b08f585f5c8a",
    "created_by_ref": "identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5",
    "created": "2017-05-31T21:33:19.746Z",
    "modified": "2017-05-31T21:33:19.746Z",
    "name": "PowerDuke",
    "description": "PowerDuke is a backdoor that was used by APT29 in 2016. It has primarily been delivered through Microsoft Word or Excel attachments containing malicious macros.[[Citation: Volexity PowerDuke November 2016]]",
    "labels": [
        "malware"
    ],
    "external_references": [
        {
            "source_name": "mitre-attack",
            "url": "https://attack.mitre.org/wiki/Software/S0139",
            "external_id": "S0139"
        },
        {
            "source_name": "Volexity PowerDuke November 2016",
            "description": "Adair, S.. (2016, November 9). PowerDuke: Widespread Post-Election Spear Phishing Campaigns Targeting Think Tanks and NGOs. Retrieved January 11, 2017.",
            "url": "https://www.volexity.com/blog/2016/11/09/powerduke-post-election-spear-phishing-campaigns-targeting-think-tanks-and-ngos/"
        }
    ],
    "object_marking_refs": [
        "marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168"
    ]
}

FileSystemSink - (if STIX content is only to be pushed to FileSystem)¶

In [7]:

from stix2 import FileSystemSink, Campaign
"""
Working with FileSystemSink for pushing STIX content.
"""
# create FileSystemSink
fs_sink = FileSystemSink("/home/michael/Desktop/sample_stix2_data")

# create STIX objects and add to sink
camp = Campaign(name="The Crusades",
               objective="Infiltrating Israeli, Iranian and Palestinian digital infrastructure and government systems.",
               aliases=["Desert Moon"])

ind = Indicator(description="Crusades C2 implant",
                labels=["malicious-activity"],
                pattern="[file:hashes.'SHA-256' = '54b7e05e39a59428743635242e4a867c932140a999f52a1e54fa7ee6a440c73b']")

ind1 = Indicator(description="Crusades C2 implant",
                 labels=["malicious-activity"],
                 pattern="[file:hashes.'SHA-256' = '54b7e05e39a59428743635242e4a867c932140a999f52a1e54fa7ee6a440c73b']")

# add Campaign object to FileSystemSink
fs_sink.add(camp)

# can also add STIX objects to FileSystemSink in on call
fs_sink.add([ind, ind1])

FileSystem¶

FileSystem API¶

FileSystem Examples¶

FileSystemStore¶

FileSystemSource - (if STIX content is only to be retrieved from FileSystem)¶

FileSystemSink - (if STIX content is only to be pushed to FileSystem)¶

stix2

Navigation

Related Topics