FileSystem

The FileSystem suite contains FileSystemStore, FileSystemSource and FileSystemSink. Under the hood, all FileSystem objects point to a file directory (on disk) that contains STIX 2 content.

The directory and file structure of the intended STIX 2 content should be:

stix2_content/
    STIX2 Domain Object type/
        STIX2 Domain Object ID/
            'modified' timestamp.json
            'modified' timestamp.json
        STIX2 Domain Object ID/
            'modified' timestamp.json
            .
            .
    STIX2 Domain Object type/
        STIX2 Domain Object ID/
            'modified' timestamp.json
            .
            .
            .
        .
        .
        .
    STIX2 Domain Object type/

The master STIX 2 content directory contains subdirectories, each of which aligns to a STIX 2 domain object type (i.e. “attack-pattern”, “campaign”, “malware”, etc.). Within each STIX 2 domain object type’s subdirectory are further subdirectories containing JSON files that are STIX 2 domain objects of the specified type; the name of each of these subdirectories is the ID of the associated STIX 2 domain object. Inside each of these subdirectories are JSON files, the names of which correspond to the modified timestamp of the STIX 2 domain object found within that file. A real example of the FileSystem directory structure:

stix2_content/
    /attack-pattern
        /attack-pattern--00d0b012-8a03-410e-95de-5826bf542de6
            20201211035036648071.json
        /attack-pattern--0a3ead4e-6d47-4ccb-854c-a6a4f9d96b22
            20201210035036648071.json
        /attack-pattern--1b7ba276-eedc-4951-a762-0ceea2c030ec
            20201111035036648071.json
    /campaign
    /course-of-action
        /course-of-action--2a8de25c-f743-4348-b101-3ee33ab5871b
            20201011035036648071.json
        /course-of-action--2c3ce852-06a2-40ee-8fe6-086f6402a739
            20201010035036648071.json
    /identity
        /identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5
            20201215035036648071.json
    /indicator
    /intrusion-set
    /malware
        /malware--1d808f62-cf63-4063-9727-ff6132514c22
            20201211045036648071.json
        /malware--2eb9b131-d333-4a48-9eb4-d8dec46c19ee
            20201211035036648072.json
    /observed-data
    /report
    /threat-actor
    /vulnerability

FileSystemStore is intended for use cases where STIX 2 content is retrieved and pushed to the same file directory. As FileSystemStore is just a wrapper around a paired FileSystemSource and FileSystemSink that point the same file directory.

For use cases where STIX 2 content will only be retrieved or pushed, then a FileSystemSource and FileSystemSink can be used individually. They can also be used individually when STIX 2 content will be retrieved from one distinct file directory and pushed to another.

FileSystem API

A note on get(), all_versions(), and query(): The format of the STIX2 content targeted by the FileSystem suite is JSON files. When the FileSystemStore retrieves STIX 2 content (in JSON) from disk, it will attempt to parse the content into full-featured python-stix2 objects and returned as such.

A note on add(): When STIX content is added (pushed) to the file system, the STIX content can be supplied in the following forms: Python STIX objects, Python dictionaries (of valid STIX objects or Bundles), JSON-encoded strings (of valid STIX objects or Bundles), or a (Python) list of any of the previously listed types. Any of the previous STIX content forms will be converted to a STIX JSON object (in a STIX Bundle) and written to disk.

FileSystem Examples

FileSystemStore

Use the FileSystemStore when you want to both retrieve STIX content from the file system and push STIX content to it, too.

[7]:
from stix2 import FileSystemStore

# create FileSystemStore
fs = FileSystemStore("/tmp/stix2_store")

# retrieve STIX2 content from FileSystemStore
ap = fs.get("attack-pattern--0a3ead4e-6d47-4ccb-854c-a6a4f9d96b22")
mal = fs.get("malware--92ec0cbd-2c30-44a2-b270-73f4ec949841")

# for visual purposes
print(mal.serialize(pretty=True))
[7]:
{
    "type": "malware",
    "spec_version": "2.1",
    "id": "malware--92ec0cbd-2c30-44a2-b270-73f4ec949841",
    "created_by_ref": "identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5",
    "created": "2017-05-31T21:33:26.565Z",
    "modified": "2017-05-31T21:33:26.565Z",
    "name": "RTM",
    "description": "RTM is custom malware written in Delphi. It is used by the group of the same name (RTM).[[Citation: ESET RTM Feb 2017]]",
    "malware_types": [
        "malware"
    ],
    "is_family": false,
    "external_references": [
        {
            "source_name": "mitre-attack",
            "url": "https://attack.mitre.org/wiki/Software/S0148",
            "external_id": "S0148"
        },
        {
            "source_name": "ESET RTM Feb 2017",
            "description": "Faou, M. and Boutin, J.. (2017, February). Read The Manual: A Guide to the RTM Banking Trojan. Retrieved March 9, 2017.",
            "url": "https://www.welivesecurity.com/wp-content/uploads/2017/02/Read-The-Manual.pdf"
        }
    ],
    "object_marking_refs": [
        "marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168"
    ]
}
[8]:
from stix2 import ThreatActor, Indicator

# create new STIX threat-actor
ta = ThreatActor(name="Adjective Bear",
                 sophistication="innovator",
                 resource_level="government",
                 goals=[
                     "compromising media outlets",
                     "water-hole attacks geared towards political, military targets",
                     "intelligence collection"
                 ])

# create new indicators
ind = Indicator(description="Crusades C2 implant",
                pattern_type="stix",
                pattern="[file:hashes.'SHA-256' = '54b7e05e39a59428743635242e4a867c932140a999f52a1e54fa7ee6a440c73b']")

ind1 = Indicator(description="Crusades C2 implant 2",
                 pattern_type="stix",
                 pattern="[file:hashes.'SHA-256' = '64c7e05e40a59511743635242e4a867c932140a999f52a1e54fa7ee6a440c73b']")

# add STIX object (threat-actor) to FileSystemStore
fs.add(ta)

# can also add multiple STIX objects to FileSystemStore in one call
fs.add([ind, ind1])

FileSystemSource

Use the FileSystemSource when you only want to retrieve STIX content from the file system.

[9]:
from stix2 import FileSystemSource

# create FileSystemSource
fs_source = FileSystemSource("/tmp/stix2_source")

# retrieve STIX 2 objects
ap = fs_source.get("attack-pattern--0a3ead4e-6d47-4ccb-854c-a6a4f9d96b22")

# for visual purposes
print(ap)
[9]:
{
    "type": "attack-pattern",
    "spec_version": "2.1",
    "id": "attack-pattern--0a3ead4e-6d47-4ccb-854c-a6a4f9d96b22",
    "created_by_ref": "identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5",
    "created": "2017-05-31T21:30:19.735Z",
    "modified": "2017-05-31T21:30:19.735Z",
    "name": "Credential Dumping",
    "description": "Credential dumping is the process of obtaining account login and password information from the operating system and software. Credentials can be used to perform Windows Credential Editor, Mimikatz, and gsecdump. These tools are in use by both professional security testers and adversaries.\n\nPlaintext passwords can be obtained using tools such as Mimikatz to extract passwords stored by the Local Security Authority (LSA). If smart cards are used to authenticate to a domain using a personal identification number (PIN), then that PIN is also cached as a result and may be dumped.Mimikatz access the LSA Subsystem Service (LSASS) process by opening the process, locating the LSA secrets key, and decrypting the sections in memory where credential details are stored. Credential dumpers may also use methods for reflective DLL Injection to reduce potential indicators of malicious activity.\n\nNTLM hash dumpers open the Security Accounts Manager (SAM) on the local file system (%SystemRoot%/system32/config/SAM) or create a dump of the Registry SAM key to access stored account password hashes. Some hash dumpers will open the local file system as a device and parse to the SAM table to avoid file access defenses. Others will make an in-memory copy of the SAM table before reading hashes. Detection of compromised Legitimate Credentials in-use by adversaries may help as well. \n\nOn Windows 8.1 and Windows Server 2012 R2, monitor Windows Logs for LSASS.exe creation to verify that LSASS started as a protected process.\n\nMonitor processes and command-line arguments for program execution that may be indicative of credential dumping. Remote access tools may contain built-in features or incorporate existing tools like Mimikatz. PowerShell scripts also exist that contain credential dumping functionality, such as PowerSploit's Invoke-Mimikatz module,[[Citation: Powersploit]] which may require additional logging features to be configured in the operating system to collect necessary information for analysis.\n\nPlatforms: Windows Server 2003, Windows Server 2008, Windows Server 2012, Windows XP, Windows 7, Windows 8, Windows Server 2003 R2, Windows Server 2008 R2, Windows Server 2012 R2, Windows Vista, Windows 8.1\n\nData Sources: API monitoring, Process command-line parameters, Process monitoring, PowerShell logs",
    "kill_chain_phases": [
        {
            "kill_chain_name": "mitre-attack",
            "phase_name": "credential-access"
        }
    ],
    "external_references": [
        {
            "source_name": "mitre-attack",
            "url": "https://attack.mitre.org/wiki/Technique/T1003",
            "external_id": "T1003"
        },
        {
            "source_name": "Github Mimikatz Module sekurlsa",
            "description": "Delpy, B. (2014, September 14). Mimikatz module ~ sekurlsa. Retrieved January 10, 2016.",
            "url": "https://github.com/gentilkiwi/mimikatz/wiki/module-~-sekurlsa"
        },
        {
            "source_name": "Powersploit",
            "description": "PowerSploit. (n.d.).  Retrieved December 4, 2014.",
            "url": "https://github.com/mattifestation/PowerSploit"
        }
    ],
    "object_marking_refs": [
        "marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168"
    ]
}
[10]:
from stix2 import Filter

# create filter for type=malware
query = [Filter("type", "=", "malware")]

# query on the filter
mals = fs_source.query(query)

for mal in mals:
    print(mal.id)
[10]:
malware--92ec0cbd-2c30-44a2-b270-73f4ec949841
[10]:
malware--b42378e0-f147-496f-992a-26a49705395b
[10]:
malware--96b08451-b27a-4ff6-893f-790e26393a8e
[10]:
malware--6b616fc1-1505-48e3-8b2c-0d19337bff38
[10]:
malware--6b616fc1-1505-48e3-8b2c-0d19337bff38
[10]:
malware--6b616fc1-1505-48e3-8b2c-0d19337bff38
[10]:
malware--6b616fc1-1505-48e3-8b2c-0d19337bff38
[11]:
# add more filters to the query
query.append(Filter("modified", ">" , "2017-05-31T21:33:10.772474Z"))

mals = fs_source.query(query)

# for visual purposes
for mal in mals:
    print(mal.id)
[11]:
malware--92ec0cbd-2c30-44a2-b270-73f4ec949841
[11]:
malware--6b616fc1-1505-48e3-8b2c-0d19337bff38
[11]:
malware--6b616fc1-1505-48e3-8b2c-0d19337bff38
[11]:
malware--6b616fc1-1505-48e3-8b2c-0d19337bff38

FileSystemSink

Use the FileSystemSink when you only want to push STIX content to the file system.

[13]:
from stix2 import FileSystemSink, Campaign, Indicator

# create FileSystemSink
fs_sink = FileSystemSink("/tmp/stix2_sink")

# create STIX objects and add to sink
camp = Campaign(name="The Crusades",
                objective="Infiltrating Israeli, Iranian and Palestinian digital infrastructure and government systems.",
                aliases=["Desert Moon"])

ind = Indicator(description="Crusades C2 implant",
                pattern_type="stix",
                pattern="[file:hashes.'SHA-256' = '54b7e05e39a59428743635242e4a867c932140a999f52a1e54fa7ee6a440c73b']")

ind1 = Indicator(description="Crusades C2 implant",
                 pattern_type="stix",
                 pattern="[file:hashes.'SHA-256' = '54b7e05e39a59428743635242e4a867c932140a999f52a1e54fa7ee6a440c73b']")

# add Campaign object to FileSystemSink
fs_sink.add(camp)

# can also add STIX objects to FileSystemSink in one call
fs_sink.add([ind, ind1])