Extensions & Capabilities

• 1: Fin Python Library
• 2: Fin protocol
• 3: Native capabilities

1 - Fin Python Library

For the documentation about the Fin protocol we refer to documention page of SOARCA Fin Protocol.

Quick Start

To include the SOARCA Fin library, you can use the following command to install it via pip:

pip install soarca-fin-library

Example

An example on how to use the library is given below. For more examples and the source code, we will refer to the Github page of the SOARCA-Fin-python-library, where we provide /examples folder.

import os
from dotenv import load_dotenv

from soarca_fin_python_library.soarca_fin import SoarcaFin
from soarca_fin_python_library.models.agent_structure import AgentStructure
from soarca_fin_python_library.models.external_reference import ExternalReference
from soarca_fin_python_library.models.step_structure import StepStructure
from soarca_fin_python_library.models.capability_structure import CapabilityStructure
from soarca_fin_python_library.enums.workflow_step_enum import WorkFlowStepEnum
from soarca_fin_python_library.models.command import Command
from soarca_fin_python_library.models.result_structure import ResultStructure

from soarca_fin_python_library.models.variable import Variable
from soarca_fin_python_library.enums.variable_type_enum import VariableTypeEnum


def capability_pong_callback(command: Command) -> ResultStructure:
    print("Received ping, returning pong!")

    result = Variable(
        type=VariableTypeEnum.string,
        name="pong_output",
        description="If ping, return pong",
        value="pong",
        constant=True,
        external=False)

    context = command.command.context

    return ResultStructure(
        state="success", context=context, variables={"result": result})


def main(mqtt_broker: str, mqtt_port: int, username: str, password: str) -> None:

    finId = "soarca-fin--pingpong-f877bb3a-bb37-429e-8ece-2d4286cf326d"
    agentName = "soarca-fin-pong-f896bb3b-bb37-429e-8ece-2d4286cf326d"
    externalReferenceName = "external-reference-example-name"
    capabilityId = "mod-pong--e896aa3b-bb37-429e-8ece-2d4286cf326d"

    # Create AgentStructure
    agent = AgentStructure(
        name=agentName)

    # Create ExternalReference
    external_reference = ExternalReference(name=externalReferenceName)

    # Create StepStructure
    step_structure = StepStructure(
        name="step_name",
        description="step description",
        external_references=[external_reference],
        command="pong",
        target=agentName)

    # Create CapabilityStructure
    capability_structure = CapabilityStructure(
        capability_id=capabilityId,
        type=WorkFlowStepEnum.action,
        name="Ping Pong capability",
        version="0.0.1",
        step={
            "test": step_structure},
        agent={
            "testagent": agent})

    # Create Soarca fin
    fin = SoarcaFin(finId)
    # Set config for MQTT Server
    fin.set_config_MQTT_server(mqtt_broker, mqtt_port, username, password)
    # Register Capabilities
    fin.create_fin_capability(capability_structure, capability_pong_callback)
    # Start the fin
    fin.start_fin()


if __name__ == "__main__":
    load_dotenv()
    MQTT_BROKER = os.getenv("MQTT_BROKER", "localhost")
    MQTT_PORT = int(os.getenv("MQTT_PORT", "1883"))
    USERNAME = os.getenv("MQTT_USERNAME", "soarca")
    PASSWD = os.getenv("MQTT_PASSWD", "password")

    main(MQTT_BROKER, MQTT_PORT, USERNAME, PASSWD)

Below we have provided an example env file. Note that this changes according to your setup.

MQTT_BROKER = "localhost"
MQTT_PORT = "1883"
MQTT_USERNAME = "soarca"
MQTT_PASSWD = "password"

Env file can be exported by running:

export $(cat .env | grep -v "#" | xargs)

Architecture

The main object of the application is the SoarcaFin object, which is responsible for configuring and creating and controlling the capabilities. The SoarcaFin creates MQTTClients for each capability registered, plus one for registering, unregistering and controlling the fi itself. MQTTClients each have their own connection to the MQTT Broker and own Parser and Executor objects. The Parser object parsers the raw MQTT messages and tries to convert them to one of the objects in src/models. The Executor runs in their own thread and handles the actual execution of the messages. The Executor polls a thread-safe queue for new messages and performs IO operations, such as sending messages to the MQTT broker and calling capability callbacks.

Setup SOARCA Capabilities

To register a fin to SOARCA, first create a SoarcaFin object and pass the fin_id in the constructor. The SOARCA fin_id must be in the format of: sourca-fin-<capability>-<uuid4>. Call set_config_MQTT_server() to set the required configurations for the fin to connect to the MQTT broker. For each capability to be registered, call create_fin_capability(). The capability callback funtion should return an object of type ResultStructure. When all capabilities are initialized, call start_fin() for the SOARCA Fin to connect to the MQTT broker and register itself to SOARCA.

An example is given in this project in the file [examples/pong_example.py]

Class Overview

interface IParser {
 Message parse_on_message()
}

interface IMQTTClient {
 void on_connect()
 void on_message()
}

interface ISoarcaFin {
 void set_config_MQTTServer()
 void set_fin_capabilities()
 void start_fin()
}

interface IExecutor {
 void queue_message()
}


class SoarcaFin
class MQTTClient
class Parser
class Executor

ISoarcaFin <|.. SoarcaFin
IMQTTClient <|.. MQTTClient
IParser <|.. Parser
IExecutor <|.. Executor

IMQTTClient <- SoarcaFin
MQTTClient -> IExecutor
IParser <-MQTTClient

Sequence Diagrams

Command

Soarca -> "MQTTClient (Capability 1)" : Command Message [Capability ID Topic]

"MQTTClient (Capability 1)" -> Parser : parse_on_message(message)
"MQTTClient (Capability 1)" <-- Parser : Message.Command

"MQTTClient (Capability 1)" -> "Executor (Capability 1)" : Command message
Soarca <-- "Executor (Capability 1)" : Ack

"Executor (Capability 1)" -> "Capability Callback" : Command
"Executor (Capability 1)" <-- "Capability Callback" : Result


Soarca <- "Executor (Capability 1)" : Result
Soarca --> "MQTTClient (Capability 1)" : Ack

"MQTTClient (Capability 1)" -> Parser : parse_on_message(message)
"MQTTClient (Capability 1)" <-- Parser : Message.Ack

"MQTTClient (Capability 1)" -> "Executor (Capability 1)" : Ack message

Register

Soarca -> Soarca : Create Soarca Topic

Library -> SoarcaFin : Set MQTT Server config

Library -> SoarcaFin : Set Capability1
SoarcaFin -> "MQTTClient (Capability 1)" : Create capability

Library -> SoarcaFin : Set Capability2
SoarcaFin -> "MQTTClient (Capability 2)" : Create capability


Library -> SoarcaFin : Start Fin


SoarcaFin -> "MQTTClient (Capability 1)" : Start capability
"MQTTClient (Capability 1)" -> "MQTTClient (Capability 1)" : Register Capability Topic
SoarcaFin -> "MQTTClient (Capability 2)" : Start capability
"MQTTClient (Capability 2)" -> "MQTTClient (Capability 2)" : Register Capability Topic

SoarcaFin -> "MQTTClient (Fin)" : Register Fin
"MQTTClient (Fin)" -> "MQTTClient (Fin)" : Register SoarcaFin Topic

"MQTTClient (Fin)" -> "Executor (Fin)" : Send Register Message

Soarca <- "Executor (Fin)" : Message.Register [Soarca Topic]

Soarca --> "MQTTClient (Fin)" :  Message.Ack [Fin ID Topic]

"MQTTClient (Fin)" -> "Parser (Fin)" : parse_on_message(ack)
"MQTTClient (Fin)" <-- "Parser (Fin)" : Message.Ack

"MQTTClient (Fin)" -> "Executor (Fin)" : Message.Ack

Bugs or Contributing

Want to contribute to this project? It is possible to contribute here. Have you found a bug or want to request a feature? Please create an issue here.

2 - Fin protocol

Goals

The goal of the protocol is to provide a simple and robust way to communicate between the SOARCA orchestrator and the capabilities (Fins) that can provide extra functions.

MQTT

To allow for dynamic communication MQTT is used to provide the backbone for the fin communication. SOARCA can be configured using the environment to use MQTT or just run stand-alone.

The Fin will use the protocol to register itself to SOARCA via the register message. Once register, it will communicate over the channel new channel designated by the fin UUID.

Commands to a specific capability will be communicated of the capability UUID channel.

Messages

Messages defined in the protocol

• ack
• nack
• register
• unregister
• command
• pause
• resume
• stop

legend

|field |content |type |description |field name have the (optional) key if the field is not required |content indication |type of the value could be string, int etc. |A description for the field to provide extra information and context

ack

The ack message is used to acknowledge messages.

@startjson
{
    "type": "ack",
    "message_id": "uuid"
}
@endjson

nack

The nack message is used to non acknowledgements, message was unimplemented or unsuccessful.

@startjson
{
    "type": "nack",
    "message_id": "uuid"
}
@endjson

register

The message is used to register a fin to SOARCA. It has the following payload.

capability structure

step structure

agent structure

@startjson
{
    "type": "register",
    "message_id": "uuid",
    "fin_id" : "uuid",
    "name": "Fin name",
    "protocol_version": "<semantic-version>",
    "security": {
        "version": "0.0.0",
        "channel_security": "plaintext"
    },
    "capabilities": [
        {
            "capability_id": "uuid",
            "name": "ssh executer",
            "version": "0.1.0", 
            "step": { 
                "type": "action",
                "name": "<step_name>",
                "description": "<description>",
                "external_references": { 
                    "name": "<reference name>",
                    "...": "..."
                    },
                "command": "<command string example>",
                "target": "<target uuid>"
            },
            "agent" : {
                "soarca-fin--<uuid>": {
                    "type": "soarca-fin",
                    "name": "soarca-fin--<name>-<capability_uuid>"
                }
            }

        }
    ],
    "meta": {

        "timestamp": "string: <utc-timestamp-nanoes + timezone-offset>",
        "sender_id": "uuid"
    }
}
@endjson

unregister

The message is used to unregister a fin to SOARCA. It has the following payload.

@startjson
{
    "type": "unregister",
    "message_id": "uuid",
    "capability_id" : "capability uuid",
    "fin_id" : "fin uuid",
    "all" : "true | false"
}
@endjson

command

The message is used to send a command from SOARCA. It has the following payload.

command substructure

@startjson
{
    "type": "command",
    "message_id": "uuid",
    "command": {
        "command": "command",
        "authentication": {"auth-uuid": "<cacao authentication struct"},
        "context": {
            "generated_on": "string: <utc-timestamp-nanoes + timezone-offset>",
            "timeout": "string: <utc-timestamp-nanoes + timezone-offset>",
            "step_id": "uuid",
            "playbook_id": "uuid",
            "execution_id": "uuid"
        },
        "variables": {
            "__<var1>__": {
                "type": "<cacao.variable-type-ov>",
                "name": "__<var1>__",
                "description": "<string>",
                "value": "<string>",
                "constant": "<bool>",
                "external": "<bool>"
            },
            "__<var2>__": {
                "type": "<cacao.variable-type-ov>",
                "name": "__<var2>__",
                "description": "<string>",
                "value": "<string>",
                "constant": "<bool>",
                "external": "<bool>"
            }
        }
    },
    "meta": {
        "timestamp": "string: <utc-timestamp-nanoes + timezone-offset>",
        "sender_id": "uuid"
    }
}
@endjson

result

The message is used to send a response from the Fin to SOARCA. It has the following payload.

result structure

@startjson
{   
    "type": "result",
    "message_id": "uuid",
    "result": {
        "state": "enum(success | failure)",
        "context": {
            "generated_on": "string: <utc-timestamp-nanoes + timezone-offset>",
            "timeout": "string: <utc-timestamp-nanoes + timezone-offset>",
            "step_id": "uuid",
            "playbook_id": "uuid",
            "execution_id": "uuid"
        },
        "variables": {
            "__<var1>__": {
                "type": "<cacao.variable-type-ov>",
                "name": "__<var1>__",
                "description": "<string>",
                "value": "<string>",
                "constant": "<bool>",
                "external": "<bool>"
            },
            "__<var2>__": {
                "type": "<cacao.variable-type-ov>",
                "name": "__<var2>__",
                "description": "<string>",
                "value": "<string>",
                "constant": "<bool>",
                "external": "<bool>"
            }
        }
    },
    "meta": {
        "timestamp": "string: <utc-timestamp-nanoes + timezone-offset>",
        "sender_id": "uuid"
    }
}
@endjson

control

pause

The message is used to halt the further execution of the Fin. The following command will be responded to with a nack, unless it is resumed or stopped.

@startjson
{
    "type": "pause",
    "message_id" : "uuid",
    "capability_id": "uuid"
}
@endjson

resume

The message is used to resume a paused Fin, the response will be an ack if ok or a nack when the Fin could not be resumed.

@startjson
{
    "type": "resume",
    "message_id" : "uuid",
    "capability_id": "uuid"
}
@endjson

stop

The message is used to shut down the Fin. this will be responded to by ack, after that there will follow an unregister.

@startjson
{
    "type": "stop",
    "message_id" : "uuid",
    "capability_id": "uuid"
}
@endjson

progress

Ask for the progress of the execution of the

@startjson
{
    "type": "progress",
    "message_id" : "uuid",
    "capability_id": "uuid"
}
@endjson

Status response

Report the progress of the execution of the capability

@startjson
{
    "type": "status",
    "message_id" : "uuid",
    "capability_id": "uuid",
    "progress": "<execution status>"
}
@endjson

Common

These contain command parts that are used in different messages.

Security

@startjson
{
    "security": {
        "version": "0.0.0",
        "channel_security": "plaintext"
    }
}
@endjson

Variables

Variables information structure

@startjson
{
    "__<var1>__": {
        "type": "<cacao.variable-type-ov>",
        "name": "<string>",
        "description": "<string>",
        "value": "<string>",
        "constant": "<bool>",
        "external": "<bool>"
        }
}
@endjson

Context

CACAO playbook context information structure

@startjson
{
    "context": {
        "completed_on": "string: <utc-timestamp-nanoes + timezone-offset>",
        "generated_on": "string: <utc-timestamp-nanoes + timezone-offset>",
        "timeout": "string: <utc-timestamp-nanoes + timezone-offset>",
        "step_id": "uuid",
        "playbook_id": "uuid",
        "execution_id": "uuid"
    }
}
@endjson

Meta

Meta information for the fin protocol structure

@startjson
{
    "meta": {
        "timestamp": "string: <utc-timestamp-nanoes + timezone-offset>",
        "sender_id": "uuid"
    }
}
@endjson

Sequences

Registering a capability

@startuml

participant "SOARCA" as soarca
participant Capability as fin

soarca -> soarca : create [soarca] topic

fin -> fin : create [fin UUID] topic
soarca <- fin : [soarca] register
soarca --> fin : [fin UUID] ack 

@enduml

Sending command

@startuml

participant "SOARCA" as soarca
participant Capability as fin

soarca -> fin : [capability UUID] command
soarca <-- fin : [capability UUID] ack 

.... processing .... 

soarca <- fin : [capability UUID] result
soarca --> fin: ack

@enduml

Unregistering a capability

@startuml

participant "SOARCA" as soarca
participant Capability as fin
participant "Second capability" as fin2

... SOARCA initiate unregistering one fin ...

soarca -> fin : [SOARCA] unregister fin-id
soarca <-- fin : [SOARCA] ack 
note right fin2
    This capability does not respond to this message
end note

... Fin initiate unregistering ...

soarca <- fin : [SOARCA] unregister fin-id
soarca --> fin : [SOARCA] ack 
note right fin2
    This capability does not respond to this message
end note

... SOARCA unregister all ...

soarca -> fin : [SOARCA] unregister all == true
soarca <-- fin : [SOARCA] ack 
soarca <-- fin2 : [SOARCA] ack
note over soarca, fin2
    soarca will go down after this command
end note
@enduml

Control

@startuml

participant "SOARCA" as soarca
participant Capability as fin


soarca -> fin : [fin UUID] control message
soarca <-- fin : [fin UUID] status 

@enduml

3 - Native capabilities

This page contains a list of capabilities that are natively implemented in SOARCA see details here. For MQTT-message-based capabilities, check here.

OpenC2 capability

The OpenC2 HTTP capability uses the http(s) transport layer as specified in OpenC2 HTTPS. It allows executing actions on an OpenC2-compatible security actuator.

CACAO documentation: OpenC2 HTTP Command

HTTP API capability

The HTTP capability allows sending arbitrary HTTP requests to other servers.

CACAO documentation: HTTP API Command

SSH capability

The SSH capability allows executing commands on systems running an SSH-server.

CACAO documentation: SSH Command