Workflow to Flow Designer Migration using Integrations Capabilities framework 2.0 The goal of this Knowledge Base is to give an idea on how to migrate from Workflows to Flows, using the Integration Capabilities Framework 2.0 Contents: What is Capabilities framework 2.0?Improvements in Capabilities Framework 2.0 over the previous framework using workflowsAdvantages of using Capabilities framework 2.0Need for migration to Capabilities framework 2.0Timelines for complete migration to Capabilities framework 2.0Steps to migrate from Workflows to Flows using Capability Implementation Framework V2Example of Implementing "Have I been Pwned" integration for "Threat Lookup" using Capabilities implementation framework 2.0Testing the newly created flowBackward compatibility What is Capabilities framework 2.0? The new Capabilities Framework 2.0 has been developed to enable implementation of capabilities (like Threat lookup, Observable Enrichment, etc..) in a simple and consistent manner.This ensures a consistent experience across different implementations of the same capability. Improvements in Capabilities Framework 2.0 over the previous framework using workflows : Previously while using the workflows, in order to perform the capability look up, one has to do the following for every capability implementation Filter unsupported observablesApply rate limitsFetch the data from 3rd party sources and persist the results.Update capability execution record Ex: "Have I been Pwned" capability implementation for Threat lookup using workflows. Using the Capability implementation framework, most of the above tasks like rate limiting, filtering, updating execution records are taken care by the framework. One needs to just Fetch the data from 3rd party source and persist the result in the capability implementation flow. Ex: "Have I been Pwned" capability implementation for Threat lookup using Flows and new Capabilities Framework. Advantages of using Capabilities framework 2.0: The capability flows will now include only business level components without any implementation specific logic.The capability flows now accept a broad array of inputs and formats for maximum flexibility (For example, observable references, CI references, tasks, any table or sys_id combinations).Rate limiting or throttling on integration executions are now easy to configure (removing the need to do this using custom code or changes to implementation workflows).Enhanced auditing and execution tracking capabilities now enable better reporting and easier troubleshooting.Robust error handling functions are built into the capability flows to avoid duplicating implementation routines.Ability to configure conditional triggering of the capabilities or the integrations. This provides flexibility to automatically launch automations based on incident category.A default filter condition has been introduced on all capabilities to filter allow listed observables before inputs are provided to the integrations. Need for migration to Capabilities framework 2.0: Currently the capabilities are using a mixture of both the newer Capabilities framework 2.0 and the older framework using the workflows.To bring consistent behaviour and similar user experience across all the capabilities and their implementations, we are migrating completely to the latest Capabilities framework 2.0. Timelines for complete migration to Capabilities framework 2.0: All the workflow based implementations will be completely moved to the latest capabilities framework 2.0 by 1st Feb 2025.Support for migration of all the workflow based implementations of capabilities will be only till 1st Feb 2025. For complete documentation of Integration Capabilites Framework 2.0 please refer to the Integrations Capabilities framework 2.0 documentation here. Steps to migrate from Workflows to Flows using Capability Implementation Framework V2: 1. Create sub-flow for capability implementation: Create a new sub-flow in flow designer for the capability implementation with following inputs and outputs. Inputs for the sub-flow: Label Name Type Input Parameters input_parameters JSON Task task Reference.Task Capability Implementation Execution capability_implementation_execution Reference.Capability Implementation Execution Domain Id domain_id Domain ID Outputs for the sub-flow: Label Name Type Results results JSON Status Message status_message String State state Choice 2. From the input parameters, get the records from the corresponding table. The input parameters are always in this format: {"tableName":[sys_id1,sys_id2,…]}. 3. Send request to the external system and fetch the results. 4. Persist the results obtained from external system. For the OOB capabilities, the results are persisted in the following tables. These tables can be used to persist the results while implementing same capability for different providers. Capability Results Table Threat Lookup Threat Lookup Result (sn_ti_lookup_result) Enrich Observable Observable Enrichment Result (sn_ti_observable_enrichment_result) Sightings Search Sightings Search Results (sn_ti_sighting_search) 5. Assign the sub-flow outputs. 2. Create Capability Implementation record for the flow created above: 1. Create a new record in Capability Implementation table (sn_sec_cmn_capability_implementation). The details to be filled in each field are as follows: Field Name Description Active Select this check box to enable/disable this integration. Note: If you configure this integration using the integration tile in the Security Operations > Integrations > Integrations Configurationspage, this flag is automatically set to Active. Order Indicates the order in which the integrations are executed. Capability The capability implemented by this integration. Flow The sub flow that implements the capability. The sub flow created in step-1 should be passed here. Configuration The integration configuration for this capability. Note: This is initially set to the default configuration provided with the base system. When an integration is configuring using the integration tile in the Integration Configurations page, this value is automatically reset to the new configuration created. Rate Limit Indicates the number of integrations that can be executed at run-time (in parallel or per unit of time). You can choose the default rate limiters provided or create new ones as per the requirement. Batch Inputs Size The batch input size for each execution. Ex: for a Sighting Search integration you may want to group the observables into batches of 50 so that the queries generated do not become too large. 0 indicates that there is no limit. Timeout Period The maximum duration before the capability implementation flow is cancelled. 0 indicates that there is no timeout period. Total Requests The maximum number of implementation execution requests that are allowed per period. This field in conjunction with the Total Reqs Period field, can be used to limit the number of requests to the service. For example, you can limit the number to 4 requests per minute. Total Reqs Period The time period in which the implementation executions are to be considered for rate limiting. This in conjunction with Total Requests can be used to limit the number of requests to the service. Retry Limit The number of retries allowed for a failed execution request. This limit will be applicable if the Retry flag is set in your integration to retry an execution request when a condition is met. For example, a retry request is made when you have exceeded your license limit for that service for a time period or the service is down. Retry After The period after which an attempt is made to retry a failed execution request. Max Concurrent Reqs The maximum number of concurrent implementation execution requests. 0 indicates no limit. Sighting Search Configurations The default sighting search queries that can be executed. ( Needed only for Sighting Search capability and can be ignored for rest of other capabilities.) Add the filters, in the "Filter Conditions" related list of capability implementation record to filter out entities at Capability implementation level. To add a new filter create new record in Filter Conditions table (sn_sec_cmn_filter_condition) and select the above created record as capability implementation record. The details to be filled in each field are as follows: Field Name Description Active Select this check box to enable the filter. Order Indicates the order in which the filters are executed. Lower the order, it gets executed first. Capability The capability for which the filter needs to be applied. This field is used to apply filter at capability level to remove entities which are not supported by the capability. Capability implementation The capability implementation for which the filter needs to be applied. This field is used to apply filter at capability implementation level to remove entities which are not supported by the capability implementation. Name Name of the filter Optional Select this checkbox if this is an optional filter. Add worknote to task Select this checkbox if a worknote needs to be added to task if any of the inputs are filtered out. Description Description of the filter. Conditions Use this field to create conditions to filter out the unwanted entities. Only the entities which are satisfying the conditions will be sent for further processing. Evaluation Script Use this field to create complex filter conditions using a script. If the output of the script is true for any entity then the entity will be filtered out and will not be sent for further processing. Note: a. Currently in the present latest version of "Security Support Common" plugin, both "Capability" and "Capability Implementation" fields are marked as "read-only". Please use admin overrides or background scripts to override these fields. In the newer releases of the plugin, the issue will be resolved. b.These are exclusive filter type i.e if the filter returns true, the value is filtered and will not be sent for further processing. Example of Implementing "Have I been Pwned" integration for "Threat Lookup" using Capabilities implementation framework 2.0 : Create a new Sub flow with required inputs and outputs with Name "Security Operation 'Have I been pwned?' Integration V1".Create an action to get observable records from input parameters. Note: The input parameters are in the following format - input_parameters = {table_name: [sys_id1, sys_id2, sys_id3, …]} Send request to "Have I been pwned" system and persist the results in Threat Lookup Results table (sn_ti_threat_lookup).Assign sub flow outputs based on the response received. The sub-flow implementing "Have I been Pwned" integration: Create a new record in Capability Implementation (sn_sec_cmn_capability_implementation) with Name "Have I been pwned?".Fill all the other required details in the record as mentioned above and save the record. To add filters create new records in Filter Conditions table (sn_sec_cmn_filter_condition) and select the above created capability implementation record in capability implementation field. Once the record is selected the filter appears on the "Filter conditions" releated list of capability implementation record. Note: Currently in the present latest version of "Security Support Common" plugin, both "Capability" and "Capability Implementation" fields are marked as "read-only". Please use admin overrides or background scripts to override these fields. In the newer releases of the plugin, the issue will be resolved. In the above filter, only those observables whose type is not email-address and URL will pass the filter conditions. Testing the newly created flow: Currently some of the capabilities like threat lookup trigger both capability framework V1(workflows) and capability framework V2 (flows). However there are few capabilities which only trigger workflows i.e. capability framework V1. When you try to test the newly created flow-based implementation for a capability implementation via an existing UI action, you might sometimes observe that the flow is not getting triggered. This issue will continue to exist until the migration is done completely. However you can test the functionality and working of the newly created flow-based capability implementation by triggering the parent flow for the capability. By early 2025, the migration will be complete and we will stop triggering the workflows from then. Backward Compatilbility: With the latest versions (released on or after January 1st 2025) of Security Operations plugins, the workflows for capability implementations do not get triggered.To ensure backward compatibility and to continue using the capability implementation workflows, please use this update set.Also ensure that the corresponding integration capability implementation record in the "Integration Capability Implementation" table is active and configured as shown in the below screenshot.