Expression cache memory leak caused when security tag is automatically applied to an observable based on the security tag rule which has a large amount of observable matched under it. - Known Error

Expression cache memory leak caused when security tag is automatically applied to an observable based on the security tag rule which has a large amount of observable matched under it. Description Root cause: The memory issue happens when a security tag is automatically applied to an observable based on the security tag rule which has a tons of observable matched under it.

Back Trace: When an observable is inserted or updated, the business rule 'Automatically apply security tags' triggers the script include 'SecurityTagRulesUtil', and then triggers the script include 'FilterGroupUtil'. The problematic method is 'getObservablesForRange' within script include 'FilterGroupUtil', it concatenates all the sys_ids of the matching observables, and this is where the memory is running out if there are thousands of matching records.

Steps to Reproduce

1. Go to 'sn_sec_cmn_security_tag_rule.list' 2. Create security tag rules that return large number of matching observables 3. Ensure the 'Use filter group' checkbox 4. Set the 'Table' as 'Observable[sn_ti_observable]' 5. Configure the condition according to the rule to return large number of maching records

Workaround To avoid this issue, we should not use filter group for security tag rules when you have tons of observables matching, we can configure the security tag rule to use plain query builder to avoid this issue:

1. Go to 'sn_sec_cmn_security_tag_rule.list' 2. Open up the problematic security tag rules to configure 3. Uncheck the 'Use filter group' checkbox 4. Set the 'Table' as 'Observable[sn_ti_observable]' 5. Configure the condition according to the rule as listed below:

- For 'Local IP Address Range: 10.0.0.0 - 10.255.255.255', set the condition to 'Observable type is IP address (V4)' AND 'Value starts with "10."'

- For 'Local IP Address Range: 172.16.0.0 - 172.31.255.255', set the condition to ''Observable type is IP address (V4)' AND ('Value matches regex (172.1[6-9].*)|(172.2[0-9].*)|(172.3[0-1].*)').

- For 'Local IP Address Range: 192.168.0.0 - 192.168.255.255', set the condition to 'Observable type is IP address (V4)' AND 'Value starts with "192.168."'

- For 'Loopback IP Address Range: 127.0.0.0 - 127.255.255.254', set the condition to 'Observable type is IP address (V4)' AND 'Value starts with "127."'