Threat Intelligence Security Center Entities Aggregation Logic - 2.0<!-- /*NS Branding Styles*/ --> .ns-kb-css-body-editor-container { p { font-size: 12pt; font-family: Lato; color: var(--now-color--text-primary, #000000); } span { font-size: 12pt; font-family: Lato; color: var(--now-color--text-primary, #000000); } h2 { font-size: 24pt; font-family: Lato; color: var(--now-color--text-primary, black); } h3 { font-size: 18pt; font-family: Lato; color: var(--now-color--text-primary, black); } h4 { font-size: 14pt; font-family: Lato; color: var(--now-color--text-primary, black); } a { font-size: 12pt; font-family: Lato; color: var(--now-color--link-primary, #00718F); } a:hover { font-size: 12pt; color: var(--now-color--link-primary, #024F69); } a:target { font-size: 12pt; color: var(--now-color--link-primary, #032D42); } a:visited { font-size: 12pt; color: var(--now-color--link-primary, #00718f); } ul { font-size: 12pt; font-family: Lato; } li { font-size: 12pt; font-family: Lato; } img { display: ; max-width: ; width: ; height: ; } } Aggregation Logic Updates — Threat Intelligence Security Center v4.2.0 (March Store Release) Types of Field Level Aggregation: All Unique Values: All unique values for a field across different sources are consolidated in the aggregated record field Eg: If tag of sourceRecord1 is t1 and tag of sourceRecord2 is t2. Then Aggregated record tag value would be t1,t2 Value from source with highest confidence and latest to report: If there are two source records which have been aggregated to the same parent record , then for a field value on the aggregated record , Value from the source which has the highest confidence in combination with latest reported time is persisted on to the record Eg: If for sourceRecord1 has a confidence of 40 and reported time as 04-01-2024 and for sourceRecord2 has a confidence of 75 and reported time as 02-01-2024. On the basis of this Description Field sourceRecord2 is considered and its value is persisted to aggregated record If confidence is same for the source records If for sourceRecord1 has a confidence of 90 and reported time as 04-01-2024 and for sourceRecord2 has a confidence of 90 and reported time as 02-01-2024. On the basis of this Description Field sourceRecord1 is considered and its value is persisted to aggregated record as it is the latest to report Latest of Sources: The latest value for a particular field among all the sources is considered and set to the field of the aggregated record Eg: If sourceRecord1 has first seen field timestamp as 02-01-2024 and sourceRecord2 has first seen field timestamp as 15-12-2023. Then the value from sourceRecord1 Is persisted onto the aggregated record Last of Sources: The latest value for a particular field among all the sources is considered and set to the field of the aggregated record Eg: If sourceRecord1 has last seen field timestamp as 02-01-2024 and sourceRecord2 has last seen field timestamp as 15-12-2023. Then the value from sourceRecord1 Is persisted onto the aggregated record Value from first source record: Value of a field from the first source record that is aggregated to the parent is set as the value for the field on the aggregated record Eg: If sourceRecord1 has additional information and sourceRecord2, sourceRecord3 also contain additional information. Then Aggregated record additional information value would be set from sourceRecord1 as it is the first record. Sum of sources: Value of aggregated record is calculated as a collective sum of all the field values of different source records Eg: If for sourceRecord1 has a sighting count as 2 , sourceRecord2 has a sighting count as 5 ,sourceRecord3 has a sighting count as 10 . Aggregated sighting count field Value would be 17. Highest of all sources: Value of the field in aggregated record is set as the highest of the values from all sources. Eg: If for sourceRecord1 has a confidence as 45 , sourceRecord2 has a confidence as 50 . Aggregated confidence field Value would be 50. Most restrictive of all sources: Value of aggregated record is set as most restrictive setting from among all the values of sources Eg: If for sourceRecord1 has a TLP as CLEAR , sourceRecord2 has a TLP as RED . Aggregated TLP field value would be RED. Common Aggregation Logic For all Entities: Aggregating Relationships and External references is common step while calculating filed values of aggregated record from source records. Aggregating Relationships: All the relationships that are ingested as part of source records will be aggregated and created as relationships between corresponding parent/Aggregated records. Aggregating External References: All the external reference that are related to the source records are aggregated as external references to corresponding parent record. Entity Field Wise Aggregation logic: Observable Source Aggregation Type of Aggregation Fields All Unique Values Source, Usage Categories, Attack Phases, Tags, Taxonomies, Authors Value from source with highest confidence and latest to report Confidence, Description, Threat Level, Threat Severity, Attributes,Additional Context Earliest of sources First Seen, First Observed Last of sources Last Seen, Last Observed, Expiration Time Value from first source record - Sum of sources - Highest of all sources Source Reported Score Most Restrictive of all sources Reputation, TLP Indicator Source Aggregation Type of Aggregation Fields All Unique Values Source, Usage Categories, Attack Phases, Tags, Taxonomies, Authors, Platforms, Indicator Types Value from source with highest confidence and latest to report Confidence, Description, Threat Level, Threat Severity, Pattern, Pattern Version, IOC Classification, Additional Context Earliest of sources First Seen, First Observed, First Detected, Valid From Last of sources Last Seen, Last Observed, Valid Until, Expiration Time Value from first source record - Sum of sources - Highest of all sources - Most Restrictive of all sources TLP Vulnerability Source Aggregation Type of Aggregation Fields All Unique Values Tags, Taxonomies, Affected Software, Source Value from source with highest confidence and latest to report Description, CVE ID, CVE Published Date, CNA (CVE Naming Authority), CNA Last Modified, Risk Rating, Vulnerability Class, Affected Software, CVSS 2.0 Base Score, CVSS 3.x Base Score, CVSS 4.0 Base Score, CVSS 2.0 Vector, CVSS 3.x Vector, CVSS 4.0 Vector, EPSS Score, EPSS Percentile, PoC exists, PoC State, Exploitation Status, First Known Exploit Date, Exploit skill level, Exploit attack vector, Is Zero Day, Known Ransomware Campaign Use, KEV Date Added, KEV Action Due Date, KEV Vendor Project, KEV Required Action, Dark Web Mentions, Social Media Mentions, Vulnerability Remediation Status, Technical Details, Revoked Date, Revoked Reason, Additional Context, Lang Earliest of sources - Last of sources - Value from first source record - Sum of sources - Highest of all sources Confidence Most Restrictive of all sources TLP Threat Event Source Aggregation Type of Aggregation Fields All Unique Values Tags, Taxonomies, Source Value from source with highest confidence and latest to report Description, Additional Context Earliest of sources - Last of sources - Value from first source record - Sum of sources - Highest of all sources Confidence Most Restrictive of all sources TLP Location Source Aggregation Type of Aggregation Fields All Unique Values Tags, Taxonomies, Source Value from source with highest confidence and latest to report Description, Latitude, Longitude , Precision, RegionCountry, Administrative Area, City, Street Address, Postal Code,Additional Context Earliest of sources - Last of sources - Value from first source record - Sum of sources - Highest of all sources Confidence Most Restrictive of all sources TLP Threat Opinion Source Aggregation Type of Aggregation Fields All Unique Values Tags, Taxonomies, Source, Authors Value from source with highest confidence and latest to report Explanation,Additional Context Earliest of sources - Last of sources - Value from first source record - Sum of sources - Highest of all sources Confidence Most Restrictive of all sources TLP Attack Pattern Source Aggregation Type of Aggregation Fields All Unique Values Tags, Taxonomies, Source, Attack Phases, Aliases, Permissions Required Value from source with highest confidence and latest to report Description,Additional Context Earliest of sources - Last of sources - Value from first source record - Sum of sources - Highest of all sources Confidence Most Restrictive of all sources TLP Identity Source Aggregation Type of Aggregation Fields All Unique Values Tags, Taxonomies, Source, Sectors, Roles, Contact Information Value from source with highest confidence and latest to report Description, Additional Context Earliest of sources - Last of sources - Value from first source record - Sum of sources - Highest of all sources Confidence Most Restrictive of all sources TLP Campaign Source Aggregation Type of Aggregation Fields All Unique Values Tags, Taxonomies, Source, Aliases, Objective, Extensions Value from source with highest confidence and latest to report Description, Additional Context Earliest of sources First Seen Last of sources Last Seen Value from first source record - Sum of sources - Highest of all sources Confidence Most Restrictive of all sources TLP Course of Action Source Aggregation Type of Aggregation Fields All Unique Values Tags, Taxonomies, Source, Aliases, External ID Value from source with highest confidence and latest to report Description, Action, Additional Context Earliest of sources - Last of sources - Value from first source record - Sum of sources - Highest of all sources Confidence Most Restrictive of all sources TLP Intrusion Set Source Aggregation Type of Aggregation Fields All Unique Values Tags, Taxonomies, Source, Aliases, Secondary Motivation, Goals Value from source with highest confidence and latest to report Description, Resource Level, Primary Motivation, Additional Context Earliest of sources First Seen Last of sources Last Seen Value from first source record - Sum of sources - Highest of all sources Confidence Most Restrictive of all sources TLP Threat Actor Source Aggregation Type of Aggregation Fields All Unique Values Tags, Taxonomies, Source, Aliases, Threat Actor Role, Threat Actor TypePersonal Motivation, Secondary Motivation, Goals Value from source with highest confidence and latest to report Description, Resource Level, Sophistication, Primary Motivation, Additional Context Earliest of sources First Seen Last of sources Last Seen Value from first source record - Sum of sources - Highest of all sources Confidence Most Restrictive of all sources TLP Threat Note Source Aggregation Type of Aggregation Fields All Unique Values Tags, Taxonomies, Source, Authors Value from source with highest confidence and latest to report Abstract,Additional Context Earliest of sources - Last of sources - Value from first source record - Sum of sources - Highest of all sources Confidence Most Restrictive of all sources TLP Tool Source Aggregation Type of Aggregation Fields All Unique Values Tags, Taxonomies, Source, Aliases, Tool Type, Attack Phases Value from source with highest confidence and latest to report Description, Tool Version,Additional Context Earliest of sources - Last of sources - Value from first source record - Sum of sources - Highest of all sources Confidence Most Restrictive of all sources TLP Object Sighting Source Aggregation Type of Aggregation Fields All Unique Values Tags, Taxonomies, Source Value from source with highest confidence and latest to report Description, Additional Context Earliest of sources First Seen Last of sources Last Seen Value from first source record - Sum of sources Count Highest of all sources Confidence Most Restrictive of all sources TLP Infrastructure Source Aggregation Type of Aggregation Fields All Unique Values Tags, Taxonomies, Source, Aliases, Attack Phases, Infrastructure Types Value from source with highest confidence and latest to report Description, Additional Context Earliest of sources First Seen Last of sources Last Seen Value from first source record - Sum of sources - Highest of all sources Confidence Most Restrictive of all sources TLP Observed Data Source Aggregation Type of Aggregation Fields All Unique Values Tags, Taxonomies, Source Value from source with highest confidence and latest to report Additional Context Earliest of sources - Last of sources - Value from first source record - Sum of sources - Highest of all sources Confidence Most Restrictive of all sources TLP Threat Grouping Source Aggregation Type of Aggregation Fields All Unique Values Tags, Taxonomies, Source Value from source with highest confidence and latest to report Name, Description, Additional Context Earliest of sources - Last of sources - Value from first source record - Sum of sources - Highest of all sources Confidence Most Restrictive of all sources TLP Threat Report Source Aggregation Type of Aggregation Fields All Unique Values Tags, Taxonomies, Source, Report Types Value from source with highest confidence and latest to report Description, Additional Context Earliest of sources - Last of sources - Value from first source record - Sum of sources - Highest of all sources Confidence Most Restrictive of all sources TLP Malware Source Aggregation Type of Aggregation Fields All Unique Values Tags, Taxonomies, Source, Aliases, Malware Types, Attack PhasesExecutable Process Architecture, Implementation Languages ,Malware Capabilities Value from source with highest confidence and latest to report Description, Additional Context Earliest of sources First Seen Last of sources Last Seen Value from first source record - Sum of sources - Highest of all sources Confidence Most Restrictive of all sources TLP Malware Analysis Source Aggregation Type of Aggregation Fields All Unique Values Tags, Taxonomies, Source, Modules Value from source with highest confidence and latest to report Version, Analysis Definition Version, Analysis Engine VersionConfiguration Version, Submitted, Analysis Started, Analysis Ended, Result, Result Name, Additional Context Earliest of sources - Last of sources - Value from first source record - Sum of sources - Highest of all sources Confidence Most Restrictive of all sources TLP Marking Definition Source Aggregation Type of Aggregation Fields All Unique Values Tags, Taxonomies, Source Value from source with highest confidence and latest to report Additional Context Earliest of sources - Last of sources - Value from first source record - Sum of sources - Highest of all sources Confidence Most Restrictive of all sources TLP Data Component Source Aggregation Type of Aggregation Fields All Unique Values Tags, Taxonomies, Source Value from source with highest confidence and latest to report Description, Additional Context Earliest of sources - Last of sources - Value from first source record - Sum of sources - Highest of all sources Confidence Most Restrictive of all sources TLP Data Source Source Aggrgeation Type of Aggregation Fields All Unique Values Tags, Taxonomies, Source, Platforms, Collection Layers, Contributors Value from source with highest confidence and latest to report Description, Additional Context Earliest of sources - Last of sources - Value from first source record - Sum of sources - Highest of all sources Confidence Most Restrictive of all sources TLP 1.Notes field is no longer updated during aggregation, allowing analysts to use it freely to capture relevant information without it being overwritten by incoming source data.2.Additional Context aggregation behavior has been updated — the value is now sourced from the highest confidence record (latest to report in case of a tie), replacing the previous behavior of taking the value from the first source record Older Version Aggregation Logic - KB1587758 Related KB Please refer to the following knowledge base articles for detailed information regarding other elements of TISC processing workflow: KB2920879: This KB article provides details regarding the De-duplication logic for entities in TISC, which happens prior to the Aggregation Phase in the processing layer. KB2920946: This KB article documents the unique identification keys used for different threat intelligence entities within the TISC system. These keys are utilized during the Parent Identification phase, which occurs prior to the De-duplication process.