Audit - Frequently Asked Questions


 

The goal of this article is to answer generic frequent requests/questions ServiceNow Technical Support receives in relation to Audit and History. I if you have follow-up questions, please contact Technical Support.

FAQ

Time differences between "system fields" and audit recordsHow to audit Request item variables?After upgrading the instance from London to NewYork, em_alert table inclusion listed for audits
Archive related record delete actions logged in sys_audit_deleteWhen the audit is enabled on the sys_attachment table changes are not captured in the sys_audit history.Audit tables are taking a long time to load and the transaction gets canceled
How to audit when the user elevates their role?Long Field Names Shortened Before Being Stored in Audit.Can a variable be inclusion listed from auditing?
Alert audit history shows only work notesInclude Audit history during the cloneAudit History is not matching the Planned Start Date in Dev Environment
Why the activity stream (sys_audit / sys_history_line) is not capturing a specific state transition?How to enable or capture or add the field in Audit history table when the field is not auditedAudit history calendar will only show a max of 250 rows by default
Is it possible to activate auditing on the tables sc_cat_item and user_criteria?Enabling auditing for a table that is set to no_audit OOBTable column names that start with a number cause audit history viewing to fail
Excessive record Audit History Data can cause record inaccessibilityForm not loading: A sys_history_line / sys_audit field causes a StringIndexOutOfBoundsExceptionHistory for record insertion is not shown when audit have records with update count 0
How can the Activity Stream show a field displayed with a date which is before the field was added to the table?Why are journal entries made when a record was inserted missing from the activity stream?Audit history will not show audit information that was created before the record's sys_created_on date/time.
What does Update 0 Represent in the History Set?Reuse of GlideRecord object to insert and update causes incorrect recording of audit (History Line) data for initial and default values.

What is the meaning of audit records with the value DELETED or RECOVERED for the fieldname/oldvalue/newvalue fields on the sys_audit record?

 

Time differences between "system fields" and audit records

Time on System field (sys_created_on, sys_updated_on, etc) and the time on audit record will not be the same and it is expected behavior. The Date fields on the sys_audit record are populated with the time that the sys_audit record is created, so it will be a different value when compared to the sys_updated_on value in the actual record that is being audited.   

The update_time field in the sys_history_line records is populated using the sys_created_on value from the sys_audit records.  The sys_created_on time on the sys_history_line record reflects the time the history line was itself created and not the time that the audited change occurred.  Its for this reason the sys_created_on field is not visible out-of-the-box on the sys_history_line related list when viewing a history set.  Similarly the sys_updated_on and sys_updated_by fields on the sys_history_line records are never populated since any modification to the same field will be capture in a new sys_audit record and will generate its own new sys_history_line record.  

For the user information the user_name/user_id/user fields on sys_history_line capture who made the audited change, not the sys_updated_by field which is not populated.

 

How to audit Request item variables?

By setting the "glide.sc.audit.variables" property value to True, We can audit the variable on sc_req_item table.

https://.service-now.com/sys_properties_list.do?sysparm_query=nameSTARTSWITHglide.sc.audit.variables&sysparm_view=

After upgrading the instance from London to NewYork, em_alert table inclusion listed for audits?

After upgrade to NewYork, the em_alert dictionary got a new attribute 'audit_type=whitelist' (as part of the fix to PRB1312649 in Madrid Release). Only a few em_alert columns will be audited

If all fields are to be added, remove attribute 'audit_type=whitelist' from the em_alert table dictionary collection record.

Archive related record delete actions logged in sys_audit_delete

Archive rules can be configured to create additional rules for handling related records audits. Ex: Set up a rule to delete related records of a record that is being archived. If the record is deleted by these rules, it creates an entry in 'sys_audit_delete' unless 'no_audit_delete=true' is defined for its parent table(collection record in 'sys_dictionary') and this is expected behavior.

If no audit required for these deletes on a table, please review KB0725007 - Disable auditing of deleted records on a specific table

When the audit is enabled on the sys_attachment table changes are not captured in the sys_audit history.

Adding or removing attachments will not be tracked on the Task table, even if we enable audit for the sys_attachment table.

If needed such functionality can be done via customization. Community article - Attachment Logging/Auditing - Where did that attachment go?

Audit tables are taking a long time to load and the transaction gets canceled

Audit tables generally will have huge records and hence will take a lot of time to load the complete list

Use the below URL and add filters to fetch the records

https://<instance-name>.service-now.com/sys_audit_list.do?sysparm_filter_only=true

How to audit when the user elevates their role?

Audit when users elevate to the role with elevated privilege on the instance
Ex: elevate to security_admin role.

Instructions
1. When user elevates to the privileged role, the event is created with the event name "security.elevated_role.enabled" on sysevent table
2. Parm1 is the user_name of user and Parm2 is role name.
3. Can use the "Script Action" to process this event and audit the event as required.

Long Field Names Shortened Before Being Stored in Audit.

In the Audit table, the "field name" column does not allow more than 40 characters length by default because the type of this column is "Short Field Name". For longer field names, the name of the field is shortened before being stored.
It uses storage alias name from the sys_storage_alias table for that field.

Can a variable be excluded from auditing?

Auditing is enabled by default for service catalog variables. We can turn on/off auditing variables via system property: glide.sc.audit.variables. It is set to true by default.

We cannot turn on or off auditing for a variable individually.

Alert audit history shows only work notes

We can enable audit on required fields by adding 'audit=true' in Attributes of that sys_dictionary record.

Include Audit history during the clone

In Madrid and newer versions, the sys_audit table is not listed in the Excludes list by default. The users just have to uncheck the 'Exclude audit and log data' checkbox during the clone request submission.

In London and earlier versions,
- Remove the sys_audit table from the Exclude list of tables.
- Uncheck the 'Exclude audit and log data' checkbox while submitting the clone request.

Audit History is not matching the Planned Start Date in Dev Environment

If Audit was not copied over during Cloning, this kind of issues occur.
To check whether the Audit table is added in the Exclusion list.
- Go to Datacenter and check for the exclusion conditions on that Clone History of Change Request

{"ExcludeTablesSpecifiedInExclusionList":true,"ExcludeAuditAndLogData":true,"ExcludeLargeAttachmentData":true,"PreserveTheme":true,"TableExcluded": false}

Why activity stream (sys_audit / sys_history_line) is not capturing a specific state transition?

Ideally, the state field on the incident record is audited by default, and hence any state change on the incident record will be captured in the sys_audit table, then the same would be populated in the sys_history_line table, eventually, it would be populated in the activity stream on incident record.

All the state transitions were captured in the activity stream, but if only a specific state transition was not captured, we should check how the state is updated.If the business rule script has setWorkflow(false), then the record update will not be audited.

This is because setWorkflow(false) disables processing of all engines and the update does not appear in the audit history of the task.

How to enable or capture or add the field in the Audit history table when the field is not audited.

how to enable auditing for the table

Note: Please make to set a limit for the number of records being updated in the Audit tables! Generating Audit data for all incidents/fields could potentially cause performance issues.

Audit history calendar will only show a max of 250 rows by default

The max limit for audits show in the calendar view is 250. 

We can increase the limit in system property: glide.history.max_enteries.

Is it possible to activate auditing on the tables sc_cat_item and user_criteria?

[sc_cat_item] and [user_criteria] are extensively used tables and can generate a large number of updates and in turn, can have an impact on the instance performance.

Admin can enable auditing for these tables by changing the dictionary settings, however, this is not recommended.

Enabling auditing for a table that is set to no_audit OOB

The "no_audit" attribute ignores the true or false portion and checks only for its presence. Therefore, setting no_audit=false does not work, and still blocks auditing from the field. 

Instead, reconfigure the table to use Inclusion List Auditing instead of Exclusion List Auditing (no_audit). This way, admins are able to specify exactly which fields to audit, and it will not be overwritten in an upgrade. 

Table column names that start with a number cause audit history viewing to fail

Fields with a Field name starting with number causes XML errors resulting in viewing audit history for the table records to fail.

Example field names: 90_day, 278_to_days

Excessive record Audit History Data can cause record inaccessibility

Disable auditing by adding 'no_audit' to the attributes of fields with large amounts of audit data so that future records do not accumulate large amounts of audit data.

Form not loading: A sys_history_line / sys_audit field causes a StringIndexOutOfBoundsException

This issue occurs when a record has too many multibyte characters that are greater than 2 bytes

These kinds of scenarios occur when comments field containing a large email thread
Ex: Comments on an incident by hitting reply-all to an email, this causes massive audit entries and sometimes they can contain too many multibyte characters, which will prevent the form from loading. 

- The support team will help investigate and remove certain sys_audit records and the related sys_history_line records. Deleting these should fix the issue.  

History for record insertion is not shown when audit have records with update count 0

sys_audit will not have entries with update count =0.

Update count=0 means record insertions and they are not part of the audit.

If there is an entry in sys_audit with update count=0, it means audit has record insertion data which is not valid.

If found such type of entries in sys_audit, the history of the record will not be shown properly.

What Does Update 0 Represent in the History? / How can the Activity Stream show a field displayed with a date which is before the field was added to the table?

The majority of instances do not audit inserts as it increases the amount of database activity and the overall size of the sys_audit table significantly.  If the instance is auditing inserts then the audit records with update 0 provide the information for the history set, however for the majority of the instances the history lines for update 0 are generated based on the oldest audit for each field as explained below.

When the History Set is being built to be used in the construction of the Activity Stream history lines are generated to show the initial values for all the fields on the record, that is for update count=0, and will have a date/time which is the same as the sys_created_on of the record.

If there are audit records for a field then the old value from the field's oldest audit record is used for the field's initial value.   If there are no audit records for a field it is assumed the field's value has never changed since the record was created, and the current value is used for as the initial value for update count=0.  In both cases the history line will show the sys_created_on for the update count=0 value.

Therefore if a new field is added to a table, any new History Set will have history lines for update count=0 for the new field, showing the sys_created_on value for the record.

Putting it another way, when the initial values for update count=0 are being generated there is no logic to work out when a field was first added to a table, and therefore you can see fields in the activity stream which appear to pre-date the creation of the field.

Why are journal entries made when a record was inserted missing from the activity stream?

If a record is inserted with a journal field (e.g. comments) set to a value then there is no sys_audit record and only a sys_journal_field record inserted.  When the History Set is being built the earliest time of any audit for a journal is remembered and used to query sys_journal_field to find any records inserted before the first audited update to the field.   If there is an audited update to the same journal field at the same second so both sys_journal_field records have the same sys_created_on value, then only the audited value will be seen in the activity stream.   See the known error article KB0759130 for PRB653064.

 

Audit history will not show audit information that was created before the record's sys_created_on date/time.

When a History Set is built for a record, the platform queries sys_audit records which were created at or after the record's sys_created_on date/time up until the sys_updated_on date/time + 10 minutes.  If a record's sys_created_on value has been updated to a more recent date/time, then this will have the effect of excluding any earlier audit information from the history set. 

This could occur if the same record was imported from another system where it had a later sys_created_on value, or as part of some other import activity.  It is also possible to update the sys_created_on value of a record from Javascript, even specifying its value at the point a new record is inserted if GlideRecord.autoSysFields(false) is used to suppress the default behaviour of populating the sys_created_on, sys_created_by, sys_updated_on, sys_updated_by and sys_mod_count fields.  

Note that if the sys_mod_count on a record is lowered, which again could occur if an XML export of the record is taken and imported from a system where the record has a lower sys_mod_count, then this can also impact the ordering of the history, in the history calendar for example.

If this has occurred then an option would be to use the oldest sys_audit for the record to determine the oldest known sys_created_on value (also sys_created_by if that is different) and perform an update using GlideRecord.autoSysFields(false) to set the sys_created_on to that oldest value.  This should be thoroughly tested on a full clone of the affected instance if this will be attempted.

 

Reuse of GlideRecord object to insert and update causes incorrect recording of audit (History Line) data for initial and default values.

This is best explained with a simplified example, which is as follows:

var gr = new GlideRecord('incident');
gr.setValue('short_description', 'Audit test');
gr.setWorkflow(false); // This is only to suppress OOTB BRs which trigger if you have active=false
// since OOTB an instance does not audit inserts no sys_audit would have been produced.
gr.setValue('active', false); // Set active=false
gs.info(gr.insert()); // Output sys_id so the incident can be opened and the history reviewed
gr.setWorkflow(true); // Turn workflow back on as we need the audit to be generated
gr.setValue('active', true); // Set active=true
gr.update();

When this is run the history will show:

  1.  Old is empty, New is false
  2.  Old is empty, New is true

For update=1 the expectation would be the old value should be false, not empty.

This is expected behaviour and the script needs to be modified to re-query the newly inserted record:

var gr = new GlideRecord('incident');
gr.setWorkflow(false);
gr.setValue('active', false);
gs.info(gr.insert());
if (gr.get(gr.getUniqueValue()) { // Re-query the inserted record.
gr.setWorkflow(true);
gr.setValue('active', true);
gr.update();
}

When this is run the history will show:

  1.  Old is empty, New is false
  2.  Old is false, New is true

which is what is expected.

 

What is the meaning of audit records with the value DELETED or RECOVERED for the fieldname/oldvalue/newvalue?

These audit records record when a record in an audited table was deleted.  The deleted record should be found in the sys_audit_delete table and if the deleted record is recovered then an audit record will be created with the fieldname/oldvalue/newvalue set to RECOVERED.