Performance Analytics Data Collection Job fails with: "Error collecting for xxxxxxxx java.lang.NullPointerException at com.snc.pa.dc.Breakdown.getElements(Breakdown.java:96)


Description

A Performance Analytics Data Collection Job run shows "Collected with errors" and the error observed in Job Logs is this one:

Error collecting for 20190424 java.lang.NullPointerException
at com.snc.pa.dc.Breakdown.getElements(Breakdown.java:96)
at com.snc.pa.dc.DataCollector.map(DataCollector.java:630)
at com.snc.pa.dc.DataCollector.collect(DataCollector.java:387)
at com.snc.pa.dc.DataCollector.collect(DataCollector.java:295)
at com.snc.pa.dc.DataCollector.collect(DataCollector.java:229)
at com.snc.pa.dc.DataCollectorJob.collect(DataCollectorJob.java:291)
at com.snc.pa.dc.DataCollectorJob.collectWithMutex(DataCollectorJob.java:217)
at com.snc.pa.dc.DataCollectorJob.execute(DataCollectorJob.java:202)
at com.glide.schedule.JobExecutor.lambda$executeJob$171(JobExecutor.java:107)
at com.glide.schedule.JobExecutor.executeJob(JobExecutor.java:110)
at com.glide.schedule.JobExecutor.execute(JobExecutor.java:94)
at com.glide.schedule.GlideScheduleWorker.executeJob(GlideScheduleWorker.java:236)
at com.glide.schedule.GlideScheduleWorker.lambda$process$234(GlideScheduleWorker.java:165)
at com.glide.worker.TransactionalWorkerThread.executeInTransaction(TransactionalWorkerThread.java:35)
at com.glide.schedule.GlideScheduleWorker.process(GlideScheduleWorker.java:165)
at com.glide.schedule.GlideScheduleWorker.run(GlideScheduleWorker.java:75)

Cause

While a NullPointerException can have many root causes, in this case, the second line in the stack trace above provides a clue that it has something to do with a Breakdown.

To troubleshoot the issue, note the time of the error, and examine the application node log entries with the same timestamp. For the above error, the following was observed:

2019-04-25 11:44:13 (847) worker.4 worker.4 txid=c11cf255db49 WARNING *** WARNING *** Get for non-existent record: pa_dimensions:b832d9e3d7e00200b96d45a3ce6103bf, initializing 
...
2019-04-25 11:44:13 (973) worker.4 worker.4 txid=c11cf255db49 SEVERE *** ERROR *** Data Collector: Unexpected exception occurred java.lang.NullPointerException 

The warning message above is the key: "pa_dimensions" is the Breakdown Source table, and the Sys ID corresponds to a Breakdown Source, a missing one in this case. In this particular example, the Sys ID corresponds to the OOB Breakdown Source named "Incident.ContactType".  

Navigate to the list of Breakdowns (pa_breakdowns.list) and locate the Breakdown(s) that does not have a Breakdown Source. Open the XML payload for that Breakdown and confirm that the sys_id seen for the Breakdown Source ("dimension" in the XML) matches that one seen in the above warning, for example:

<dimensiondisplay_value="">b832d9e3d7e00200b96d45a3ce6103bf</dimension>

Resolution

Import the missing object from one of your sub-production instance to resolve the issue.