MID Server throws fatal error EXCEPTION_ACCESS_VIOLATION and goes Down when using Java 8 Update 261 (JRE 1.8.0_261) - Known Error

MID Server throws fatal error EXCEPTION_ACCESS_VIOLATION and goes Down when using Java 8 Update 261 (JRE 1.8.0_261) Description Using Oracle Java 8 Update 261 (JRE 1.8.0_261) with a MID Server installation causes the MID Server to go down with a EXCEPTION_ACCESS_VIOLATION exception.

Potentially large memory dump (.mdmp) files will be left in the agent folder for each crash, that can have an impact on disk space of the whole server as well, eventually bringing the host server down too.

Oracle 1.8.0_251 does not cause the issue, and nor does the bundled OpenJDK 1.8.0_231.

Steps to Reproduce

Install a MID Server Edit wrapper-override.conf to use a standalone Java install of Oracle 1.8.0_251, or earlier, or leave it using the bundled 1.8.0_231 jre. Upgrade that java install to 1.8.0_261 The MID Server will go down on startup with an exception relating to sigar-amd64-winnt.dll EXCEPTION_ACCESS_VIOLATION.

The agent log may show:

07/23/20 03:48:50 (075) Collector SEVERE *** ERROR *** Sigar exception when attempting to instantiate Cpu org.hyperic.sigar.SigarException : Unknown OS Error at org.hyperic.sigar.Cpu.gather(Native Method) at org.hyperic.sigar.Cpu.fetch(Cpu.java:30) at org.hyperic.sigar.Sigar.getCpu(Sigar.java:320) at com.service_now.monitor.StatusMonitor$Collector.run(StatusMonitor.java:310) at java.util.TimerThread.mainLoop(Unknown Source) at java.util.TimerThread.run(Unknown Source) The wrapper log and hs_err_pidXXXX files in the agect folder will contain this error message:

# A fatal error has been detected by the Java Runtime Environment: # # EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x0000000010014ed4 , pid=6116, tid=0x0000000000001b7c # # JRE version: Java(TM) SE Runtime Environment ( 8.0_261 -b12) (build 1.8.0_261-b12) # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.261-b12 mixed mode windows-amd64 compressed oops) # Problematic frame: # C [ sigar-amd64-winnt.dll +0x14ed4] # # Core dump written. Default location: \agent\hs_err_pid6116.mdmp # # If you would like to submit a bug report, please visit: # http://bugreport.java.com/bugreport/crash.jsp # The crash happened outside the Java Virtual Machine in native code. # See problematic frame for where to report the bug. The pid and tid values will be dirrect each time. e.g.

# EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x00000000006d4ed4, pid=492, tid=0x00000000000016ac # Core dump written. Default location: \agent\hs_err_pid492.mdmp Once disk space on the server is used up, you may see this error in the wrapper log:

2020/07/24 13:00:47 | java.util.logging.ErrorManager: 2 2020/07/24 13:00:47 | java.io.IOException: There is not enough space on the disk 2020/07/24 13:00:47 | at java.io.FileOutputStream.writeBytes(Native Method) 2020/07/24 13:00:47 | at java.io.FileOutputStream.write(Unknown Source) 2020/07/24 13:00:47 | at java.io.BufferedOutputStream.flushBuffer(Unknown Source) 2020/07/24 13:00:47 | at java.io.BufferedOutputStream.flush(Unknown Source) 2020/07/24 13:00:47 | at java.util.logging.FileHandler$MeteredStream.flush(Unknown Source) 2020/07/24 13:00:47 | at sun.nio.cs.StreamEncoder.implFlush(Unknown Source) 2020/07/24 13:00:47 | at sun.nio.cs.StreamEncoder.flush(Unknown Source) 2020/07/24 13:00:47 | at java.io.OutputStreamWriter.flush(Unknown Source) 2020/07/24 13:00:47 | at java.util.logging.StreamHandler.flush(Unknown Source) 2020/07/24 13:00:47 | at java.util.logging.FileHandler.publish(Unknown Source) 2020/07/24 13:00:47 | at java.util.logging.Logger.log(Unknown Source) 2020/07/24 13:00:47 | at java.util.logging.Logger.doLog(Unknown Source) 2020/07/24 13:00:47 | at java.util.logging.Logger.log(Unknown Source) 2020/07/24 13:00:47 | at java.util.logging.Logger.info(Unknown Source) 2020/07/24 13:00:47 | at com.glide.util.Log.info0(Log.java:155) 2020/07/24 13:00:47 | at com.glide.util.Log.info(Log.java:134) 2020/07/24 13:00:47 | at com.snc.core_automation_common.logging.LoggerImpl.info(LoggerImpl.java:45) 2020/07/24 13:00:47 | at com.service_now.mid.services.Monitors.createMonitor(Monitors.java:228) 2020/07/24 13:00:47 | at com.service_now.mid.services.Monitors.loadInternalMonitor(Monitors.java:153) 2020/07/24 13:00:47 | at com.service_now.mid.services.Monitors.startFileSyncer(Monitors.java:141) 2020/07/24 13:00:47 | at com.service_now.mid.services.StartupSequencer.waitForInitialFileSync(StartupSequencer.java:332) 2020/07/24 13:00:47 | at com.service_now.mid.services.StartupSequencer.testsSucceeded(StartupSequencer.java:118) 2020/07/24 13:00:47 | at com.service_now.mid.services.StartupSequencer.access$300(StartupSequencer.java:76) 2020/07/24 13:00:47 | at com.service_now.mid.services.StartupSequencer$Starter.run(StartupSequencer.java:430) Workaround 1. If customer desires the security updates from newer versions of java, upgrade your JVM to use Java 11. It is important to note that this option will disable MID CPU monitoring functionality on Windows.

2. Downgrade your JVM to 8.0_251, or switch to using the bundled 1.8.0_231

You can contact ServiceNow Technical Support or subscribe to this Known Error article by clicking the Subscribe button at the top right of this form to be notified when more information will become available.