Add or replace Java Classes in the MID Server, without using the JAR File synchronisation from the instance - Support and Troubleshooting

Add or replace Java Classes in the MID Server, without using the JAR File synchronisation from the instance Summary .ns-kb-css-body-editor-container { p { font-size: 12pt; font-family: Lato; color: var(--now-color--text-primary, #000000); } span { font-size: 12pt; font-family: Lato; color: var(--now-color--text-primary, #000000); } h2 { font-size: 24pt; font-family: Lato; color: var(--now-color--text-primary, black); } h3 { font-size: 18pt; font-family: Lato; color: var(--now-color--text-primary, black); } h4 { font-size: 14pt; font-family: Lato; color: var(--now-color--text-primary, black); } a { font-size: 12pt; font-family: Lato; color: var(--now-color--link-primary, #00718F); } a:hover { font-size: 12pt; color: var(--now-color--link-primary, #024F69); } a:target { font-size: 12pt; color: var(--now-color--link-primary, #032D42); } a:visited { font-size: 12pt; color: var(--now-color--link-primary, #00718f); } ul { font-size: 12pt; font-family: Lato; } li { font-size: 12pt; font-family: Lato; } img { display: ; max-width: ; width: ; height: ; } } The out-of-box MID Server application includes a lot of java classes as .jar files in the .\agent\lib folder. This \lib folder should not be modified as a long term implementation, because it will be replaced each time the MID Servers upgrade after an instance patch or upgrade, deleting any additions or reverting changes.

Note: This approach was suggested for a log4j vulnerability in KB1001211 , but this could only be used temporarily before the next upgrade, or would need re-doing after every upgrade. This was OK that time because ServiceNow already had hotfixes ready to upgrade to that included this upgrade out-of-box shortly after, so that the next upgrade would be the fixed one.

The supported way to add extra java classes is by attaching .jar files to records in the instance JAR table [ecc_agent_jar]. These .jar files are automatically synchronised to the agent\extlib\ folder on all MID Servers.

Details of exactly how this synchronisation works is in: KB0852276 How MID Server File Synchronisation works - Troubleshooting guide

If you try to use this system to replace out-of-box java classes, the java classes in the jar files in \lib will still be used because the wrapper.conf file defines the class search order as \lib first, then \extlib:

# Java Classpath (include wrapper.jar) Add class path elements as needed starting from 1 wrapper.java.classpath.1=lib/*.jar wrapper.java.classpath.2=extlib/*.jar

To override that with the opposite order, you would need to edit the agent\conf\wrapper-override.conf file ( never wrapper.conf !), to add those lines at the end, in the opposite order:

wrapper.java.classpath.1=extlib/*.jar wrapper.java.classpath.2=lib/*.jar

Note: The JAR file from the instance record will be synchronised to all MID Servers, but only MID Servers that have had the wrapper-override.conf file manually edited will be using them in this situation.

That trick is the official workaround for this known problem with the out-of-box oracle driver agent\lib\ojdbc6.jar: PRB1433170/KB0862383 The jar files for Oracle/MSSQL/MySQL JDBC drivers on MID Servers are in the lib folder, not the JAR Files table, and so can't be swapped out for different versions to be compatible with all versions of those databases

Other limitations with this system include:

JAR files apply to all MID Servers . There is no feature to select specific MID Servers, like there is on credential records. (see closed enhancement request Idea - it didn't get enough upvotes this time round) JAR files apply to all Domain Separation Domains. The ecc_agent_jar table is not Domain Separated. (This might now not be true) Any changes to ecc_agent_jar records, such as inserting, activating/deactivating, or replacing the attachments, triggers re-synchronisation of the files to all MID Servers at the same time. This often involves 2 or 3 restarts to delete then replace then load the new file, because the file is locked due to having to be loaded on MID Server Startup. That will end up with all MID Servers down at the same time, defeating any failover cluster arrangement. (See PRB1452537/KB1157147 ) Potential use cases where none of the above is going to solve it:

Customer's Security policy prevents the use of the JAR File synchronisation records (ecc_agent_jar) in the instance at all. e.g. a credential resolver jar that includes passwords compiled within it, or restricted code, might not be allowed in the hosted cloud instance. MID Server specific classes are required. Perhaps a different credential resolver jar file, or JDBC driver version is needed on specific MID Servers only. It's the same java class name, but different MID Servers must use different versions of that class for their integrations. A security fix needs adding immediately, before you can upgrade to an instance version where ServiceNow have fixed it out-of-box, and it might be a while before there is a servicenow instance version with that upgraded class. Release .ns-kb-css-body-editor-container { p { font-size: 12pt; font-family: Lato; color: var(--now-color--text-primary, #000000); } span { font-size: 12pt; font-family: Lato; color: var(--now-color--text-primary, #000000); } h2 { font-size: 24pt; font-family: Lato; color: var(--now-color--text-primary, black); } h3 { font-size: 18pt; font-family: Lato; color: var(--now-color--text-primary, black); } h4 { font-size: 14pt; font-family: Lato; color: var(--now-color--text-primary, black); } a { font-size: 12pt; font-family: Lato; color: var(--now-color--link-primary, #00718F); } a:hover { font-size: 12pt; color: var(--now-color--link-primary, #024F69); } a:target { font-size: 12pt; color: var(--now-color--link-primary, #032D42); } a:visited { font-size: 12pt; color: var(--now-color--link-primary, #00718f); } ul { font-size: 12pt; font-family: Lato; } li { font-size: 12pt; font-family: Lato; } img { display: ; max-width: ; width: ; height: ; } } Any

Instructions .ns-kb-css-body-editor-container { p { font-size: 12pt; font-family: Lato; color: var(--now-color--text-primary, #000000); } span { font-size: 12pt; font-family: Lato; color: var(--now-color--text-primary, #000000); } h2 { font-size: 24pt; font-family: Lato; color: var(--now-color--text-primary, black); } h3 { font-size: 18pt; font-family: Lato; color: var(--now-color--text-primary, black); } h4 { font-size: 14pt; font-family: Lato; color: var(--now-color--text-primary, black); } a { font-size: 12pt; font-family: Lato; color: var(--now-color--link-primary, #00718F); } a:hover { font-size: 12pt; color: var(--now-color--link-primary, #024F69); } a:target { font-size: 12pt; color: var(--now-color--link-primary, #032D42); } a:visited { font-size: 12pt; color: var(--now-color--link-primary, #00718f); } ul { font-size: 12pt; font-family: Lato; } li { font-size: 12pt; font-family: Lato; } img { display: ; max-width: ; width: ; height: ; } } The following instructions show how to add a 3rd folder, to a specific MID Server, outside of the control of the MID Server or instance, yet still available to be used by probes running int he MID Server platform:

Create a new folder alongside the agent folder. It would need to be outside the agent folder to prevent Upgrades deleting it. e.g. if the MID Server is C:\MID_SERVER\prod_disco_mid\agent, you could create C:\MID_SERVER\prod_disco_mid\extlib2 Place the jar file in that new folder Edit agent\conf\wrapper-override.conf, (not wrapper.conf) adding these extra lines on the end: wrapper.java.classpath.1=lib/*.jar wrapper.java.classpath.2=extlib/*.jar wrapper.java.classpath.3=../extlib2/*.jar

Note: That order preserves the out-of-box 'lib' then 'extlib' order, but adds your additional folder as a 3rd path. This assumes the jar file you are adding is an extra custom one.

If your jar file is to replace a jar file in the lib folder , perhaps as a temporary workaround provided by servicenow support for a known problem, then the additional folder would need to come first :

wrapper.java.classpath.1=../extlib2/*.jar wrapper.java.classpath.2=lib/*.jar wrapper.java.classpath.3=extlib/*.jar

Restart the MID Server Note: If you need to add/replace/delete these files in future, you would first have to Stop the MID Server service to clear the file lock, do those changes, then start it again.

Huge warning:

If you choose to do this, you are now responsible for maintaining and testing the MID Server and JVM still work properly, and continue to work properly after every patch and upgrade. ServiceNow do allow customising the JVM, but don't support this. There are security implications. You could be adding vulnerable code to the MID Server, or code that may turn out to be vulnerable in future if not regularly updated. When you consider what a MID Server fundamentally is, that's a big deal. You need to be sure file/folder permissions on the new lib folder are secure. You don't want anyone to be able to add more files into that folder, but the mid server service user still needs to read it. There is no visibility that this has been done from the instance, so could cause confusion when tech support is involved in troubleshooting a support case. This changes the Java JVM underneath the MID Server application, and so can change how the MID Server application code behaves. If upgraded/newer versions of out-of-box classes are used, they will not have been tested by ServiceNow, and may not be compatible. If upgraded version of some classes are used, and then ServiceNow upgrade the same class in the lib folder, then the old version you added will still be used. Your older version may include security vulnerabilities, or be incompatible with the rest of the new version of the MID Server code. Java files you add may not be compatible with future Java versions that the MID Server automatically upgrades to. e.g. is fine in 11.0.9, but not 11.0.15. You need to test this as the MID Server java version changes. Class Dependencies of the added jar files may be there out-of-box now, and be compatible now, but be changed to different versions or even removed in future MID Server upgrades. This is a customisation, and would need reverting manually after you no longer need to do it. Reverting the customisation: Edit agent\conf\wrapper-override.conf, (not wrapper.conf) deleting any wrapper.java.classpath lines Restart the MID Server Delete the extra folder after the restart, when it will no longer be locked