MID Server issues for mTLS (mutual authentication/certificate-based authentication)Issue For a MID Server which is mutual authentication enabled, we sometimes see the MID Server issues: MID Server encryption keys do not match and are no longer valid. To restore proper functionality, invalidate and re-validate the MID Server.Could not decrypt file discovery whitelist after sync Since this is a mutual authentication enabled MID Server, both 'Rekey' UI link or 'Invalidate/Validate' UI links are not available. Also, the steps of deleting the agent_keystore file and restarting the MID Server would give you errors similar to below: 01/20/22 13:57:15 (566) StartupSequencer SEVERE *** ERROR *** SOAP Response: Status code=401, Response body=<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"><SOAP-ENV:Header/><SOAP-ENV:Body><SOAP-ENV:Fault><faultcode>User Not Authenticated</faultcode><faultstring>User Not Authenticated</faultstring><detail>Error completing SOAP request</detail></SOAP-ENV:Fault></SOAP-ENV:Body></SOAP-ENV:Envelope>01/20/22 13:57:15 (567) StartupSequencer WARNING *** WARNING *** Could not authenticate user 'null' on the ServiceNow instance01/20/22 13:57:15 (567) StartupSequencer SEVERE *** ERROR *** test failurejava.lang.IllegalStateException: User cannot be authenticated or is missing the proper roles. If you have deleted or changed the MID server keystore, and config.xml mid.instance.password value is encrypted, you may need to change this value to plain text (during MID startup, password is re-encrypted using current keystore and written back to mid.instance.password).at com.service_now.mid.services.StartupSequencer.runTests(StartupSequencer.java:668)at com.service_now.mid.services.StartupSequencer.startupSequencerRunnable(StartupSequencer.java:717)at java.base/java.lang.Thread.run(Thread.java:834) Despite what the error message says, deleting the agent_keystore file is not an option as you would delete the custom PEM Certificate necessary for the MID Server to authenticate with mutual authentication. Those steps are useful only when the MID should generate a key on start up, but not when a key/cert is being manually installed for mTLS. Deleting the security\agent_keystore or keystore\agent_keystore.jks file probably should not be done for mTLS MID.CauseThese are the most probable steps for reproducing this issue: Install a MID Server using a username/password (basic auth) and connect to the instanceValidate MIDShut down MIDInstall key/cert necessary to MID'S agent_keystore (mutual auth), using the batch file \agent\bin\scripts\install-certificate.bat (or .sh)Restart MID In this state, the public key at the instance corresponds to the private key that was in the agent_keystore when basic auth was being used (when it was previously validated). After mTLS is enabled, when the MID calls to update the public key, if the MID had been previously validated the the old key is verified vs. the signature included in the message. This would be the signature of the new key (installed for mTLS), and this verification fails. In this situation you will see this logging: MID Server encryption keys do not match and are no longer valid. To restore proper functionality, invalidate and re-validate the MID Server. Since the MID Server is now mTLS enabled there really is no way to invalidate or rekey from the UI. Those controls should be absent from the MID form. So the suggestion in this message is not helpful, though the cause indicated is accurate.ResolutionTo avoid this you should only enable mTLS by uploading keys/certs if: the MID has never been started at all yetOr, the MID has first been invalidated before shutting down to reconfigure for mTLS But once this issue has occurred, as of now, the only possible workaround is to install a fresh MID Server and enable mutual authentication on it, keeping in mind the above points.Related LinksPRB1556363 has been opened to change the error message : 'MID Server encryption keys do not match and are no longer valid. To restore proper functionality, invalidate and re-validate the MID Server' displayed for 'Mutual authentication' enabled midservers as these midservers do not have invalidate/revalidate or rekey ui links and this error message is quite misleading.