Employee Document Management (EDM) FAQEmployee Document Management (EDM) FAQ Employee Document Management (EDM) provides a centralized storage space and a complete view for all employee documents. The EDM Bulk Imports feature copies employee documents from an external third-party cloud-based storage or a local network directory to Employee Document Management. We have seen issues related to Import the most but this KB will cover any issues related to EDM. EDM Product Documentation: https://docs.servicenow.com/bundle/utah-employee-service-management/page/product/human-resources/concept/hr-employee-doc-management.html Bulk Import Product Documentation: https://docs.servicenow.com/bundle/utah-employee-service-management/page/product/human-resources/concept/edm-bulk-uploads.html Bulk Import Configuration/Implementation Guides: https://www.servicenow.com/community/hrsd-articles/employee-document-management-bulk-import-functionality/ta-p/2309316 Other Useful links: EDM Implementation Guidance and Best Practice (Article) EDM community resource page Common EDM issues and solutions: 1. The Import Capture step failed with an error: Failed to create attachment: Failed to get HTTP Response And you see errors in the mid server logs: Unbuffered entity enclosing request can not be repeated This could be a mid server permission issue where the "AttachmentSink" API of MID server is not uploading files to the instance. Mid server user needs to have the following role(s): sn_hr_ef.admin role OR sn_hr_ef.document_import role AND sn_hr_ef.document_writer role 2. The Import Verify step comes back with an error: Configuration test was unsuccessful. Change the configuration values and try again. Error: No credential found for types [SSH Password,SSH Private Key] with credential tag [sn_hr_ef.file_import] This could mean either the ssh credentials are not set up/verified OR the correct MID server is not being used by the import. 1) Check and verify ssh credentials are successful - see #3: how to debug ssh connection related issue 2) Check the ECC queue to see if the right MID server is being used (usually customer has set up a MID server with ssh configuration) - for this see #4 on changes needed to use a specific MID server 3. How to debug ssh connection related issues. If you see error 'Cannot connect, status is TCP_CONNECTION_DROPPED', follow the below to debug Try to establish a connection to the SSH server via SSH client like opnessh or putty on the Mid server host to see if it works2. Look at the security logs on the server to see if you can find any specific issue. /var/log/secure often has good info.3. Enable debug log on the SSH server. For openssh, this typically means editing /etc/ssh/sshd_config, setting loglevel=debug3, and restarting sshd. Then check the logs in '/var/log/secure'4. Mid server can also be started in debug mode using mid.ssh.debug=true, mid.log.level=debug recreate the problem, and then look in the agent log. More information about MID server debug parameters: https://docs.servicenow.com/bundle/utah-servicenow-platform/page/product/mid-server/reference/mid-server-parameters.html#d2631379e7765. Make sure port 22 is open and firewall has been configured to allow port 22. 4. How do I specify a MID server for the bulk import job?1. Create a new unique capability. Mid Server -> Capabilities<instance-name>/nav_to.do?uri=%2Fecc_agent_capability_list.do%3Fsysparm_userpref_module%3D0de137980a0006bc653cf70209afd11f%26sysparm_clear_stack%3Dtrue2. Add the new capability into the Capabilities related list of the MID Server that you would like to use for the bulk import. Also, remove 'All' capability from the MID server. Make sure, SSH capability is added3. Update ef_LocalFileCapture::findMidServerForConfig method Replace the line 'var serverList = String(new sn_automation.AutomationAPI().selectUpMidServers(null, [host], null)).split(',');' with the following codevar capabilities = [{capability: 'EdmBulkImport'}];var serverList = String(new sn_automation.AutomationAPI().selectUpMidServers(null, [host], capabilities)).split(',');4. Update the 'Test configuration' activity designer activities by going to the list via 'Orchestration' -> 'Activity Designer Activities'(a) Click 'Checkout' button(b) In the 'Execution Command' tab, add the 'EdmBulkImport' capability to the field 'Required MID Server capabilities'(c) Save the changes and publish5. Update the 'Get file names' activity designer activities by going to the list via 'Orchestration' -> 'Activity Designer Activities'(a) Click 'Checkout' button(b) In the 'Execution Command' tab, add the 'EdmBulkImport' capability to the field 'Required MID Server capabilities'(c) Save the changes and publishOR You can add a new application for your MID selection criteria and have that application associated only with the MID server that you want to test. This makes MID selector to select the MID which has the corresponding application associated. 5. Can I use a windows based MID server for EDM Import? How about the file server? The MID server can be on a windows or linux server. Both MID server and file server are linux in OOB examples. Other methods or approaches require custom development. 6. Creation of a Retention Period/Policy does not appear to be working as expected. Documents are not being purged as expected. The "run as" user for the scheduled job needs to have the "sn_hr_ef.admin" role. On giving 'sn_hr_ef.admin' role to the "run as" user the scheduled jobs are getting executed and the purge and notification flow is working fine. 7. Employee document upload custom UI page doesn't support viewing non pdf using doc viewer. This is an enhancement. At the time EDM first developed (London) doc viewer was supporting only pdf format. Workaround/solution:1. In order to show the [view] button for other document types, you need to update canView property to true in "Employee document Upload" UI page. 2. Add one more OR query in condition builder of this ACl (a10c0618dbcac4102207f1471d96192f) table_name= sys_attachment 8. The Import Capture stage step failed with error: "Unable to complete file capture step. Failure details: Failed to create attachment: Failed to get HTTP Response" and the ECC QUEUE shows the error "No sensors defined". The configured user (username in ssh credentials) needs to have admin access. This can be easily verified using system settings.https://docs.servicenow.com/bundle/utah-employee-service-management/page/product/human-resources/concept/edm-bulk-uploads.html 9. Below are some debugging tips for EDM Import issues: When specifying the directory value in the Employee document import configuration, the value has to be relative the home directory. If the full path is '~/Documents/import', then the value of the directory should be 'Documents/importTurn on MID server logging by setting 'mid.show.queries' property parameter.Read more about it here - https://docs.servicenow.com/bundle/utah-servicenow-platform/page/product/mid-server/reference/mid-server-parameters.html#d2430021e1603Check mid server logs, agent and wrapper logsAdd/turn on glide.http.log_debug system propertyCheck the ECC queue:The ECC Queue record contains information about a command either sent to or received from the MID Server. Read more about the ECC queue or find assistance with MID Server troubleshooting.https://docs.servicenow.com/bundle/utah-it-operations-management/page/product/discovery/reference/r_DiscoveryStatusECCQueue.html?cshalt=yeshttps://docs.servicenow.com/bundle/utah-servicenow-platform/page/product/mid-server/reference/r_MIDServerTroubleshooting.html?cshalt=yes 10. Import Capture job fails with an error "Invalid uri" If the file name contain spaces then, then the capture process will result in a "java.lang.IllegalArgumentException: Invalid uri "HTTP error that can be seen in the ECC queue response for these files.Solution: use workaround from KB0965070.This issue is found on Paris and fixed in Orlando. 11. Tips for debugging the Bulk Import jobs: Setup Mid Server that will be used to communicate with the file server and the instance. Go to HR Administration -> Bulk Import -> Import ConfigurationEdit desired configuration and set "Staging Record State" to "Ready" After which to import files follow following process:Click UI action: "Start Verify Configuration" which will kick off the verification step of credentials and the connectivity of Mid Server to File Server If you get errors with connectivity those needs to be resolved before staging step. Staging step: If verification of credentials passes then click "Start staging job" UI action. If successful this job will retrieve metadata about the files that are ready to be transferred over. If Staging step is successful then click "Start capture job" One note here: Because we changed the state to "Ready" Capture Job will get all "Ready" files and download them from File Server -> Mid Server and from Mid Server -> Instance into sys_Attachment table. In case the job fails, each failed file will have an Error state and error can be found in the Document Staging related list Related list that is far to the right). 12. PDF attachments in some employee document records are being deleted automatically. PDF attachments might get deleted from the platform scheduled job "Clean up converted documents genereated by PDF Generation Utilities plugin" It queries the table: sys_pdf_generation_statusfor records that have :a "sys_updated_on" less than the retention sys_property "com.snc.documentviewer.retention_days_converted_file" which defaults to 7 days agoa "conversion_status" of completeand non null converted_attachment_idThen it attempts to delete those attachments and update the status for "sys_pdf_generation_status" records to "expired" Workarounds:1) To stop files from being deleted as soon as 7 days ago increase the value of the retention sys_properties: com.snc.documentviewer.retention_days_converted_file.Navigate to /nav_to.do?uri=sys_properties.do?sys_id=a23429377323230070128f758bf6a733Set the "Value" field to greater than 7 OR 2) Disable the scheduled job "Clean up converted documents genereated by PDF Generation Utilities plugin"