Scan container image pattern fails with "TOOMANYREQUESTS" - Known Error

Scan container image pattern fails with "TOOMANYREQUESTS" Description Scan Container Image pattern uses the Trivy tool to scan the image. Trivy tool is downloading images in order to scan them.

When you pull images from the docker hub as non-paying , there is a limit to the number of downloads in a given time window.

Regarding pricing and limits refer to the below links

https://www.docker.com/pricing/

https://www.docker.com/increase-rate-limits/

"The rate limits of 100 container image requests per six hours for anonymous usage, and 200 container image requests per six hours for free Docker accounts are now in effect. Image requests exceeding these limits will be denied until the six hour window elapses. NOTE: Docker Pro and Docker Team accounts enable 5,000 pulls in a 24 hour period from Docker Hub. "

This issue is related to Limits and Limits may vary based on the Docker hub subscription. Hence, Based on your subscription, you need to modify the schedule job frequency and System property " sn_itom_pattern.container_image_batch_size" values.

Steps to Reproduce

Steps: 1. Navigate to the sn_itom_pattern_container_image_scan_status table and scan_status is None. 2. Scan Container Images scheduled job triggers Scan Container Image pattern. 4. Scan Container Image pattern failed with the error " TOOMANYREQUESTS "

Error Observed: 2023-01-05T08:16:22.855-0500 [31mFATAL[0m scan error: unable to initialize a scanner: unable to initialize a docker scanner: 3 errors occurred: * unable to inspect the image (busybox:latest): Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running? * unable to initialize Podman client: no podman socket found: stat /run/user/1000/podman/podman.sock: no such file or directory * GET https://index.docker.io/v2/library/busybox/manifests/latest: TOOMANYREQUESTS: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit

Workaround By default, the " Scan Container Images " scheduled job runs every 1 minute . If the " Scan Container Image" pattern fails with the error and message containing " TOOMANYREQUESTS: You have reached your pull rate limit " then do the following:

Modify Scan Container Images schedule Job frequency to longer intervals (maybe 6 hours or more) which optimally works for you. Modify scan_status from " error " to " none " in sn_itom_pattern_container_image_scan_status for which images scan failed with TOOMANYREQUESTS error. If scan_status is changed from error to none, the Schedule job picks this image and triggers the "Scan Container Image " pattern.

Note : sn_itom_pattern.container_image_batch_size default value is 10. Scan Container Images scheduled job picks at max value of sn_itom_pattern.container_image_batch_size of not processed images in each run and trigger pattern for each image.