Cloud Cost Management | CCM Infra Stack – Read Replica ImpactIssue CCM 8.1 along with CCM Infra Stack 8.1 utilizes shared services infrastructure to process large billing data. The processed data is then sent to Glide for storage into DB. This data transfer between Fenix Job and Glide happens via REST APIs. During CCM billing job execution and specially towards the end of it, we sink a lot of billing and spend data using these APIs on to the Glide. During this period, usually lasting between 5 to 30 mins, the insertion/update rate could grow as much as 500k records/minute. This high insertion rate may cause Read Replica lag. The overall duration of high insertion rate and the rate of insertion and the replica lag will vary based on billing data being processed and other environment/instance state.Note: Above numbers mentioned are based on billing data upto 100 Million line items.SymptomsRead replica lag is observed during CCM billing job executionReleaseCCM Infra stack 8.1 onwardsCauseDuring CCM billing job execution and specially towards the end of it, we sink a lot of billing and spend data using these APIs on to the Glide. During this period, usually lasting between 5 to 30 mins, the insertion/update rate could grow as much as 500k records/minute. This high insertion rate may cause Read Replica lag.ResolutionRecommendations: We recommend running CCM Billing jobs outside of business hours.We also recommend to not run any other application process/job which is heavy on DB insertions/update along with CCM billing job.