Adding products to an open bill of a company profile results in an error

Problem

On 23rd of July, we experienced an issue with adding products to a bill for company profiles. We identified the root cases to be a faulty code change that was deployed few hours before incident was raised.

Action

After getting alerted we immediately spotted the faulty version of deployment and rolled back to previously stable version. Afterwards we were monitoring if there are any more enterprises running into same problem.

Causes

The issue was caused by faulty code change in add product dialog, which allowed calling specific API endpoint for company profiles, which wasn’t available for company profiles and started returning invalid responses.

Solutions

• We are continuously improving our alerting and monitoring, as well as automatic testing measures to prevent user impact.