Connector API latency spikes and request timeouts

Incident Report for Mews

Postmortem

Problem

Starting 14 July 2025 06:00 UTC, the Mews Connectivity team observed short periods of higher response times and request timeouts in the Mews Connector API. The issue occurred daily at regular intervals and affected some partners and customers.

Action

From the first observation, the Mews team increased system capacity and reduced load on the affected services. They moved Connector API traffic to a dedicated group and isolated background workloads. They ensured there’s stability during peak traffic by refining autoscaling rules.

Causes

The problem was caused by thread exhaustion under high concurrent load. This created brief delays, even when system resources were available.

Solutions

The Connector API now runs on a dedicated resources group with increased capacity. The Mews team is monitoring the services closely and optimizes performance to remove blocking workloads. The system is fully operational.

Posted Sep 23, 2025 - 21:23 CEST

Resolved

The mitigation steps and migration were completed.

Posted Sep 04, 2025 - 11:55 CEST

Update

The migration of Booking Engine API and Channel Manager API on the production environment has been completed. All Open APIs are now fully operational.

While we have taken steps to minimize timeout spikes, occasional timeouts may still occur. We are continuing to investigate the underlying causes.

Posted Aug 19, 2025 - 13:40 CEST

Update

The planned migration of Booking Engine API and Channel Manager API on the production environment will resume tomorrow (August 19) between 11:00 and 13:00 UTC.

During the migration window, you may experience increased response latency for the affected APIs.

Posted Aug 18, 2025 - 15:34 CEST

Update

The migration of Connector API on the production environment was successfully completed yesterday. Booking Engine API and Channel Manager API have also been migrated on the demo environment.

The planned migration of Booking Engine API and Channel Manager API on the production environment is postponed. We will provide the updated schedule next week.

Posted Aug 14, 2025 - 14:53 CEST

Update

Connector API migration on the demo environment was completed successfully yesterday.

Today (August 13), we will also migrate Booking Engine API and Channel Manager API on the demo environment between 12:00 and 13:00 UTC.

The migration of Connector API on the production environment will proceed as planned today between 13:00 and 16:00 UTC.

Tomorrow (August 14), Booking Engine API and Channel Manager API will be migrated on the production environment between 13:00 and 16:00 UTC.

During each migration window, you may experience increased response latency in the affected APIs in the respective environments.

Posted Aug 13, 2025 - 10:58 CEST

Update

We are migrating Connector API traffic to dedicated infrastructure.

Schedule:
- Demo environment: August 12, 13:00–16:00 UTC
- Production environment: August 13, 13:00–16:00 UTC

You may experience increased response latency for Connector API during these times in the respective environments.

Posted Aug 12, 2025 - 13:41 CEST

Update

We continue the investigation and talking to our cloud provider.
Our platform team is actively monitoring, and taking mitigation steps as needed.
Further updates will follow as more information becomes available.

Posted Aug 07, 2025 - 17:24 CEST

Update

We continue to investigate and are closely monitoring the system’s status. Periodic latency spikes may occasionally occur in short intervals on individual instances. Our platform team is actively working to identify the root cause, monitoring metrics, and taking mitigation steps as needed.

We have upscaled our compute infrastructure to reduce the load on affected instances, which should help minimize customer impact.

Further updates will follow as more information becomes available.

Posted Jul 23, 2025 - 17:53 CEST

Monitoring

Correction, please disregard the previous update as it was related to another incident: https://status.mews.com/incidents/3q6n596dqzd9

Posted Jul 16, 2025 - 17:26 CEST

Investigating

We have noticed a downgraded performance of the Smart Tips.
We are currently looking into it.
Thank you for your patience.

Posted Jul 16, 2025 - 16:56 CEST

Monitoring

We have resolved the issue, and we are closely monitoring the system to ensure stability.

Posted Jul 15, 2025 - 17:48 CEST

Investigating

We are currently investigating reports of increased latency across our public API. Initial indicators show periodically elevated response times and timeouts affecting most endpoints.

Thank you for your patience as we work to resolve the issue.

Posted Jul 15, 2025 - 13:43 CEST

This incident affected: Mews Open API.