One Codex Status - Incident History

AWS degraded performance

2025-10-20T22:40:57Z

Oct 20, 22:40 UTC
Resolved - AWS have resolved the issue affecting our services. The affected workflows are now running.

Oct 20, 18:55 UTC
Investigating - AWS reported degraded performance, beginning Oct 20th 2025, 12:11am PDT. This has resulted in some of our systems also suffering from degraded performance. Some workflows and features may not begin processing during this issue. We are monitoring the situation and will run any queued workflows upon AWS recovery.

Intermittent issues in One Codex web app

2025-04-08T19:44:48Z

Apr 8, 19:44 UTC
Resolved - This incident has been resolved.

Apr 8, 17:46 UTC
Monitoring - We have a fix in place and are monitoring our application health.

Apr 8, 17:05 UTC
Identified - We are currently experiencing intermittent outages affecting the One Codex web app. Our team has identified the root cause and is actively working on a resolution. We will provide updates as we make progress. Thank you for your patience.

Database Maintenance

2025-03-15T19:00:21Z

Mar 15, 19:00 UTC
Completed - The scheduled maintenance has been completed.

Mar 15, 18:52 UTC
Update - We are completing the database migration and expect downtime to conclude by 3pm US Eastern.

Mar 15, 18:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Mar 15, 12:20 UTC
Scheduled - We expect to perform final database maintenance between 2-3pm US Easter (18:00-19:00 UTC) on March 15th. During this window, we expect some intermittent downtime plus a period of prolonged downtime. We expect service to be up by 3pm US Eastern.

Database Maintenance

2025-03-15T02:00:21Z

Mar 15, 02:00 UTC
Completed - The scheduled maintenance has been completed.

Mar 15, 00:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Mar 13, 20:38 UTC
Scheduled - We will be performing routine database updates at midnight UTC (8pm ET, 5pm PT) on Friday evening/Saturday morning. We expect a brief period of app interruption (10-15 minutes), but users may experience degraded performance or intermediate availability during the maintenance window.

Database Maintenance

2024-03-28T00:45:01Z

Mar 28, 00:45 UTC
Completed - The scheduled maintenance has been completed.

Mar 28, 00:32 UTC
Verifying - Verification is currently underway for the maintenance items.

Mar 28, 00:24 UTC
Update - Database maintenance is taking longer than expected but should completed by 01:00 UTC.

Mar 27, 23:30 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Mar 23, 00:12 UTC
Scheduled - We will be undergoing scheduled maintenance during this time.

Database Maintenance

2024-03-08T01:19:27Z

Mar 8, 01:19 UTC
Completed - The scheduled maintenance has been completed successfully.

Mar 8, 01:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Mar 6, 17:44 UTC
Scheduled - We will be performing scheduled database maintenance during this time.

Degraded Performance of the One Codex Application

2024-02-28T19:13:51Z

Feb 28, 19:13 UTC
Resolved - This incident has been resolved.

Feb 28, 19:11 UTC
Update - We have implemented a fix for the issue and the One Codex application is once again stable.

Feb 28, 18:45 UTC
Update - We are continuing to investigate this issue.

Feb 28, 18:39 UTC
Investigating - We are aware of an issue causing degraded performance of our application and are investigating remediation options.

Errors for some multi-step custom workflows

2024-01-17T21:26:00Z

Jan 17, 21:26 UTC
Resolved - Some multi-step workflows are experiencing unexpected internal failures across our workflow orchestration platform.

Database Upgrade

2023-09-14T02:40:08Z

Sep 14, 02:40 UTC
Completed - The scheduled maintenance has been completed.

Sep 14, 01:50 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Sep 11, 17:22 UTC
Scheduled - We will be undergoing scheduled maintenance during this window for a routine database upgrade. We expect some intermittent availability or performance issues, with 20-30 minutes of downtime to the core platform application and APIs starting at approximately 01:50 UTC on September 14.

Database Update

2023-06-23T23:54:04Z

Jun 23, 23:54 UTC
Completed - Scheduled maintenance completed successfully. All components are now operational.

Jun 23, 23:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Jun 21, 15:56 UTC
Scheduled - We will be undergoing scheduled maintenance during this window for a routine database upgrade. We expect some intermittent availability or performance issues, with 5-15 minutes of downtime to the core platform application and APIs around approximately 23:00 UTC on June 23.

Database Upgrade

2023-03-31T23:55:25Z

Mar 31, 23:55 UTC
Completed - The scheduled maintenance has been completed.

Mar 31, 23:03 UTC
Update - The application is temporarily paused while we perform a database migration.

Mar 31, 23:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Mar 29, 12:28 UTC
Scheduled - We will be undergoing scheduled maintenance during this window for a routine database upgrade. We expect some intermittent availability or performance issues, with 5-15 minutes or downtime to the core platform application and APIs around approximately 23:00 UTC on March 31.

Database Upgrade

2023-03-25T15:49:01Z

Mar 25, 15:49 UTC
Completed - The scheduled maintenance has been completed.

Mar 25, 15:48 UTC
Update - Scheduled maintenance is still in progress. We will provide updates as necessary.

Mar 25, 14:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Mar 25, 10:28 UTC
Scheduled - We will be undergoing scheduled maintenance during this window for a routine database upgrade. We expect some intermittent availability or performance issues, with 5-15 minutes or downtime to the core platform application and APIs around approximately 15:00 UTC.

Database Update

2022-03-22T00:00:22Z

Mar 22, 00:00 UTC
Completed - The scheduled maintenance has been completed.

Mar 21, 23:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Mar 17, 19:58 UTC
Scheduled - We will be undergoing scheduled maintenance during this time.

Versioning system migration

2021-11-05T12:30:41Z

Nov 5, 12:30 UTC
Completed - The scheduled maintenance has been completed.

Nov 5, 11:31 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Nov 2, 22:44 UTC
Scheduled - We will be undergoing scheduled maintenance during this time.

Database Update

2021-04-09T22:30:31Z

Apr 9, 22:30 UTC
Completed - The scheduled maintenance has been completed.

Apr 9, 22:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Apr 9, 19:44 UTC
Scheduled - We will be undergoing scheduled maintenance during this time.

Database Update

2020-12-11T23:00:36Z

Dec 11, 23:00 UTC
Completed - The scheduled maintenance has been completed.

Dec 11, 22:30 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Dec 9, 18:02 UTC
Scheduled - We will be undergoing scheduled maintenance during this time.

Database Update

2020-07-27T22:12:59Z

Jul 27, 22:12 UTC
Completed - The scheduled maintenance has been completed.

Jul 27, 22:11 UTC
Verifying - Verification is currently underway for the maintenance items.

Jul 27, 22:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Jul 24, 16:27 UTC
Scheduled - We will be undergoing scheduled maintenance during this time.

Database Update

2020-04-24T22:50:33Z

Apr 24, 22:50 UTC
Completed - The scheduled maintenance has been completed.

Apr 24, 22:30 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Apr 23, 16:54 UTC
Scheduled - We will be undergoing scheduled maintenance during this time.

Database Update

2019-12-11T00:13:39Z

Dec 11, 00:13 UTC
Completed - The scheduled maintenance has been completed.

Dec 11, 00:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Dec 6, 21:15 UTC
Scheduled - We will be undergoing scheduled maintenance to perform database updates at 12am UTC on December 11, 2019 (4pm Pacific, 7pm Eastern on December 10th).

Database Update

2019-09-28T00:30:30Z

Sep 28, 00:30 UTC
Completed - The scheduled maintenance has been completed.

Sep 28, 00:15 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Sep 28, 00:10 UTC
Scheduled - We will be undergoing scheduled maintenance during this time.

Database Update

2019-09-06T22:10:40Z

Sep 6, 22:10 UTC
Completed - The scheduled maintenance has been completed.

Sep 6, 21:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Sep 4, 18:29 UTC
Scheduled - We will be updating our database on Friday evening, leading to brief unavailability of the app (up to 1 hour though we expect less than 30 minutes).

Database Update

2019-06-19T23:23:42Z

Jun 19, 23:23 UTC
Completed - The scheduled maintenance has been completed.

Jun 19, 23:22 UTC
Verifying - Verification is currently underway for the maintenance items.

Jun 19, 23:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Jun 18, 23:39 UTC
Scheduled - We will be undergoing scheduled maintenance during this time. We are planning for 30 minutes of downtime, but expect no more than 10-15 minutes.

Database Update

2019-05-22T22:13:24Z

May 22, 22:13 UTC
Completed - The scheduled maintenance has been completed.

May 22, 22:12 UTC
Verifying - Verification is currently underway for the maintenance items.

May 22, 22:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

May 22, 00:26 UTC
Scheduled - We will be undergoing scheduled maintenance during this time. We are planning for 30 minutes of downtime, but expect no more than 10-15 minutes.

Database Update

2019-05-09T23:28:16Z

May 9, 23:28 UTC
Completed - The scheduled maintenance has been completed.

May 9, 23:26 UTC
Verifying - Verification is currently underway for the maintenance items.

May 9, 23:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

May 9, 18:57 UTC
Scheduled - We will be undergoing scheduled maintenance and performing database updates during this time. We expect downtime of no more than 30 minutes, but please plan for up to 1 hour of unavailability.

Database Update

2019-02-15T01:30:55Z

Feb 15, 01:30 UTC
Completed - The scheduled maintenance has been completed.

Feb 15, 01:16 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Feb 14, 21:33 UTC
Scheduled - We will be performing database updates from 1:15am-1:30am UTC on February 15 (8:15pm-8:30pm Eastern on February 14). We expect downtime of no more than 15 minutes, with likely downtime of only several minutes.