One Codex Status - Incident History

AWS degraded performance

Mon, 20 Oct 2025 22:40:57 +0000

Oct 20, 22:40 UTC
Resolved - AWS have resolved the issue affecting our services. The affected workflows are now running.

Oct 20, 18:55 UTC
Investigating - AWS reported degraded performance, beginning Oct 20th 2025, 12:11am PDT. This has resulted in some of our systems also suffering from degraded performance. Some workflows and features may not begin processing during this issue. We are monitoring the situation and will run any queued workflows upon AWS recovery.

Intermittent issues in One Codex web app

Tue, 08 Apr 2025 19:44:48 +0000

Apr 8, 19:44 UTC
Resolved - This incident has been resolved.

Apr 8, 17:46 UTC
Monitoring - We have a fix in place and are monitoring our application health.

Apr 8, 17:05 UTC
Identified - We are currently experiencing intermittent outages affecting the One Codex web app. Our team has identified the root cause and is actively working on a resolution. We will provide updates as we make progress. Thank you for your patience.

Database Maintenance

Sat, 15 Mar 2025 19:00:21 +0000

Mar 15, 19:00 UTC
Completed - The scheduled maintenance has been completed.

Mar 15, 18:52 UTC
Update - We are completing the database migration and expect downtime to conclude by 3pm US Eastern.

Mar 15, 18:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Mar 15, 12:20 UTC
Scheduled - We expect to perform final database maintenance between 2-3pm US Easter (18:00-19:00 UTC) on March 15th. During this window, we expect some intermittent downtime plus a period of prolonged downtime. We expect service to be up by 3pm US Eastern.

Database Maintenance

Sat, 15 Mar 2025 02:00:21 +0000

Mar 15, 02:00 UTC
Completed - The scheduled maintenance has been completed.

Mar 15, 00:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Mar 13, 20:38 UTC
Scheduled - We will be performing routine database updates at midnight UTC (8pm ET, 5pm PT) on Friday evening/Saturday morning. We expect a brief period of app interruption (10-15 minutes), but users may experience degraded performance or intermediate availability during the maintenance window.

Database Maintenance

Thu, 28 Mar 2024 00:45:01 +0000

Mar 28, 00:45 UTC
Completed - The scheduled maintenance has been completed.

Mar 28, 00:32 UTC
Verifying - Verification is currently underway for the maintenance items.

Mar 28, 00:24 UTC
Update - Database maintenance is taking longer than expected but should completed by 01:00 UTC.

Mar 27, 23:30 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Mar 23, 00:12 UTC
Scheduled - We will be undergoing scheduled maintenance during this time.

Database Maintenance

Fri, 08 Mar 2024 01:19:27 +0000

Mar 8, 01:19 UTC
Completed - The scheduled maintenance has been completed successfully.

Mar 8, 01:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Mar 6, 17:44 UTC
Scheduled - We will be performing scheduled database maintenance during this time.

Degraded Performance of the One Codex Application

Wed, 28 Feb 2024 19:13:51 +0000

Feb 28, 19:13 UTC
Resolved - This incident has been resolved.

Feb 28, 19:11 UTC
Update - We have implemented a fix for the issue and the One Codex application is once again stable.

Feb 28, 18:45 UTC
Update - We are continuing to investigate this issue.

Feb 28, 18:39 UTC
Investigating - We are aware of an issue causing degraded performance of our application and are investigating remediation options.

Errors for some multi-step custom workflows

Wed, 17 Jan 2024 21:26:00 +0000

Jan 17, 21:26 UTC
Resolved - Some multi-step workflows are experiencing unexpected internal failures across our workflow orchestration platform.

Database Upgrade

Thu, 14 Sep 2023 02:40:08 +0000

Sep 14, 02:40 UTC
Completed - The scheduled maintenance has been completed.

Sep 14, 01:50 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Sep 11, 17:22 UTC
Scheduled - We will be undergoing scheduled maintenance during this window for a routine database upgrade. We expect some intermittent availability or performance issues, with 20-30 minutes of downtime to the core platform application and APIs starting at approximately 01:50 UTC on September 14.

Database Update

Fri, 23 Jun 2023 23:54:04 +0000

Jun 23, 23:54 UTC
Completed - Scheduled maintenance completed successfully. All components are now operational.

Jun 23, 23:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Jun 21, 15:56 UTC
Scheduled - We will be undergoing scheduled maintenance during this window for a routine database upgrade. We expect some intermittent availability or performance issues, with 5-15 minutes of downtime to the core platform application and APIs around approximately 23:00 UTC on June 23.

Database Upgrade

Fri, 31 Mar 2023 23:55:25 +0000

Mar 31, 23:55 UTC
Completed - The scheduled maintenance has been completed.

Mar 31, 23:03 UTC
Update - The application is temporarily paused while we perform a database migration.

Mar 31, 23:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Mar 29, 12:28 UTC
Scheduled - We will be undergoing scheduled maintenance during this window for a routine database upgrade. We expect some intermittent availability or performance issues, with 5-15 minutes or downtime to the core platform application and APIs around approximately 23:00 UTC on March 31.

Database Upgrade

Sat, 25 Mar 2023 15:49:01 +0000

Mar 25, 15:49 UTC
Completed - The scheduled maintenance has been completed.

Mar 25, 15:48 UTC
Update - Scheduled maintenance is still in progress. We will provide updates as necessary.

Mar 25, 14:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Mar 25, 10:28 UTC
Scheduled - We will be undergoing scheduled maintenance during this window for a routine database upgrade. We expect some intermittent availability or performance issues, with 5-15 minutes or downtime to the core platform application and APIs around approximately 15:00 UTC.

Database Update

Tue, 22 Mar 2022 00:00:22 +0000

Mar 22, 00:00 UTC
Completed - The scheduled maintenance has been completed.

Mar 21, 23:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Mar 17, 19:58 UTC
Scheduled - We will be undergoing scheduled maintenance during this time.

Versioning system migration

Fri, 05 Nov 2021 12:30:41 +0000

Nov 5, 12:30 UTC
Completed - The scheduled maintenance has been completed.

Nov 5, 11:31 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Nov 2, 22:44 UTC
Scheduled - We will be undergoing scheduled maintenance during this time.

Database Update

Fri, 09 Apr 2021 22:30:31 +0000

Apr 9, 22:30 UTC
Completed - The scheduled maintenance has been completed.

Apr 9, 22:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Apr 9, 19:44 UTC
Scheduled - We will be undergoing scheduled maintenance during this time.

Database Update

Fri, 11 Dec 2020 23:00:36 +0000

Dec 11, 23:00 UTC
Completed - The scheduled maintenance has been completed.

Dec 11, 22:30 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Dec 9, 18:02 UTC
Scheduled - We will be undergoing scheduled maintenance during this time.

Database Update

Mon, 27 Jul 2020 22:12:59 +0000

Jul 27, 22:12 UTC
Completed - The scheduled maintenance has been completed.

Jul 27, 22:11 UTC
Verifying - Verification is currently underway for the maintenance items.

Jul 27, 22:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Jul 24, 16:27 UTC
Scheduled - We will be undergoing scheduled maintenance during this time.

Database Update

Fri, 24 Apr 2020 22:50:33 +0000

Apr 24, 22:50 UTC
Completed - The scheduled maintenance has been completed.

Apr 24, 22:30 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Apr 23, 16:54 UTC
Scheduled - We will be undergoing scheduled maintenance during this time.

Database Update

Wed, 11 Dec 2019 00:13:39 +0000

Dec 11, 00:13 UTC
Completed - The scheduled maintenance has been completed.

Dec 11, 00:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Dec 6, 21:15 UTC
Scheduled - We will be undergoing scheduled maintenance to perform database updates at 12am UTC on December 11, 2019 (4pm Pacific, 7pm Eastern on December 10th).

Database Update

Sat, 28 Sep 2019 00:30:30 +0000

Sep 28, 00:30 UTC
Completed - The scheduled maintenance has been completed.

Sep 28, 00:15 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Sep 28, 00:10 UTC
Scheduled - We will be undergoing scheduled maintenance during this time.

Database Update

Fri, 06 Sep 2019 22:10:40 +0000

Sep 6, 22:10 UTC
Completed - The scheduled maintenance has been completed.

Sep 6, 21:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Sep 4, 18:29 UTC
Scheduled - We will be updating our database on Friday evening, leading to brief unavailability of the app (up to 1 hour though we expect less than 30 minutes).

Database Update

Wed, 19 Jun 2019 23:23:42 +0000

Jun 19, 23:23 UTC
Completed - The scheduled maintenance has been completed.

Jun 19, 23:22 UTC
Verifying - Verification is currently underway for the maintenance items.

Jun 19, 23:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Jun 18, 23:39 UTC
Scheduled - We will be undergoing scheduled maintenance during this time. We are planning for 30 minutes of downtime, but expect no more than 10-15 minutes.

Database Update

Wed, 22 May 2019 22:13:24 +0000

May 22, 22:13 UTC
Completed - The scheduled maintenance has been completed.

May 22, 22:12 UTC
Verifying - Verification is currently underway for the maintenance items.

May 22, 22:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

May 22, 00:26 UTC
Scheduled - We will be undergoing scheduled maintenance during this time. We are planning for 30 minutes of downtime, but expect no more than 10-15 minutes.

Database Update

Thu, 09 May 2019 23:28:16 +0000

May 9, 23:28 UTC
Completed - The scheduled maintenance has been completed.

May 9, 23:26 UTC
Verifying - Verification is currently underway for the maintenance items.

May 9, 23:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

May 9, 18:57 UTC
Scheduled - We will be undergoing scheduled maintenance and performing database updates during this time. We expect downtime of no more than 30 minutes, but please plan for up to 1 hour of unavailability.

Database Update

Fri, 15 Feb 2019 01:30:55 +0000

Feb 15, 01:30 UTC
Completed - The scheduled maintenance has been completed.

Feb 15, 01:16 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Feb 14, 21:33 UTC
Scheduled - We will be performing database updates from 1:15am-1:30am UTC on February 15 (8:15pm-8:30pm Eastern on February 14). We expect downtime of no more than 15 minutes, with likely downtime of only several minutes.