A journey to fix the bug in Aurora MySQL version 3 where tables become invisible from the reader instance.
This is Yuta Kikai (@fat47) from the Service Reliability Group (SRG) of the Media Division.
#SRGThe Service Reliability Group primarily provides comprehensive support for the infrastructure surrounding our media services, focusing on improving existing services, launching new ones, and contributing to open-source software (OSS).
This article summarizes an issue where tables become invisible from the reader endpoint during DDL execution in Aurora MySQL version 3.
I hope this is of some help.
A bug where the table becomes invisible to the leader endpoint during DDL execution.Actually, this bug has been repeatedly fixed.2023-10-25 3.05.0 (MySQL MySQL 8.0.32 compatible)2024-03-15 3.04.2 (MySQL MySQL 8.0.28 compatible)2024-11-18 3.08.0 (MySQL 8.0.39 compatible)2026-02-17 3.12.0 (MySQL 8.0.44 compatible)2026-05-21 Aurora MySQL 8.4.7 (MySQL 8.4.7 compatible)2026-07-30 Aurora MySQL 3.10.5 (MySQL 8.0.42 compatible)Bug fixes implemented in Aurora MySQL 3.08 (compatible with MySQL 8.0.39)Patterns that reproduce the bug in Aurora MySQL 3.08 (compatible with MySQL 8.0.39)The bug persists in Aurora MySQL 3.12.0 (MySQL 8.0.44 compatible).Aurora MySQL 8.4.7 (MySQL 8.4.7 compatible) has fixed the bug.Bug fixes also available in Aurora MySQL 3.10.5 (compatible with MySQL 8.0.42).In conclusion
A bug where the table becomes invisible to the leader endpoint during DDL execution.
A bug has been occurring since Aurora MySQL v3 where the relevant table becomes invisible from the reader endpoint during DDL execution.
This bug itselfThis issue was resolved in v3.04.2 (MySQL 8.0.28 compatible), released in March 2024.Although it has been fixed, there are still situations where the bug can be reproduced.
I summarized this in the following blog post back in July 2024.
The blog post is about what happens after the first DDL is completed.If a second DDL is executed without any SELECT queries being made to the target table in the leader endpoint, the target table will become invisible to the leader endpoint during the DDL execution.。
This bug was reproducible even in the then-latest version, v3.07.0 (compatible with MySQL 8.0.36).
Actually, this bug has been repeatedly fixed.
In fact, this type of bug that causes such behavior has been fixed in subsequent releases of Aurora MySQL.
2023-10-25 3.05.0 (MySQL MySQL 8.0.32 compatible)
Fixed an issue where the reader instance is unable to open a table, with ERROR 1146. This issue occurs when executing certain types of online Data Definition Language (DDL) while the INPLACE algorithm is being used on the writer instance.
2024-03-15 3.04.2 (MySQL MySQL 8.0.28 compatible)
Fixed an issue where the reader DB instance is unable to open a table, with ERROR 1146. This issue occurs when running certain types of online Data Definition Language (DDL) statements while the INPLACE algorithm is being used on the writer DB instance.
2024-11-18 3.08.0(Compatible with MySQL 8.0.39)
Fixed an issue that, in rare cases, caused either intermittent unavailability of an Aurora read replica or table definition inconsistencies, sometimes with the error Table does not exist, on the replica. This is due to concurrent read queries on the replica and Data Definition Language (DDL) operations on the writer DB instance.
2026-02-17 3.12.0 (MySQL 8.0.44 compatible)
Fixed an issue, which in rare cases, caused intermittent unavailability of an Aurora Read Replica or table definition inconsistencies with error 'Table does not exist' on the replica due to concurrent read queries on the replica and DDL operations on the writer.
2026-05-21 Aurora MySQL 8.4.7(Compatible with MySQL 8.4.7)
Fixed an issue where the reader reports "ERROR 1146" (table not found) during certain online DDL operations on the writer when using INPLACE algorithm. This can occur when the reader has not previously opened the table before the DDL begins. It can also occur when the reader restarts or a new reader is created while the DDL is in progress.
2026-07-30 Aurora MySQL 3.10.5 (MySQL 8.0.42 compatible)
Fixed an issue where the reader reports ERROR 1146 (table not found) during certain online DDL operations on the writer when using the INPLACE algorithm. This can occur when either the reader has not previously opened the table before the DDL begins, or the reader restarts or a new reader is created while the DDL is in progress.
Bug fixes implemented in Aurora MySQL 3.08 (compatible with MySQL 8.0.39)
I'll now test the procedure that allowed me to reproduce the bug up to v3.07, as described in my previous blog post, using v3.08.
Check the version
Create a reproduction table and add approximately 1 million records to it.
Execute two DDL statements.
Executing a SELECT statement on the leader endpoint from another terminal while the second DDL is running will not result in an error. This bug has been fixed in v3.08!
Patterns that reproduce the bug in Aurora MySQL 3.08 (compatible with MySQL 8.0.39)
That should be the end of it! Or so you might think, but there are still other patterns where the same bug occurs.
it isThe table in question is partitioned.This is what's happening. Here are the steps to reproduce it:
First, let's check the version.
We will create a table for reproduction. It is partitioned.
Approximately 3 million records will be inserted into each partition of the writer endpoint.
Example script for INSERT operation
I have confirmed that there are 3 million records.
Execute the DDL that triggers the INPLACE rebuild.
If a SELECT statement is executed against the leader endpoint from another terminal during the second DDL execution (approximately 7 seconds), the issue of the relevant table disappearing can be reproduced.
Once the DDL execution is complete, the table will become visible again.
Aurora MySQL 3.12.0(MySQL 8.0.44 compatible) but the bug persists.
We will now reproduce the bug using the partitioned table we created earlier.
Version check
DDL execution
If a SELECT statement is executed on the leader endpoint during the second DDL execution, the error recurs.
Aurora MySQL 8.4.7 (MySQL 8.4.7 compatible) has fixed the bug.
We will also verify this in the next major version, Aurora MySQL 8.4.7.
Version check
DDL execution
No errors occur when I perform a SELECT query on the leader endpoint during the second DDL execution!
Bug fixes also available in Aurora MySQL 3.10.5 (compatible with MySQL 8.0.42).
Let's check with version 3.10.5, which was just released on July 30, 2026.
Version check
DDL execution
No errors occur when I perform a SELECT query on the leader endpoint during the second DDL execution!
In conclusion
This article discusses a long-standing, deeply rooted bug that has persisted since the release of Aurora MySQL version 3 in 2021.
In this case, we were able to confirm that the bug has been resolved in the latest Aurora MySQL 8.4.7 and Aurora MySQL 3.10.5.
However, the bug may still reappear depending on other table conditions.
When executing DDL on tables with a significant impact, exercise extreme caution.
If you are interested in SRG, please contact us here.