Create a secure load testing environment with AWS Database Migration Service (DMS)'s new "Data Masking" feature.
#SRGThe Service Reliability Group primarily provides comprehensive support for the infrastructure surrounding our media services, focusing on improving existing services, launching new ones, and contributing to open-source software (OSS).
This article explains how to build a MySQL load testing environment with personal information masked using the new data masking feature of AWS DMS.
I hope this is of some help.
AWS Database Migration Service adds a new feature: data masking.Let's actually try masking.DB advance preparationDMS creationLet's check the data.In conclusion
AWS Database Migration Service adds a new feature: data masking.
The data masking feature is a new feature of the Database Migration Service (DMS) released on November 25, 2024.
The "Conversion Rules" in DMS's "Database Migration Tasks" allowed for data conversion that could be used for migration to other databases, which was possible more easily than before.
With this release, the following actions can now be performed:
- Number masking
- Randomization of numbers
- Hash masking
Let's actually try masking.
DB advance preparation
First, from the RDS screen, create source cluster A and target cluster B.
Connect to cluster A and create a verification table like the one below.
Create a person table
Add a record to the person table.
DMS creation
Next, we will create a replication instance from the DMS screen to relay data.
"Replication Instance" → "Create Replication Instance"
3.5.4As of January 2025, the default version is 3.5.3, so please be aware of this.
Next, select "Database Migration Tasks" → "Create Data Migration Task".
Give the task identifier an appropriate name,
For the replication instance, select the relay instance that we created earlier.
Select a writer from cluster A for the source database and a writer from cluster B for the target database.
Scroll down and click "Add New Selection Rule".
Click on "Add conversion rule" that appears.
Enter the rules as follows:
Rule target: column
Source name: Schema input (test)
Source table name: person
Column name: tel
Action: Masking numbers
ハッシュマスキング
Once you have finished creating the task, the full load and conversion from cluster A to cluster B will start automatically.
Let's check the data.
First, connect to cluster A and view the record list to see the original data.
Next, connect to cluster B and check the record list.
As per the conversion rules, we were able to confirm that name and address have been hashed, and that the numbers in tel have been replaced with x.
By using this Cluster B endpoint for connections from load testing applications, it becomes possible to handle production-level data volumes while protecting personal information.
In conclusion
Previously, data masking had to be implemented using Lambda or similar tools, but this new DMS feature makes the conversion process much easier.
It's incredibly helpful to be able to quickly establish a safe load testing environment!
SRG is looking for new team members.
If you are interested, please contact us here.