Testing the new AWS Security Agent
IntroductionAWS Security Agent Overview and Features1. Design Review2. Code Security Review3. Penetration TestingVerification: Diagnosis using the homemade app VulnBankSetting up AWS Security AgentConfiguring Code ReviewPenetration Test SettingsSummary of this article
Introduction
My name is Taninari, and I work in the Service Reliability Group (SRG) of the Media Headquarters.
#SRG(Service Reliability Group) is a group that mainly provides cross-sectional support for the infrastructure of our media services, improving existing services, launching new ones, and contributing to OSS.
This article is based on the "AWS Security AgentThis article aims to provide information for decision-makers considering the introduction and utilization of new services by actually testing "AI-based design review" and clarifying the effectiveness of AI-based design review and the challenges of dynamic testing.
At re:Invent 2025 in December 2025, AWS announced "AWS Security Agent," a new security service that leverages generative AI.
While previous security tools have focused on matching known patterns and rule-based detection, this new service stands out in that it uses AI to understand the context of the application.
Its appeal lies in its end-to-end support throughout the development lifecycle, from design to implementation and ongoing testing, but what about its actual usability and accuracy?
In this article, we report on the results of testing the preview version of AWS Security Agent using "VulnBank," a vulnerability app we created for educational purposes.
I plan to divide this article into two parts. In the first part, I will introduce the overview of AWS Security Agent and how to set up AWS Security Agent. In the next part, I will summarize by explaining how each feature works.
AWS Security Agent is currently in preview and available only in US East (N. Virginia) as of December 2025.
AWS Security Agent Overview and Features
AWS Security Agent is a service that uses AI to analyze application blueprints, source code, and operating environments to detect security risks and propose remediation measures.
Its greatest feature is that it goes beyond simply detecting code defects by reading documents such as architecture diagrams and requirements specifications to understand the intent of the entire system before assessing risks.
Traditional static analysis (SAST) and dynamic analysis (DAST) tools have issues with high false positives and overlooking vulnerabilities caused by business logic.
AWS Security Agent aims to perform security reviews from a perspective closer to, or even better than, that of a human, by using generative AI to interpret what data the app handles and what flow it follows.
This service consists of the following three main functions.
1. Design Review
During the design stage before implementation, we upload architecture diagrams and specifications (Markdown or PDF) and identify security risks in the design.
We point out deficiencies in authentication and authorization, as well as data handling issues, based on standards such as the AWS Well-Architected Framework and OWASP Top 10.
2. Code Security Review
It works with repositories such as GitHub and analyzes code when a pull request is created.
It detects SQL injections, hard-coded secrets, and more, not just by looking at the difference in changes, but also by taking into account information from the entire repository and related design documents.
Another major feature is that it doesn't just point out issues, but also suggests the code to be fixed.
3. Penetration Testing
An AI agent performs simulated attacks on live web applications to verify vulnerabilities.
It can also test private resources within a VPC, and if a vulnerability is discovered, it will provide reproduction steps and suggested fixes.
Verification: Diagnosis using the self-made app VulnBank
This time, I created a web application called "VulnBank" using Go that intentionally contains vulnerabilities. The application intentionally contains the following vulnerabilities:
- SQL injection(Multiple places) - Query construction by string concatenation
- IDOR（Insecure Direct Object Reference）- Lack of authorization checks
- Broken Access Control- Unauthorized access to administrator functions
- Hard-coded secrets- JWT private key, administrator password, etc.
- Weak password policies- Lack of password validation
/debug/*
- Mass Assignment- Set any field from the request
Setting up AWS Security Agent
We will set up the AWS Security Agent to use it.
First, create an agent space. All you need to create is a name and description.
Once creation is complete, it will look like this:
Penetration testingConfiguring Code Review
We will configure each one individually.
Enable code review
Install and authorize
Install & Authorizeenable code review
Pentest remediation enabledCode reviewPenetration Test Settings
ペネトレーションテストをセットアップ
This time, I wanted to deploy the application to EC2 and test it on the EC2 application, so I verified it using the HTTP route.
As I will explain later, this method ultimately resulted in a failed status and did not work.
到達不能検証トークン失敗In this state of failure
- Change port from 8080 to 80
- Change port from 80 to 443 (self-signed certificate)
- Assign a domain and try accessing via the domain
- Create a certificate with ACM and change access to via ALB
失敗DNS テキストレコード検証済み
Once the above settings are complete, you can confirm that all functions are now available.
Summary of this article
In this blog, we introduced AWS Security Agent and how to set it up.
失敗Next time, I'll write a blog post summarizing the code reviews and penetration tests I actually performed.
SRG is looking for people to work with us.
If you are interested, please contact us here.