Background
Typically I hear folks say, we should always do TDD with our code. TDD is so significantly better. However what’s TDD? And why is it higher? And the way can or not it’s added to for instance CDK? If you wish to know the WHY, simply observe on…
Primary information of CDK is required although, as I cannot clarify what CDK is. In order for you that, go observe the workshops offered by AWS
- cdkworkshop.com
- cdk-advanced.workshop.aws
TDD
So TDD stands for Check Pushed Improvement. Check Pushed Improvement (TDD) is a software program improvement methodology that emphasizes writing checks earlier than writing the precise code.
The principle aim of TDD is to create a transparent, dependable, and maintainable codebase. That is achieved by writing checks first in order that builders can make sure that their code meets the required specs and behaves as anticipated.
The TDD course of consists of three foremost steps, sometimes called “Pink, Inexperienced and Refactor”:
-
Pink (Write a failing take a look at): On this step, builders ought to write a take a look at for a particular performance or characteristic earlier than implementing the code. The take a look at ought to initially fail, because the corresponding code has not but been written.
-
Inexperienced (Write the code to go the take a look at): Builders then write the minimal code essential to go the take a look at. The main focus is on making the take a look at go, not on creating an optimized or full resolution.
-
Refactor (Enhance the code): As soon as the take a look at passes, builders can refactor the code to enhance its design, readability, and effectivity, whereas guaranteeing that the take a look at nonetheless passes. This step is essential to sustaining a clear and maintainable codebase.
These steps are repeated iteratively for each bit of performance, leading to a complete suite of checks that cowl the whole software.
So what are the advantages you may ask, nicely it helps you with a number of issues:
-
Improved code high quality: Writing checks earlier than the precise code encourages you to assume critically concerning the desired behaviour and design, resulting in cleaner and extra dependable code.
-
Simpler debugging: When a take a look at fails, it is a lot simpler to establish and repair the difficulty, because the take a look at is targeted on particular performance.
-
Sooner improvement: By catching errors early within the improvement course of, TDD helps stop pricey and time-consuming debugging periods afterward.
-
Higher collaboration: A well-tested codebase is simpler for different group members to grasp and modify, facilitating collaboration and decreasing the probability of introducing new bugs.
-
Enhanced maintainability: A complete take a look at suite serves as a security web, guaranteeing that future adjustments to the codebase don’t inadvertently break current performance.
So all and all fairly good. However how on this planet can we apply that to CDK and constructing infrastructure in AWS?
Actual-world state of affairs
In order a real-world state of affairs, I wish to create a easy safe Easy Queue Service (SQS) queue. Following one of the best practices of AWS. So the rules we wish to create for the SQS queue are:
- the Queue should use KMS encryption
- the Queue will need to have a lifeless letter queue to behave as an overflow of messages which may’t be processed.
In a while, we will add a lambda operate which is able to learn the queue and retailer the message in for instance S3.
Go Construct
Now the enjoyable stuff begins. Let’s begin with initializing a brand new CDK mission.
➜ Hashnode mkdir secure_sqs
➜ Hashnode cd secure_sqs
➜ secure_sqs cdk init app --language=python
Making use of mission template app for python
# Welcome to your CDK Python mission!
It is a clean mission for CDK improvement with Python.
<...SNIPPIT...>
✅ All finished!
➜ secure_sqs git:(foremost) supply .venv/bin/activate
(.venv) ➜ secure_sqs git:(foremost)
What I did within the above code block is creating an empty new CDK mission in a listing secure_sqs.
Wanting on the CDK listing construction, you see that there’s a listing referred to as checks. That is the place the place we’ll add our first take a look at.
Write a failed take a look at (RED)
We’ll use the assertions library which comes with CDK. Extra on this may be discovered within the docs of AWS CDK right here.
Open the file checks/unit/test_secure_sqs_stack.py in your favorite code editor. The file is already boilerplate with the next code (see beneath). I’ve uncommented the final block the place it checks if an SQS queue is created:
import aws_cdk as core
import aws_cdk.assertions as assertions
from secure_sqs.secure_sqs_stack import SecureSqsStack
# instance checks. To run these checks, uncomment this file together with the instance useful resource in secure_sqs/secure_sqs_stack.py
def test_sqs_queue_created():
app = core.App()
stack = SecureSqsStack(app, "secure-sqs")
template = assertions.Template.from_stack(stack)
template.has_resource_properties("AWS::SQS::Queue", {
"VisibilityTimeout": 300
})
What occurs right here if you happen to run the python take a look at is that the take a look at tries to examine if the CloudFormation template which is synthesised comprises a property of AWS::SQS::Queue and that the queue has a Visibility Timeout of 300 seconds. As we don’t have any code created the pytest ought to fail. Let’s take a look at this:
(.venv) ➜ secure_sqs git:(foremost) pytest <aws:abn>
============================================================= take a look at session begins =============================================================
platform darwin -- Python 3.9.16, pytest-7.2.0, pluggy-1.0.0
rootdir: /Customers/yvthepief/Code/Hashnode/secure_sqs
plugins: black-0.3.12, typeguard-2.13.3, cov-4.0.0, syrupy-3.0.5
collected 1 merchandise
checks/unit/test_secure_sqs_stack.py F [100%]
================================================================== FAILURES ===================================================================
___________________________________________________________ test_sqs_queue_created ____________________________________________________________
jsii.errors.JavaScriptError:
@jsii/kernel.RuntimeError: Error: Template has 0 assets with kind AWS::SQS::Queue.
No matches discovered
<...SNIPPIT...>
=========================================================== quick take a look at abstract information ===========================================================
FAILED checks/unit/test_secure_sqs_stack.py::test_sqs_queue_created - RuntimeError: Error: Template has 0 assets with kind AWS::SQS::Queue.
============================================================== 1 failed in 3.25s ==============================================================
As you may see the take a look at fails. Because the template has 0 assets with kind AWS::SQS::Queue but. And that is appropriate, as we did not write any single line of code but. Within the Actual-World state of affairs part, we described that we wish the Queue to be encrypted and that it comprises a lifeless letter queue. So create checks for that.
KMS Encrypted Queue
Because the assertions library utilized by CDK is searching for matches with the CloudFormation template, a helpful method may be by trying into the official CloudFormation docs on the AWS::SQS::Queue useful resource. Right here you may see that for including KMS encryption to an SQS Queue, it is advisable have a KmsMasterKeyId.
So begin with making a take a look at that checks if the Queue comprises a KmsMasterKeyId. As we do not know the worth of the important thing, we will use the Match class from the Assertions library.
from aws_cdk.assertions import Match
def test_sqs_queue_is_encrypted():
app = core.App()
stack = SecureSqsStack(app, "secure-sqs")
template = assertions.Template.from_stack(stack)
template.has_resource_properties(
"AWS::SQS::Queue", {"KmsMasterKeyId": Match.any_value()}
)
This block of code above checks if within the CloudFormation template, a useful resource is of kind “AWS::SQS::Queue” and that the useful resource is created with a “KmsMasterKeyId”.
Let’s run the pytest, which is able to fail, in fact, btw I’ve added the -v for verbose, and the –tb=no to disable the traceback:
(.venv) ➜ secure_sqs git:(foremost) ✗ pytest -v --tb=no <aws:abn>
============================================================= take a look at session begins =============================================================
platform darwin -- Python 3.9.16, pytest-7.2.0, pluggy-1.0.0 -- /choose/homebrew/choose/python@3.9/bin/python3.9
cachedir: .pytest_cache
rootdir: /Customers/yvthepief/Code/Hashnode/secure_sqs
plugins: black-0.3.12, typeguard-2.13.3, cov-4.0.0, syrupy-3.0.5
collected 2 objects
checks/unit/test_secure_sqs_stack.py::test_sqs_queue_created FAILED [ 50%]
checks/unit/test_secure_sqs_stack.py::test_sqs_queue_is_encrypted FAILED [100%]
----------------------------------------------------------- snapshot report abstract -----------------------------------------------------------
=========================================================== quick take a look at abstract information ===========================================================
FAILED checks/unit/test_secure_sqs_stack.py::test_sqs_queue_created - RuntimeError: Error: Template has 0 assets with kind AWS::SQS::Queue.
FAILED checks/unit/test_secure_sqs_stack.py::test_sqs_queue_is_encrypted - RuntimeError: Error: Template has 0 assets with kind AWS::SQS::Queue.
============================================================== 2 failed in 3.22s ==============================================================
Now add 1 extra take a look at for the DeadLetterQueue.
Useless letter queue connected to Queue
Within the context of SQS, a lifeless letter queue is a queue that’s used to retailer messages that can not be processed by the primary queue after a sure variety of retries. These messages are sometimes failed messages that would not be processed attributable to points resembling incorrect message formatting or unhandled exceptions within the processing code, f.e. Lambda. Through the use of a DLQ, you may isolate problematic messages and examine the foundation reason behind the difficulty. This may also help you establish and repair points in your software, and forestall the identical errors from occurring sooner or later. Additionally, it offers you the choice to reprocess the failed messages as soon as extra.
Wanting on the CloudFormation documentation of “AWS::SQS::Queue” once more, we will see that it is advisable give a RedrivePolicy with a deadletterTargetArn pointing to a Queue which acts as a lifeless letter queue. So create the take a look at:
def test_sqs_queue_has_dead_letter_queue():
app = core.App()
stack = SecureSqsStack(app, "secure-sqs")
template = assertions.Template.from_stack(stack)
template.has_resource_properties(
"AWS::SQS::Queue", {"RedrivePolicy": {"deadLetterTargetArn": Match.any_value()}}
)
Once more we use the Match any worth assertion as a result of we have no idea the lifeless letter queue arn at this second.
Operating pytest ought to fail 3 checks now:
FAILED checks/unit/test_secure_sqs_stack.py::test_sqs_queue_created - RuntimeError: Error: Template has 0 assets with kind AWS::SQS::Queue.
FAILED checks/unit/test_secure_sqs_stack.py::test_sqs_queue_is_encrypted - RuntimeError: Error: Template has 0 assets with kind AWS::SQS::Queue.
FAILED checks/unit/test_secure_sqs_stack.py::test_sqs_queue_has_dead_letter_queue - RuntimeError: Error: Template has 0 assets with kind AWS::SQS::Queue.
Write the code to go the take a look at (GREEN)
Creating SQS queue with CDK
As we completed our fundamental checks, we now want to put in writing the precise take a look at to go the checks. So create an encrypted SQS queue with a lifeless letter queue connected.
Open the file secure_sqs/secure_sqs_stack.py. This file has a boilerplate as nicely, however we’re going to modify it.
from aws_cdk import (
Length,
Stack,
aws_kms,
aws_sqs,
)
from constructs import Assemble
class SecureSqsStack(Stack):
def __init__(self, scope: Assemble, construct_id: str, **kwargs) -> None:
tremendous().__init__(scope, construct_id, **kwargs)
# Create key with rotation and alias
key = aws_kms.Key(
self,
"SecureQueueKmsKey",
alias="/kms/secure_queue_key",
enable_key_rotation=True,
description="Key for encrypting SQS queue",
)
# Create safe encrypted queue with
# visibility timeout of 300 seconds
queue = aws_sqs.Queue(
self,
"SecureQueue",
queue_name="secure_queue",
encryption=aws_sqs.QueueEncryption.KMS,
encryption_master_key=key,
enforce_ssl=True,
visibility_timeout=Length.seconds(300),
)
Wanting on the code above, we create a Customized Managed KMS key which can have an alias and key rotation enabled. This key might be used to encrypt the SQS queue. We additionally set a queue coverage to solely enable Safe Transport (SSL). Lastly, we set 300 seconds for the visibility timeout, as specified within the checks earlier created.
So with the queue created, let’s take a look at how the checks are operating:
(.venv) ➜ secure_sqs git:(foremost) ✗ pytest -v --tb=no
================================================================== take a look at session begins ==================================================================
platform darwin -- Python 3.11.3, pytest-6.2.5, py-1.11.0, pluggy-1.0.0 -- /Customers/yvthepief/Code/Hashnode/secure_sqs/.venv/bin/python3.11
cachedir: .pytest_cache
rootdir: /Customers/yvthepief/Code/Hashnode/secure_sqs
plugins: typeguard-2.13.3
collected 3 objects
checks/unit/test_secure_sqs_stack.py::test_sqs_queue_created PASSED [ 33%]
checks/unit/test_secure_sqs_stack.py::test_sqs_queue_is_encrypted PASSED [ 66%]
checks/unit/test_secure_sqs_stack.py::test_sqs_queue_has_dead_letter_queue FAILED [100%]
================================================================ quick take a look at abstract information ================================================================
FAILED checks/unit/test_secure_sqs_stack.py::test_sqs_queue_has_dead_letter_queue - RuntimeError: Error: Template has 1 assets with kind AWS::SQS::Q...
============================================================== 1 failed, 2 handed in 5.01s ==============================================================
2 out of three handed already. However it’s nonetheless not the 100% mark we’re aiming for. To have the test_sqs_queue_has_dead_letter_queue handed as nicely, we have to add a Useless Letter Queue. Add the lifeless letter queue between the important thing and queue, and discuss with the lifeless letter queue useful resource for the queue:
from aws_cdk import (
Length,
Stack,
aws_kms,
aws_sqs,
)
from constructs import Assemble
class SecureSqsStack(Stack):
def __init__(self, scope: Assemble, construct_id: str, **kwargs) -> None:
tremendous().__init__(scope, construct_id, **kwargs)
# Create key with rotation and alias
key = aws_kms.Key(
self,
"SecureQueueKmsKey",
alias="/kms/secure_queue_key",
enable_key_rotation=True,
description="Key for encrypting SQS queue",
)
# Create safe encrypted lifeless letter queue with
# visibility timeout of 300 seconds
dead_letter_queue = aws_sqs.Queue(
self,
"SecureDeadLetterQueue",
queue_name="secure_dead_letter_queue",
encryption=aws_sqs.QueueEncryption.KMS,
encryption_master_key=key,
enforce_ssl=True,
visibility_timeout=Length.seconds(300),
)
# Create safe encrypted queue with
# visibility timeout of 300 seconds and discuss with the dlq
queue = aws_sqs.Queue(
self,
"SecureQueue",
queue_name="secure_queue",
encryption=aws_sqs.QueueEncryption.KMS,
encryption_master_key=key,
enforce_ssl=True,
visibility_timeout=Length.seconds(300),
dead_letter_queue=aws_sqs.DeadLetterQueue(
max_receive_count=5, queue=dead_letter_queue
),
)
Shall the checks go now?
(.venv) ➜ secure_sqs git:(foremost) ✗ pytest -v --tb=no
================================================================== take a look at session begins ==================================================================
platform darwin -- Python 3.11.3, pytest-6.2.5, py-1.11.0, pluggy-1.0.0 -- /Customers/yvthepief/Code/Hashnode/secure_sqs/.venv/bin/python3.11
cachedir: .pytest_cache
rootdir: /Customers/yvthepief/Code/Hashnode/secure_sqs
plugins: typeguard-2.13.3
collected 3 objects
checks/unit/test_secure_sqs_stack.py::test_sqs_queue_created PASSED [ 33%]
checks/unit/test_secure_sqs_stack.py::test_sqs_queue_is_encrypted PASSED [ 66%]
checks/unit/test_secure_sqs_stack.py::test_sqs_queue_has_dead_letter_queue PASSED [100%]
=================================================================== 3 handed in 4.98s ===================================================================
(.venv) ➜ secure_sqs git:(foremost) ✗
AWSome, it passes! Now we will construct additional.
As you may see within the examples, I solely have created 3 checks, however trying on the CloudFormation output you may see 4 AWS assets are created. To be compliant and have all of the assets examined, it’s smart so as to add checks for these assets as nicely.
Additional choices can be for instance, making a take a look at for a Lambda which is able to course of the messages within the Safe queue, or including checks for the SQS queue coverage. However that’s all as much as you! Now Go Construct!
Abstract
On this put up, I confirmed how the method ought to work utilizing Check Pushed Improvement with CDK. Begin small and step for step. It’s not smart to create take a look at for all of the assets you’ll create in your finish aim software. This may solely muddle your checks. So crucial is that you just begin with checks, and as soon as extra preserve these checks small. Iteration is crucial issue right here!