In the previous blog post, we have described Infrastructure as Code and differences between different types of IaC. Also, described what Pulumi is and created a simple Pulumi project which creates an S3 bucket in AWS.
Following is the AWS component diagram for our application:
We have one VPC, with two subnets: public, private subnet. We want to create our web application in our public subnet using Elastic Beanstalk and create our database in the private subnet, so it is not accessible from outside of the VPC. This network design is a recommended practice to keep your database secure. Our application is a Node.js application running in a docker container in Elastic Beanstalk.
Let's have a look at the folder structure first:
infra folder is for infrastructure for Pulumi and source code for the application written Node.js is in
Let's start with the application source code. It is a simple application that tests the connection to the database. The following code snippet is in the
First, we are assigning a few variables like Port and importing a few packages like MSSQL, HTTP, fs. Then we are defining a function called
try_connect_sql. This function attempts to connect to the database with the connection string passed to it. If it is successful, it returns
true; otherwise, it returns
The next block of code creates a simple HTTP server that responds to
GET requests only. For any
GET request, it attempts to connect to the database, and if the connection is successful, it returns a page with a green background saying Congratulations application is connected to RDS!. If it could not connect to the database, it returns a page with a red background saying Unfortunately application is not connected to RDS!.
In the end, it starts the server by listening to the Port and logs that the server has started.
You can run this simple app locally. If you want to test this, you can run npm start in the src folder. When you browse http://127.0.0.1:3000, if there is no database connection, it returns a page with a red background, but if you have a database running, it returns a page with green background. You can run a SQL database in docker to test the successful path by following this instruction.
package.json file, there are two scripts. The first one is the
start script, and the second one is
package. This script, package up the app folder in zip format, later, our infrastructure code deploys that zip file to Elastic BeansTalk.
The infrastructure code is in the infra folder. It is written in typescript using Pulumi library.
Pulumi encrypts secrets in the configuration file using a
PASS_PHRASE you choose, Pulumi also adds a random salt to the encryption. You can add a secure configuration using
pulumi config set ... --secrets. Sensitive configurations like database password should be added as secret. I didn't commit dbPassword configuration to the YAML file. So you can set database password using the following Pulumi command.
$ pulumi config set vpc_rds_dmz:dbPassword "[STRONG_DB_PASSWORD]" --secret
This command asks you to choose a passphrase for your secrets and confirm it.
Now, let's have a look at the infrastructure code. The main file is in
./infra/index.ts. The programming language for Pulumi code is typescript. Let's go through different blocks of code to describe each block and what they mean.
At the top it just imports different libraries
@pulumi/awsx then, initialize the config object to get some values from the configuration file, in this case, the configuration value is
dbPassword which we set up earlier. Then we create a VPC called custom using
awsx.ec2.Vpc class. This class encapsulates a complete configuration of an AWS network, including the actual VPC itself, in addition to public and private subnets, route tables, and gateways, across multiple availability zones. But in this example, we use private, and public subnet addresses to use for our DB and Elastic Beanstalk, respectively.
This block of code creates a security group using
aws.ec2.SecurityGroup class to enable TCP ingress for SQL Port. Later, we use this security group for our database. The next step is to create a subnet group with private subnet ids using
In this step, we create the RDS itself using
aws.rds.Instance class. There are many parameters for creating rds. However, there are 3 essential parameters. First,
dbPassword which we pulled from our configuration. Second is
dbSubnetGroupName which is set by the subnetGroup created for the private subnet. It means our RDS in the private subnet. And third,
vpcSecurityGroupIds security group which we set to the security group created earlier with SQL Port ingress rule.
The next step is to upload our webapp artifacts to s3 and make them ready for deployment to AWS Elastic Beanstalk. To make the deployment package ready, we just need to run following command line command to create the zip package in
$ npm run package
This command packages up the whole
src folder into a zip file called
deployment.zip. Assuming we had already run this command and zip file is ready, the infrastructure code uploads this zip file to an S3 bucket called
eb-app-deploy which is also created by our infrastructure code.
The next step is to create Elastic Beanstalk. Creating Elastic beanstalk is a bit more complected. There are a few things that should be ready before creating the application and environment.
Instance Profile role - Instance profile role used in instance profile. There is a class in Pulumi AWS SDK to create this profile role
Instance profile - An instance profile is an IAM role that is applied to instances launched in your Elastic Beanstalk environment. When creating an Elastic Beanstalk environment, you specify the instance profile that is used when your instances. In Pulumi SDK we can use
aws.iam.InstanceProfile to create the instance profile.
Then we need to create the Elastic Beanstalk app itself. it is achievable by using
aws.elasticbeanstalk.Application class. It needs at least
name parameter's value, which we have provided and it is called
Application version - Elastic Beanstalk environment needs an application version to deploy the environment. We can create an application version with the
Zip file we uploaded to S3 and configure environment by setting the version parameter.
The next step is to create the connection string. We need to create the connection string based on values from our RDS and also database password (secure parameter we added to Pulumi configuration). Before describing how we construct the connection string we should mention how inputs and outputs are working in Pulumi. Pulumi creates resources asynchronously, which means outputs of a given resource might not be available immediately in the next step in the sequence.
Pulumi uses a special type called output. According to Pulumi documentation:
Outputs are values of type
Output<T> , which behave very much like promises; this is necessary because outputs are not fully known until the infrastructure resource has actually completed provisioning, which happens asynchronously. Outputs are also how Pulumi tracks dependencies between resources.
To construct connection string we need to wait for rds resource to be created then we can get the server address and Port. Pulumi has a utility function called pulumi.all() _This function joins over an entire list of outputs, waiting for all of them to become available, and then provides them to the supplied callback.
This block of code is waiting for both address and Port to become available, and when they are available runs
apply function and return the connection string.
The final step is to create an environment for the Elastic Beanstalk application. This is achievable using
aws.elasticbeanstalk.Environment class. We set the
app parameter and
version to previously created applications and versions. We also set the platform to "64bit Amazon Linux 2018.03 v4.13.1 running Node.js". The last property is setting which is an array of key/values. We set the
VPCId to vpc and subnet to public subnet.
IamInstanceProfile value is set to the instance profile name we created earlier.
SecurityGroups is set to the security group id we created.
CONNECTION_STRING is an environment variable and is set to the value of the connection string variable.
Then the last line returns the endpoint URL as an output so Pulumi prints int out in console after running the Pulumi command line.
Now its time to run the Pulumi and create the stack. Make sure you have created the package.zip and then in
./infra folder run following command:
$ pulumi up
After running this command, Pulumi asks for a passphrase. Provide passphrase you have selected earlier and then select
yes. It'll take a few minutes to create the whole stack. After pulumi created the entire stack successfully, it'll output connection string and Elastic Beanstalk Endpoint. If you copy-paste this URL into your browser, it should return a green page with this message in it:
Congratulations! application is connected to RDS!
Congratulations, we have created an Elastic Beanstalk application backed by secure RDS. Now if you want to destroy this environment, you can simply run following command which removes all resources been created by Pulumi.
$ pulumi destroy
If you want to clone full example, you can access the repository here: https://github.com/ahmad2x4/vpc_rds_dmz_pulumi