Pulumi — simplifying Infrastructure as Code

Managing IT infrastructure at the pace of ever-evolving tech. Especially if you plan to do it manually, my advice would be “brace yourselves.” Advancements in Cloud, Agile, and DevOps practices have revolutionized designing, developing, and maintaining the IT infrastructure. Infrastructure as Code (IaC) is one of the critical components of these practices. IaC manages your IT infrastructure using configuration files and provides benefits such as enhanced speed, efficiency, and consistency.

IaC facilitates faster infrastructure deployments as the teams don’t have to maintain the settings of individual deployment environments. There are several IaC solutions available in the market. Terraform is the current industry leader in IaC. It is a fantastic tool; however, you must be fluent in Terraform Domain-Specific Language like HashiCorp Configuration Language (HCL). Pulumi is another open-source IaC tool used to configure, deploy, and maintain cloud resources. The interesting thing about Pulumi is it supports languages such as Python, JavaScript, TypeScript, Go, and .NET. You can develop reusable functions, packages, classes, and Pulumi components with these languages. Pulumi also supports a superset of the providers that Terraform offers currently.

Let’s see what Pulumi has in store for us when it comes to IaC.

Pulumi

Pulumi is an Infrastructure as code platform that allows you to use familiar programming languages and tools to build, deploy, and manage cloud infrastructure.

You can create, deploy and manage infrastructure as code on any cloud. Pulumi offers a desired state IaC model where the code represents the desired infrastructure state. The deployment engine compares this desired state with the stack’s current state to determine what resources need to be created, updated, or deleted. Pulumi supports all the leading cloud providers, including AWS, Azure, Google Cloud, and other services like CloudFlare and Digital Ocean.

Pulumi Stack, State, and Backend

Pulumi stores metadata about your infrastructure to manage cloud resources, known as the state. Each Pulumi stack has its state that allows Pulumi to understand when and how to create, read, delete, or update cloud resources. The Pulumi program is usually deployed to a stack, an isolated, independently configurable instance of a Pulumi program. More precisely, the stack represents different phases of development or feature branches.

A backend is an API and storage endpoint that allows the CLI to coordinate updates and read & write stack states whenever required. The state can be stored in a backend of your preference. Pulumi Service is an easy-to-use, secure, and reliable hosted application having policies and safeguards. It facilitates team collaboration and supports simple object storage in AWS S3, Google Cloud Storage, Microsoft Azure Blob Storage, and any AWS S3 compatible server like Minio or Ceph or a local filesystem.

By default, you can use hosted Pulumi service that takes care of state and backend settings. If you use cloud storage or a local file system as a backend, you can choose your state’s location. However, in this scenario, you have to take care of the security, state management, auditing. Pulumi does not store cloud credentials; it only stores configuration and secrets. Encryption providers can help you do the job of encrypting these secrets. Learn more about secrets and configurations here.

Pulumi Backends

Pulumi supports two classes of backends for storing your infrastructure state viz., service and self-managed.

• Service: This includes managed cloud experience using the online or self-hosted Pulumi Service application.
• Logging Into the Pulumi Service Backend: The basic form of login will use the Pulumi Service by default.

pulumi login

• Logging Into a Self-Hosted Pulumi Service Backend: If you wish to log in to a specific self-hosted backend, pass the backend-specific URL as the sole argument. Alternatively, you may set the PULUMI_BACKEND_URL environment variable.

pulumi login <backend-url>

• Self-Managed: This includes manually managed object store, including AWS S3, Azure Blob Storage, Google Cloud Storage, any AWS S3 compatible server such as Minio or Ceph, or your local filesystem. State management, including backup, sharing, and team access synchronization, is custom and implemented manually for self-managed backends. To use a self-managed backend, specify a storage endpoint URL as Pulumi login’s <backend-url> argument.
• AWS S3

pulumi login s3://<bucket-path>

• Azure Blob

pulumi login azblob://<container-path>

• Google Cloud Storage

pulumi login gs://<bucket-path>

• Local storage

pulumi login –local OR   
pulumi login file://<fs-path>

Let me explain Pulumi in detail with a simple Pulumi program.

Pulumi AWS Example (Python):

Let’s write one simple example to create a VPC and a few subnets on AWS.

1. Pulumi Installation

You have to set up your environment by installing Pulumi, your preferred language runtime, and configuring your AWS credentials.

• To install the Pulumi on MacOS, run the following command:

brew install pulumi

• To install the Pulumi on Linux, run the following command:

curl -fsSL https://get.pulumi.com | sh

2. Pulumi requires cloud credentials to manage and provision resources. You must use an IAM user account with programmatic access with rights to deploy and manage resources handled through Pulumi. You can set the credentials for AWS cloud either using AWS CLI or by exporting them in OS environment variables.

Use the following command to set the credentials using AWS CLI:

aws configure

Set the credentials in OS environment by executing commands:

export AWS_ACCESS_KEY_ID=<YOUR_ACCESS_KEY_ID>	   
export AWS_SECRET_ACCESS_KEY=<YOUR_SECRET_ACCESS_KEY>

Once you complete the installation, you need to create a Pulumi project.

3. Pulumi project creation

Now, create a new empty directory for the Pulumi project. In the new empty directory, run the following command:

pulumi new aws-python

This command will ask for project name, project description, stack name, AWS region name. It will start the installation for required dependent python libraries. Once installed, you can see the files, including Pulumi.<stack_name>.yaml, Pulumi.yaml, __main__.py , requirements.txt, and venv directory…read more.