Pulumi — simplifying Infrastructure as Code
Managing IT infrastructure at the pace of ever-evolving tech. Especially if you plan to do it manually, my advice would be “brace yourselves.” Advancements in Cloud, Agile, and DevOps practices have revolutionized designing, developing, and maintaining the IT infrastructure. Infrastructure as Code (IaC) is one of the critical components of these practices. IaC manages your IT infrastructure using configuration files and provides benefits such as enhanced speed, efficiency, and consistency.
Let’s see what Pulumi has in store for us when it comes to IaC.
Pulumi is an Infrastructure as code platform that allows you to use familiar programming languages and tools to build, deploy, and manage cloud infrastructure.
You can create, deploy and manage infrastructure as code on any cloud. Pulumi offers a desired state IaC model where the code represents the desired infrastructure state. The deployment engine compares this desired state with the stack’s current state to determine what resources need to be created, updated, or deleted. Pulumi supports all the leading cloud providers, including AWS, Azure, Google Cloud, and other services like CloudFlare and Digital Ocean.
Pulumi Stack, State, and Backend
Pulumi stores metadata about your infrastructure to manage cloud resources, known as the state. Each Pulumi stack has its state that allows Pulumi to understand when and how to create, read, delete, or update cloud resources. The Pulumi program is usually deployed to a stack, an isolated, independently configurable instance of a Pulumi program. More precisely, the stack represents different phases of development or feature branches.
A backend is an API and storage endpoint that allows the CLI to coordinate updates and read & write stack states whenever required. The state can be stored in a backend of your preference. Pulumi Service is an easy-to-use, secure, and reliable hosted application having policies and safeguards. It facilitates team collaboration and supports simple object storage in AWS S3, Google Cloud Storage, Microsoft Azure Blob Storage, and any AWS S3 compatible server like Minio or Ceph or a local filesystem.
By default, you can use hosted Pulumi service that takes care of state and backend settings. If you use cloud storage or a local file system as a backend, you can choose your state’s location. However, in this scenario, you have to take care of the security, state management, auditing. Pulumi does not store cloud credentials; it only stores configuration and secrets. Encryption providers can help you do the job of encrypting these secrets. Learn more about secrets and configurations here.
Pulumi supports two classes of backends for storing your infrastructure state viz., service and self-managed.
- Service: This includes managed cloud experience using the online or self-hosted Pulumi Service application.
- Logging Into the Pulumi Service Backend: The basic form of login will use the Pulumi Service by default.
- Logging Into a Self-Hosted Pulumi Service Backend: If you wish to log in to a specific self-hosted backend, pass the backend-specific URL as the sole argument. Alternatively, you may set the PULUMI_BACKEND_URL environment variable.
pulumi login <backend-url>
- Self-Managed: This includes manually managed object store, including AWS S3, Azure Blob Storage, Google Cloud Storage, any AWS S3 compatible server such as Minio or Ceph, or your local filesystem. State management, including backup, sharing, and team access synchronization, is custom and implemented manually for self-managed backends. To use a self-managed backend, specify a storage endpoint URL as Pulumi login’s <backend-url> argument.
- AWS S3
pulumi login s3://<bucket-path>
- Azure Blob
pulumi login azblob://<container-path>
- Google Cloud Storage
pulumi login gs://<bucket-path>
- Local storage
pulumi login –local OR
pulumi login file://<fs-path>
Let me explain Pulumi in detail with a simple Pulumi program.
Pulumi AWS Example (Python):
Let’s write one simple example to create a VPC and a few subnets on AWS.
1. Pulumi Installation
You have to set up your environment by installing Pulumi, your preferred language runtime, and configuring your AWS credentials.
- To install the Pulumi on MacOS, run the following command:
brew install pulumi
- To install the Pulumi on Linux, run the following command:
curl -fsSL https://get.pulumi.com | sh
2. Pulumi requires cloud credentials to manage and provision resources. You must use an IAM user account with programmatic access with rights to deploy and manage resources handled through Pulumi. You can set the credentials for AWS cloud either using AWS CLI or by exporting them in OS environment variables.
Use the following command to set the credentials using AWS CLI:
Set the credentials in OS environment by executing commands:
Once you complete the installation, you need to create a Pulumi project.
3. Pulumi project creation
Now, create a new empty directory for the Pulumi project. In the new empty directory, run the following command:
pulumi new aws-python
This command will ask for project name, project description, stack name, AWS region name. It will start the installation for required dependent python libraries. Once installed, you can see the files, including Pulumi.<stack_name>.yaml, Pulumi.yaml, __main__.py , requirements.txt, and venv directory…read more.