importing AWS resources into Terraform

Terraform is a wonderful tool! It helps simplify DevOps work. It turns the thorny bramble of delicate networking, users, databases, and virtual machines into a simple and well-running machine. It allows us to chant "infrastructure as code" to the amusement of well-meaning technologists. Best of all: it lets us have consistent environments. A dev can wreak havoc, learn things, then create a Terraform patch that applies to the entire collection of systems, making everything just a little bit cleaner and better understood.

Terraform, although being a moderately baked and flexible tool, has a few warts. One challenge is that it doesn't play with manually-created resources very well. If you create some users in Terraform, and some users in the AWS Console, applying Terraform later will try to delete the manual users. Terraform imagines that it is the alpha and omega, and that all things are as it thinks they are.

Additionally, Terraform isn't very smart about importing manually-created resources. Traditionally we have to use a third-party tool, terraforming, to do this task. The combination of terraform (to create/update resources) and terraforming (to import manually-created resources) is useful.

Example: here's how to import all the SNS Topics ("snst") to a Terraform file:

$ AWS_PROFILE=myprofile terraforming snst --region=myregion | tee temp-sns.tf
resource "aws_sns_topic" "dynamodb" {
  name            = "dynamodb"
  display_name    = ""
  policy          = ...

}

Now, edit the temp-sns.tf file to make things more clear and regular, then plan and apply with Terraform as per usual.

In AWS, users aren't just users, they're defined in several different types of Identity and Access Management (IAM) resources. Here's how to import just the simple user records:

AWS_PROFILE=myprofile terraforming iamu --region=myregion | tee temp-iamu.tf
resource "aws_iam_user" "john" {
    name = "john@johntellsall.com"
    path = "/"
}

In practice, users aren't useful except as combined with Roles, Groups, and Policies. It's a whole thing. Fortunately, here's a bit of code which imports all AWS IAM user-related permissions into a single Terraform file:

terraforming help | egrep -o 'iam\w+' | AWS_PROFILE=myprofile xargs -I{} -t terraforming {} --region=myregion >> temp-users.tf

Now, you'll be left with a 1,000-line Terraform file for further editing. This isn't that fun, however once you're done, you can move this file into its own module, and apply the same users/groups/permissions on all your environments!

Terraform is a wonderful tool, and in combination with Terraforming and a bit of work, will make your DevOps work a lot simpler!