Automating IoT Cloud Infrastructure with Terraform and Soracom
Written by: Brenna Belletti
Published: March 25, 2020
Introduction
When I joined Soracom, as with starting any new company, I went through a ramp up period where I had the opportunity to read through documentation and play around with the technology. Soracom has detailed getting started guides for this purpose, including one that caught my eye describing how to connect Soracom’s Beam Service to AWS IoT.
Soracom Beam is a great tool for optimizing data and power usage in IoT devices while keeping them securely connected to the cloud. I’m already deeply familiar with AWS, but working from the device side I found that setting up AWS for quick development and prototyping can be a bit cumbersome. When building a new project, it’s not as simple as building the resources you want. You need to create policies, create roles, assign the right role to the right resources, link your resources together, and so on. What happens if one of these resources changes? You have to go and update all of the other resources that were pointing to it. Want to delete everything you just created? Have fun keeping track of it all.
I say this as a huge fan of the power and flexibility of AWS! I just think we can move a bit faster while simultaneously making our lives easier. This is where Terraform comes in.
Terraform is an open-source infrastructure as code tool created by HashiCorp that lets you write all of your resources in code, maintain them, and then seamlessly tear them down as needed. All changes are made in the code which opens up the door for collaboration and version control. If any changes are accidentally made on your AWS resources, Terraform has integrated state management to recognize it and revert it back.
Getting Started With Terraform
The number one reason why I am obsessed with Terraform is that it’s very simple to set up. All that is needed to get started is Terraform, the AWS CLI, and a file ending in .tf to put your resources in. Once set up, you can start adding resources to your file. For example, to create an AWS IoT Thing with the name “my-raspi” use the aws_iot_thing type of resource like this:
provider "aws" {
region = "us-west-2"
version = "~> 2.0"
}
resource "aws_iot_thing" "beamtest_iot_thing" {
name = "my-raspi"
}
Running the commands terraform plan and terraform apply will create this AWS IoT Thing in your account. You can now see how easy it is to add in more resources, policies, and roles, and connect to the AWS Simple Notification Service. Running the Terraform commands again will recognize the changes and build them in your AWS account.
You can find my full Terraform buildout on GitHub the Soracom Beam to AWS IoT Getting Started Guide. This Terraform code will take in IMSI IDs of your Soracom sim cards and create an AWS IoT Thing for each ID. It will automatically create the policies and configuration needed to receive MQTT messages from Soracom Beam and then forward those messages to an SNS topic that can send a text message to a number you specify.
You can take advantage of the Soracom API or CLI to get your list of IMSIs by listing subscribers, or listing subscribers in a certain group. For example, use this CLI command to list all subscribers:
soracom subscribers list | grep -o '"imsi": *"[^"]*"' | grep -o '"[^"]*"$'
(The two grep commands filter down the result to return just the list of IMSIs without the extra attributes.)
An advantage of Soracom Beam, our data transfer service, is that it simplifies this process for you by storing those certificates on the cloud. You can do this through the console as specified in the Getting Started Guide or by using the Soracom createCredential API. Having access to the API means that at scale, you can automate this step as well by using something like this Terraform provider for REST APIs. Storing the credentials on the cloud has other advantages as well:
As noted in the Developer Overview for Soracom Beam: When a Soracom Air SIM device connects to a cellular network, data transmission is secured through SIM authentication and closed carrier infrastructure, and passed to the Soracom platform over dedicated direct lines. Since communication between a device and Soracom is secured, you can utilize lightweight protocols such as HTTP or MQTT to reduce cellular data usage and power consumption, and then perform encryption in the cloud for protecting data between Soracom and your network environment.
Once Beam is configured with your certificates, you can move on to the testing section of the Getting Started Guide. Subscribe to the ‘beamtest’ topic in the AWS IoT dashboard and then send your MQTT message to Beam!
mosquitto_pub -d -h beam.soracom.io -t beamtest -m 'Hello, world!'
You will receive the message at the number you specified in your configuration.
Scalability: An Unexpected Perk
It was an easy decision to build the AWS IoT infrastructure in the Soracom Getting Started Guides using Terraform. I knew that I might want to build it up and tear it down a few times, make small changes quickly, and just be able to have all of this infrastructure on hand and ready to build quickly in the future.
An unexpected perk of this decision was once I had it set up for one IoT device, it was easy to extend it for multiple IoT devices when continuing on to the next Soracom Getting Started Guide: Using Multiple Credentials with Beam. If you asked me to build one hundred IoT devices by hand in the AWS console, I probably would have done it but I can promise you I would not have been happy about it.
This kind of automation is what makes scaling easy. As you scale up, Terraform handles everything on the cloud side, and also handles the creation of IoT certificates. Since certificates are stored at the Soracom level, not on the devices themselves, it becomes easy to add or remove as many devices as needed.
Next Steps
By now the benefits of automation are clear; agility, reusability, scalability, and manageability. Also, it’s easy to tear everything down by simply running terraform destroy so there’s no need to worry about orphaned resources running up charges in your account. Some things I would have done to take this further, as mentioned previously, would be to connect it directly to the Soracom APIs. When the IoT thing is created and its certificates are generated, they are also automatically added into Soracom.
If you are looking to use Terraform in your production environment, you will want to integrate the above workflow into a CICD pipeline. Hashicorp has an offering called Terraform Cloud for this purpose, but you could also use traditional automation servers like Jenkins or AWS CodePipeline.
An idea I have been playing around with is to create my own Terraform Provider for Soracom! Speaking of, Terraform has many providers, so this could easily be ported over to Azure IoT or Google IoT. I would be curious to hear your ideas on how to expand on this infrastructure further. Feel free to reach out to me with any questions or comments.
