Provision Cloud Infra using Terraform

How to Provision IBM Cloud Analytics Engine instance

Terraform — Overview

Terraform is an open source software from Hashicorp that enables you to easily provision and manage resources of your cloud provider using a simple declarative json like configuration files to define your requirements.

In general, using Terraform simplifies the provisioning aspects especially when you have to deal with multi clouds and multiple environments. Almost all cloud and other infra providers have the predefined templates and plugins that makes it easy to use and navigate — so that you, as a devops engineer, don’t have to get into the details of the API or write custom script specific to that provider. The idea is to be able to provision, modify and delete infrastructure resources without having to login to the UI and change it by hand — Infrastructure as Code. Here’s a quote from the blog that explains more as to Why Terraform?

I can begin to both automate that, but also now that I’ve captured it as a codified form, I can check it into version control and I get versioning. Now I can see an incremental history of who changed what, how is my infrastructure actually defined at any given point of time, and I have this transparency of documentation that I lack in a traditional point-and-click environment. It’s really an oral tradition in terms of: What is the configuration? What are the things that you need to point and click and set up?

This article is to show what you can do with Terraform and Analytics Engine 1.2

Three simple steps to get started

Step 1: Install the Terraform plugin

(Finer details are in the link above), this step involves installing the terraform base CLI that allows you to invoke the terraform CLI to invoke the “apply” or “destroy” actions on the cloud resource to provision and delete respectively.

Step 2: Install the IBM Cloud Provider plug-in for Terraform

Every cloud provider needs to provide this plugin to Terraform so that it can invoke the respective operations on the specific provider. Install the IBM plugin as given in the link.

Step 3: Configure the IBM provider plugin

To access the IBM Cloud and to be able to provision resources — you need to provide the required credentials the IAM APIKey and if you are provisioning the IBM classic infrastructure, the relevant credentials for that as well. Here’s where you provide the files provider.tf and terraform.tfvars. You can use the exact files as given in the instructions in the link.

Start provisioning!

You can provision a host of IBM Cloud resources from the catalog — ranging from Databases, Cloud Object Storage, VSIs, VPCs, Kubernetes clusters and much more. Here’s the full list.

Analytics Engine comes under IBM Resource Instance category and requires the configuration to be defined as given here. To know the parameters and values to be used for this category — look at the github template repo.

Here’s a sample ready configuration file for provisioning an instance of Analytics Engine: ae.tf

data "ibm_resource_group" "group" {
  name = "test"
}
resource "ibm_resource_instance" "resource_instance" {
  name              = "my-demo-terraform-test"
  service           = "ibmanalyticsengine"
  plan              = "standard-hourly"
  location          = "us-south"
  resource_group_id = data.ibm_resource_group.group.id
  tags              = ["test"]
parameters = {
        "hardware_config"= "default"
        "num_compute_nodes"= "1"
        "software_package"= "ae-1.2-hive-spark"
 }
}

Once you have all of these files defined. It is as simple as executing the terraform CLI.

Step1 — terraform init

Initializing of the plugins here..

Step 2 — terraform plan

The plan action shows what will happen when you actually execute

Step 3 — terraform apply

This actually executes the action — provision of the cluster in this case

Step 4 — terraform show

Cluster provisioning takes time and once it is done, you can do a show to check the instance details.

And your cluster is ready to use for your analytical workloads.

Limitations

At the time of writing this, using Terraform:

• You cannot do a resize operation for Analytics Engine. Changing the number of nodes will result in destroying the instance and creating a new one.
• Also it is not possible to provide the advanced configuration option in the analytics engine template. Resource Instance type accepts only key-value pairs of primitive types.