Provisioning Analytics Engine — Serverless Spark using Terraform

3 min readMar 27, 2022

Terraform — Overview

Terraform is an open source software from Hashicorp that enables you to easily provision and manage resources of your cloud provider using a simple declarative json like configuration files to define your requirements.

Terraform simplifies the provisioning aspects especially when you have to deal with multi clouds and multiple environments. Cloud providers have the predefined templates and plugins that makes it easy to be able to provision, modify and delete infrastructure resources.

Step1:- Install the Terraform plugin

Based on your platform — you can install the right version of Terraform plugin. Choose one from here.

On my linux based system, these are the steps, I executed:

wget https://releases.hashicorp.com/terraform/1.1.7/terraform_1.1.7_linux_amd64.zipunzip terraform_1.1.7_linux_amd64.zipexport PATH=$PATH:$HOME/terraform

For more details see here.

Once you install, you can execute the terraform binary and check if it is working fine:

Step2:- Configure the IBM Cloud Provider Plugin

To access the IBM Cloud and to be able to provision resources — you need to provide the required credentials, i.e the IAM APIKey. Additionally you need to provide the IBM Cloud Provider Plugin files.
The relavent files are provider.tf, versions.tf and terraform.tfvars.

provider.tf

variable "ibmcloud_api_key" {}
variable "region" {}provider "ibm" {
    ibmcloud_api_key   = var.ibmcloud_api_key
    region = var.region
    }

versions.tf

terraform {
required_version = ">=1.0.0, <2.0"
required_providers {
    ibm = {
    source = "IBM-Cloud/ibm"
    }
 }
}

terraform.tfvars
This contains the APIKey of cloud.ibm.com account where you want to create the instance of Analytics Engine

ibmcloud_api_key = "blahablahblahblahblah"
region = "us-south"

Step3:- Analytics Engine Serverless Terraform provision payload

Depending on the IBM Cloud resource that you are trying to create, you the terraform file will change. In this case, we are creating the Analytics Engine Serverless instance.

ae.tf

data "ibm_resource_group" "group" {
  name = "test"
}
resource "ibm_resource_instance" "resource_instance" {
  name              = "my-demo-terraform-test"
  service           = "ibmanalyticsengine"
  plan              = "standard-serverless-spark"
  location          = "us-south"
  resource_group_id = data.ibm_resource_group.group.id
  tags              = ["test"]
  parameters_json = <<PARAMETERS_JSON
  {
  "default_runtime": {
    "spark_version": "3.1"
    },
  "instance_home": {
    "region": "us-south",
    "endpoint": "https://s3.us-south.cloud-object-storage.appdomain.cloud",
    "hmac_access_key": "9dasdfasdfasdfasdfasdfasdf9e4b1",
    "hmac_secret_key": "9dasdfasdfasdfasdfasdfasdf9e4b1"
    }
 }
 PARAMETERS_JSON
}