IBM Analytics Engine’s Sparkling New Avatar!
Serverless Apache Spark
Run Pay-Per-Second, On-Demand, Spark Workloads
IBM Analytics Engine’s recently launched new plan Standard Serverless for Apache Spark, comes with a host of exciting new features for teams that need to run BigData, Machine Learning, Spark based workloads.
You, as a Developer, will dig:
- Running on latest and greatest versions :- Spark on Scala, Python, R
- You can just submit your spark application without worrying about setting up a Spark cluster, queues, priority of your application etc. Let Analytics Engine take care of bringing up dedicated internal Spark cluster for your app, in few seconds and get your job completed.
- You don’t have to think of how many executors/cores are needed to run your app.Make use of auto scaling feature and let the workload drive the executor — memory/cpu usage. If, on the other hand, you prefer to specify the executor/CPU upfront, you can pick from a standard ratio that suits your needs.
- Interface of your choice :- CLI, REST API, SDK, Livy like interfaces..
- Customization per your needs:- Bring in external standard or custom files packages and libraries, Specify default configurations for Spark..
You, as the Admin / Security person will love:
- That there is no runtime cluster to manage; only manage the Service Instance (the logical construct that houses the Spark configurations and COS instance for customization libraries)
- Granular control of team members accessing the Service Instance.
- Auditability — who did what, when?
- Compute and Network Isolation
You, as DevOps will like:
Your Finance team can count on:
- Pay-per-second billing:- Pay exactly only for the resources that you use and pay only for as long as the workload runs.
- Limits and Quotas :- Know in advance, the max limits of resource consumption that can be hit by team, so that you know that final bill will not exceed X $$$.
- Auto Scaling Feature:- Letting your workload determine the resources needed in the most optimized way as opposed to a fixed estimate ..
Have More Questions?
Try it out!
From all of us at the dev team :)
Avani G, Chetan B, Darshan K, Deepali G, Deepashree G, Dharmesh J, Harshavardhan C, Jaytheerthan K, Kapil J, Madhusudan KJ, Mrudula M, Shiv P Y, Shivangi G, S Kulki, Shyamala G, Silpi D, Subin S, Tapas S