Cloud Forms 4.0: Inventory Performance Improvements for VMware Providers

Author – Alex Krzos akrzos@redhat.com

Overview

Of the many improvements and feature enhancements made to Cloud Forms 4.0, one focus was on performance and scalability. One of the first tasks a Cloud Forms Administrator performs while managing a Hybrid cloud is inventory of the environment. Also known as refresh, this task is essential to managing various environments and provides valuable information on every object within the Cloud Forms VMDB.

 

Initial Inventory

One measurement stick used to compare CFME 5.5 to CFME 5.4 is the initial inventory of several different statically sized VMware provider environments. The environments Medium, Large and X-Large are sized at 1,000, 3,000 and 10,000 Virtual Machines. The performance boosts provided in CFME 5.5 are especially observed as the scale of the managed environment increases.

image00

There are 49%, 68% and 87% lower timing values for Medium, Large, and X-Large VMware Environments. This amounts to a huge time savings as shown in the graph.

 

Post-Initial Inventory

Initial inventories are not the only measuring stick we use to compare performance and scalability between releases for provider refresh. Post-Initial inventory represents performing a refresh after we have captured the initial inventory.

image01

Post inventory timings show another respective 27%, 30% and 36% lower timing for Medium, Large and X-Large VMware environments.

 

In summary, the performance improvements of inventory are another great enhancement to Cloud Forms 4.0.

CloudForms 4.0 Self Service User Interface

A recent post here http://cloudformsnow.com/2015/11/08/red-hat-cloudforms-4-0-public-beta-2/ detailed how in CloudForms 4.0 due in Dec15 will introduce a new Self Service UI. This article is a brief first glimpse at the UI.

The UI has been developed from the ground up in the community, and will over the forth coming releases become a rich user portal covering consumer based use cases such as;

  • Service Dashboards
  • Service Request
  • Service Reconfiguration
  • Service Retirement

Future capabilities will be described in CloudForms 4.1 and beyond roadmaps.

The classic service catalog will remain in the Admin and Operations User Interface until parity between the classic and the new Self Service User Interface is confirmed.

Login Screen

Screen Shot 2015-11-08 at 21.03.19

Service Catalog

Rich tile based service catalog,

Screen Shot 2015-11-08 at 20.45.27

supports filtering of the catalog by Catalog Name (shown), Service Name & Service Description

Screen Shot 2015-11-08 at 21.15.49

Catalog Item

Ordering a Service Item, presents the Service Dialog, with various static controls in CloudForms 4.0.

Screen Shot 2015-11-08 at 21.41.36

If the Service Item has associated to it a Service Dialog that contains dynamic controls, in CloudForms 4.0 initially these will NOT be supported, but instead you will be linked back to the classic UI to order the item.

My Requests – List

Supports filtering by Description, Request ID and Request Status (approved, denied & pending)

Screen Shot 2015-11-08 at 20.44.52

My Requests – Item

Like all the screens in the Self Service UI, this one shows the live status to the request.

Screen Shot 2015-11-08 at 20.45.12

My Services – List

Screen Shot 2015-11-08 at 20.45.41

My Services – Item

Selecting the Service Item will return the Service Properties page, detailing the aggregate information for the service Virtual Machines, the service parameters from the service dialog and the individual virtual machines in the service. In this page you can retire the service now or at a later date, reconfigure the service or remove it.

Screen Shot 2015-11-08 at 20.45.56

Dashboard

Screen Shot 2015-11-08 at 20.55.56

Red Hat CloudForms 4.0 Public Beta 2

Thought I would write a blog about CloudForms 4.0, which I am the Product Manager for. I am mega proud and thankful of the engineering effort in the community and at Red Hat. Here are some insights into whats in the drop, whats coming for General Release in early December.
If you have been keeping up-to-date with the ManageIQ sprints on youtube (https://www.youtube.com/user/ManageIQVideo) you would have seen some mega additions to the platform, also the community has a changelog for their Capablanca release that serves as the base for Red Hat CloudForms 4.0 here https://github.com/ManageIQ/manageiq/blob/master/CHANGELOG.md

Here is a summary on the main items.

Availability

You can now run CloudForms on Microsoft HyperV natively!

  • Microsoft HyperV/SCVMM VHD
  • VMware vSphere OVA
  • Red Hat Virtualization OVF
  • OpenStack QCOW2
  • Amazon AMI (CCP Program)

Providers

  • Google Kubernetes (Containers) – Inventory, Cross-Linking, Metrics (Dec), Events, NEW Dashboards, NEW Topology Viewer & SmartState Technology for Docker Images.
  • OpenShift Enterprise (PaaS) – Inventory, Cross-Linking, Metrics (Dec), Events, NEW Dashboards, NEW Topology Viewer & SmartState Technology for Docker Images.
    • Inventory – Projects, Services, Routes, Replicators, Pods, Containers, Container Images, Registries and Nodes.
    • Cross-Linking – Enables you to link through from the Node to the underlying Host. Example – OpenShift Node maybe installed on a Virtual Machine, Cross-Linking allows when both the OpenShift/Kubernetes Provider AND the Virtual Machine provider are present in CloudForms to expose the relationship between PaaS and IaaS, very cool and unique to CloudForms! Supported for VMware, RHEV and OpenStack.
    • Metrics – Get CPU, Memory & Network realtime metics for Nodes, Pods and Containers.
    • Events – Pods Starting, Stopping, Out Of Disk Space etc..Can be used to drive automation and policy state management.
    • NEW Dashboards – Brand new Dashboards being prototyped for Containers with rich graphics, heat maps, donut charts with metric and inventory detail.
    • NEW Topology Viewer – Bran new Topology Viewer being prototyped for Containers. Mega cool floaty topology map of OpenShift or Kubernetes Clusters, displaying the Cross-Linking right through to the running containers in the pods.
    • SmartState – A core CloudForms competency found in Virtual and Cloud platforms is also available on both Kubernetes and OpenShift. Docker images backing running containers can be SmartState Analyzed for package detail including name, version.
  • Microsoft Azure (Cloud) – Inventory, Power and Orchestration Stack Provisioning
    • Inventory – Regions, Zones, Instance/Virtual Machines
    • Power Operations – Control Virtual Machines power states, stop/start/terminate.
    • Orchestration Stack Provisioning – Deploy multi tiered applications in Microsoft Azure Cloud using stack provisioning. Simply give CloudForms the stack template, either from a blog, market place or self author and CloudForms will automatically create for you the Dialog and Service for the Service Catalog.
  •  OpenStack (Cloud) – Lots of behind the scenes things as wells more power operations
    • Neutron Inventory – Models and refreshers for future networking providers.
    • Power Operations – Added Pause and Stop.
    • Keystone V3 – Support for KS3 authentication.
  • VMware vSphere (Virtual) – Support for VMware vSphere 6
  • Red Hat Virtualization (Virtual) – Support for Red Hat Virtualization 3.5
  • Amazon EC2 – Support for new instance types.
  • Foreman/Satellite 6 – Additional parameters exposed to reporting.

User Interface

  • Self-Service UI – A brand new UI for CloudForms. This is the 1.0 release of the Self Service UI, featuring a modern look and feel that has big potential. For this release its aim is parity with the existing Operation UI Service Catalog.
    • One thing to note on this new UI its main contributor is the community member Booze Allen Hamilton.
    • Another thing more techie is that the UI is 100% RESTapi driven, so if you wish to automate anything you see, simply grab the RESTapi call for it and off you go.
    • Lastly, we have a great roadmap for this in forth coming releases, stay tuned!
  • Operations UI – As mentioned above for the Container providers OpenShift and Kubernetes we get a Topology Viewer and new Dashboards.
    • Other changes include dynamic fields, angular conversions.
    • Simplified Chinese Language Support (Dec)
    • Updates to charts to allow sum of values.
    • New donut charts.
    • Conversion of UI ongoing to BootStrap/Patternfly.

Tenancy

Huge effort behind the introduction of Tenancy to the core CloudForms platform. Tenancy in CloudForms 4.0 allows for resources like VMs, Hosts, Nodes, Pod, Containers etc..anything can be partitioned by tenant. New tenants can be created, sub tenancy is supported too!

Here are some of the areas that have been worked in to support the new Tenancy design;

  • Quotas – An extensible quota management model for tenants supporting CPU, Memory Storage and Counts per tenant.
  • Automate Domains, Service Catalogs, Catalog Items – Partitioned by tenant.
  • UI – Many enhancements to support Tenancy, such as Groups, Users, New roles for RBAC.

REST API

  • Ability to import reports – As its says, upload reports from RESTapi
  • Roles CRUD – Create, Update and Delete Roles
  • Chargeback Rates CRUD – Create, Update and Delete Charge Back Rate Tables
  • Reports run action – Run Reports from a RESTapi Call.
  • Report results collection – Return the results of a Reports in JSON over RESTapi.

Appliance

The core operating system platform to the appliance has gone through its biggest changes since before open sourcing. As CloudForms is a Virtual Appliance it contains everything you need to start with no additional servers or appliances being required for initial use (poke the competition with their minimum 3 server footprints).

  • Red Hat Enterprise Linux 7.1
  • PostgreSQL 9.4.1
  • Apache 2.4
  • jQuery 1.9.1
  • Rails 4
  • Ruby 2
  • Changed file system from ext4 to xfs
  • Application restructuring to match Rails application framework

Security

We have also done and continue to do lots of work around security. For those who know where I was before this venture, you can appreciate I know how important this is. We want ManageIQ & CloudForms to be globally adopted as the defacto standard in Cloud Management Platforms. To reach that goal we need to ensure that all users can run our platform in production. Areas of focus have been

  • STIG – Security Template Implementation Guide
  • SCAP – Security Content Automation Protocol (Dec)

Finally…..

We removed something!

SOAP Server – We removed the SOAP Server as it was incompatible with some of the platform changes we were making and we have the RESTapi to take its place. See here an earlier post on RESTapi to get yourself started quickly… http://cloudformsnow.com/2014/12/03/restapi-a-teach-yourself-lab/

Security

Red Hat Summit CloudForms Roadmap

Here is the 2nd session I performed at Red Hat summit, the CloudForms Roadmap.

Now, half of the deck contains the current 3.2 functionality, and is clearly labelled NOW, reason for this was because the amount of features we delivered in 3.2 and it GA’d only last week we felt it prudent to review that. After that section is the FUTURES, which has all the exciting stuff we are working on.

Here is a PDF copy… CloudForms Roadmap

Slide01 Slide02 Slide03 Slide04 Slide05 Slide06 Slide07 Slide08 Slide09 Slide10 Slide11 Slide12 Slide13 Slide14 Slide15 Slide16 Slide17 Slide18 Slide19 Slide20 Slide21 Slide22 Slide23 Slide24 Slide25 Slide26 Slide27 Slide28 Slide29

Red Hat Summit OpenStack Management with CloudForms

Here are my slides from Red Hat Summit, its a repeat of the OpenStack Summit presentation with two important things!

  1. The videos are now all up individually on youtube.
  2. It now includes the Heat/Cloudformations orchestration demonstration.

Here are the slides, the videos can be found in links after each relevant slide. Enjoy!

Here is a PDF copy too…OpenStack Management

Slide01

Slide02

Slide03

Slide04

Slide05

Slide06

Slide07

Slide08 Slide09 Slide10 Slide11

OpenStack – Power, Provision & Console
https://youtu.be/ByeCXMM-5z4

Slide12 Slide13 Slide14 Slide15

OpenStack – Capacity and Utilization
https://youtu.be/Qe-sDxENXF8

Slide16 Slide17 Slide18

OpenStack – SmartState and Genealogy
https://youtu.be/ysMDeqSddQ4

Slide19 Slide20

Slide21

OpenStack – Manual Scale
https://youtu.be/vzO90uuRjO8

Slide22

OpenStack – OverCloud AUTO Scale
https://youtu.be/bcJo7Bj7ho4

Slide23 Slide24 Slide25 Slide26

OpenStack – Heat/Cloudformations Templates
https://youtu.be/qiKrrGi51HU

Slide27 Slide29 Slide28 Slide30 Slide31

OpenStack Summit 2015

Today, Wednesday 20th May 2015 I presented the following deck to OpenStack Summit @ Vancouver. Its all ManageIQ branded, e.g. the upstream to Red Hat CloudForms. The videos have NO audio as I spoke over, if I get the audio sometime I will update! UPDATED…… you can watch the OpenStack show video of the presentation – HERE

The deck is below, but here is a link to it in Google Drive,,,Download Here

Slide01

Slide02

Slide03

Slide04

Slide05

Slide06

Slide07

Slide08

Slide09

Slide10

Slide11

Slide12

Slide13

Slide14

Slide15

Slide16

Slide17

Slide18

Slide19

Slide20

Slide21

Slide22

Slide23

Slide24

Slide25

Slide26

RESTapi – A teach yourself lab!

Here is a lab I put together for a recent event, it was well received and I think it is very valuable to the ManageIQ and CloudForms user bases.

In the lab you will GET and POST RESTapi actions. Its a basic look at some common use cases such as,

  • Query a VM’s hardware inventory.
  • Query the VMS for any who are tagged with “Function/Database”
  • Show the tags on a specific VM
  • Create new Category and Tag in the appliance
  • Assign a Category/Tag to a VM
  • Provision a VM
  • Query the status of the Provision request.

The lab expects you to have an appliance provisioned and connected to a provider (the lab uses AMAZON EC2) Therefore when querying attributes of a vm/instance the return list is specific to the provider. CPU’s are not available on AWS EC2, whereas availability zones are.

Note : Cut and Pasting the code from the slides can introduce ASCII translation errors depending on platform. Take care to ensure quotes are “.

I will do another post in the future to cover off “PUT, PATCH and DELETE” actions.

The slides are here in PDF format – https://www.dropbox.com/s/14vwhd1fcb97prg/RESTapi.pdf?dl=0

 

Slide02 Slide03 Slide04 Slide05 Slide06 Slide07 Slide08 Slide09 Slide10 Slide11 Slide12 Slide13 Slide14 Slide15 Slide16 Slide17 Slide18 Slide19 Slide20 Slide21 Slide22 Slide23 Slide24