SOAP – SAVON v2 Syntax

Soap-On-A-Rope-560x328

 

So for the past few years Savon v1.1.0 has been the default GEM in the appliance. Heres a scoop! In future releases, and the upstream builds have Savon v2.

What does this mean? Well v2 has a slightly different syntax to its connections and function call. Here is a v2 SOAP example;

require 'savon'

client = Savon.client do | globals |
globals.wsdl "https://x.x.x.x/vmdbws/wsdl"
globals.basic_auth ["admin", "smartvm"]
globals.ssl_verify_mode :none
globals.ssl_version :TLSv1
end

body_hash = {}
body_hash['version'] = '1.1'
body_hash['uri_parts'] = "namespace=Sample|class=Methods|instance=InspectMe|message=create"
body_hash['parameters'] = ""
body_hash['requester'] = "auto_approve=true"

response = client.call(:create_automation_request, message: body_hash)

Notice that the client call is different, and also the document is now just wsdl.

Another option for you if you wish to work with older v1.1.0 syntax scripts, then simply add to the top of your scripts;

require 'rubygems'
gem 'savon', '= 1.1.0'
require 'savon'

This will allow you to use your script as is, but forcing it to use the v1.1.0 Savon GEM. I have documented this route in the thinking that you are running your script REMOTE to the appliance. This is not a proposed route for modifying existing scripts running in an appliance.

Really the advice I give here is that;

  • If you are running your scripts REMOTE to the appliance then either upgrade your syntax to v2 or force the scripts to use v1 Savon gem.
  • If your script is running LOCAL to the appliance, change your syntax to v2.
  • If you are writing a NEW script, use the v2 syntax (and specify the version in your script like a happy scripter would do!)

Hope this helps.

Savon web site is http://savonrb.com/version2/

CLOUDFORMS 5.2.3 GEM List

Here is the list of GEMs and their info that are included in the CLOUDFORMS 5.2.3 appliance. I thought it would be useful to post, I need this recently for some work I am doing. It was really easy as most things in CLOUDFORMS usually are I simply wrote some ruby to utilise an existing gem called GEMS, that pulls this data from rubygems.org. I wrapped the code in some file open and close, e.g. I dumped the list of gems in the appliance using “gem list > gems.txt” then had my little nugget of code read that in. Here it is.

require 'rubygems'
require 'gems'

outFile = File.open("/Users/johnhardy/Desktop/Gems-CF-5.2.3.html","w")
outFile.write "<html>"
outFile.write "<body>"
outFile.write "<table>"

gemList = File.open("/Users/johnhardy/Desktop/Gems.txt","r") do |infile|
  while (line = infile.gets)
    gemItem = line.split
    gemInfo =  Gems.info "#{gemItem[0]}"
    puts gemInfo['name'] + "," + gemInfo['info']
    outFile.write "<tr>"
    outFile.write "<td>"
    outFile.write gemInfo['name']
    outFile.write "</td>"
    outFile.write "<td>"
    outFile.write gemInfo['info']
    outFile.write "</td>"
    outFile.write "</tr>"
    end
end

outFile.write "</table>"
outFile.write "</body>"
outFile.write "</html>"
outFile.close

 

actionmailer Email on Rails. Compose, deliver, receive, and test emails using the familiar controller/view pattern. First-class support for multipart email and attachments.
actionpack Web apps on Rails. Simple, battle-tested conventions for building and testing MVC web applications. Works with any Rack-compatible server.
actionwebservice Adds WSDL/SOAP and XML-RPC web service support to Action Pack
activemodel A toolkit for building modeling frameworks like Active Record. Rich support for attributes, callbacks, validations, serialization, internationalization, and testing.
activerecord Databases on Rails. Build a persistent domain model by mapping database tables to Ruby classes. Strong conventions for associations, validations, aggregations, migrations, and testing come baked-in.
activeresource REST on Rails. Wrap your RESTful web app with Ruby classes and work with them like Active Record models.
activesupport A toolkit of support libraries and Ruby core extensions extracted from the Rails framework. Rich support for multibyte strings, internationalization, time zones, and testing.
acts_as_list This “acts_as” extension provides the capabilities for sorting and reordering a number of objects in a list. The class that has this specified needs to have a “position” column defined as an integer on the mapped database table.
acts_as_tree A gem that adds simple support for organizing ActiveRecord models into parent–children relationships.
addressable Addressable is a replacement for the URI implementation that is part of
Ruby’s standard library. It more closely conforms to the relevant RFCs and
adds support for IRIs and URI templates.
akami Building Web Service Security
american_date American style month/day/year date parsing for ruby 1.9
amq-protocol amq-protocol is an AMQP 0.9.1 serialization library for Ruby. It is not an
AMQP client: amq-protocol only handles serialization and deserialization.
If you want to write your own AMQP client, this gem can help you with that.
ancestry Ancestry allows the records of a ActiveRecord model to be organized in a tree
structure, using a single, intuitively formatted database column. It exposes
all the standard tree structure relations (ancestors, parent, root, children,
siblings, descendants) and all of them can be fetched in a single sql query.
Additional features are named_scopes, integrity checking, integrity restoration,
arrangement of (sub)tree into hashes and different strategies for dealing with
orphaned records.
arel Arel is a SQL AST manager for Ruby. It

1. Simplifies the generation of complex SQL queries
2. Adapts to various RDBMSes

It is intended to be a framework framework; that is, you can build your own ORM
with it, focusing on innovative object and collection modeling as opposed to
database compatibility and query generation.

arrayfields string/symbol keyword access to arrays
awesome_print Great Ruby dubugging companion: pretty print Ruby objects to visualize their structure. Supports custom object formatting via plugins
aws-sdk AWS SDK for Ruby
bcrypt-ruby bcrypt() is a sophisticated and secure hash algorithm designed by The OpenBSD project
for hashing passwords. The bcrypt Ruby gem provides a simple wrapper for safely handling
passwords.
bigdecimal This library provides arbitrary-precision decimal floating-point number class.
binary_struct BinaryStruct is a class for dealing with binary structured data. It simplifies
expressing what the binary structure looks like, with the ability to name the
parts. Given this definition, it is easy to encode/decode the binary structure
from/to a Hash.
builder Builder provides a number of builder objects that make creating structured data
simple to do. Currently the following builder objects are supported:

* XML Markup
* XML Events

bundler Bundler manages an application’s dependencies through its entire life, across many machines, systematically and repeatably
bunny Easy to use, feature complete Ruby client for RabbitMQ 2.0 and later versions.
capybara Capybara is an integration testing tool for rack based web applications. It simulates how a user would interact with a website
childprocess This gem aims at being a simple and reliable solution for controlling external programs running in the background on any Ruby / OS combination.
color Color is a Ruby library to provide basic RGB, CMYK, HSL, and other colourspace
manipulation support to applications that require it. It also provides 152
named RGB colours (184 with spelling variations) that are commonly supported in
HTML, SVG, and X11 applications. A technique for generating monochromatic
contrasting palettes is also included.

The Color library performs purely mathematical manipulation of the colours
based on colour theory without reference to colour profiles (such as sRGB or
Adobe RGB). For most purposes, when working with RGB and HSL colour spaces,
this won’t matter. Absolute colour spaces (like CIE L*a*b* and XYZ) and cannot
be reliably converted to relative colour spaces (like RGB) without colour
profiles.

Color version 1.6 primarily adds a colour matching method for RGB and
experimental CIE L*a*b* and XYZ conversion methods for use with the colour
matching method.

Barring bugs introduced in this release, this is the last version of color that
supports Ruby 1.8, so make sure that your gem specification is set properly (to
~> 1.6) if that matters for your application.

colored >> puts “this is red”.red

>> puts “this is red with a blue background (read: ugly)”.red_on_blue

>> puts “this is red with an underline”.red.underline

>> puts “this is really bold and really blue”.bold.blue

>> logger.debug “hey this is broken!”.red_on_yellow # in rails

>> puts Color.red “This is red” # but this part is mostly untested

daemons Daemons provides an easy way to wrap existing ruby scripts (for example a self-written server) to be run as a daemon and to be controlled by simple start/stop/restart commands. You can also call blocks as daemons and control them from the parent or just daemonize the current process. Besides this basic functionality, daemons offers many advanced features like exception backtracing and logging (in case your ruby script crashes) and monitoring and automatic restarting of your processes if they crash.
dalli High performance memcached client for Ruby
default_value_for The default_value_for plugin allows one to define default values for ActiveRecord models in a declarative manner
diff-lcs Diff::LCS computes the difference between two Enumerable sequences using the
McIlroy-Hunt longest common subsequence (LCS) algorithm. It includes utilities
to create a simple HTML diff output format and a standard diff-like tool.

This is release 1.2.4, fixing a bug introduced after diff-lcs 1.1.3 that did
not properly prune common sequences at the beginning of a comparison set.
Thanks to Paul Kunysch for fixing this issue.

Coincident with the release of diff-lcs 1.2.3, we reported an issue with
Rubinius in 1.9 mode
({rubinius/rubinius#2268}[https://github.com/rubinius/rubinius/issues/2268]).
We are happy to report that this issue has been resolved.

elif A port of File::ReadBackwards, the Perl module by Uri Guttman, for reading a file in reverse, line by line. This can often be helpful for things like log files, where the interesting information is usually at the end.
erubis Erubis is an implementation of eRuby and has the following features:

* Very fast, almost three times faster than ERB and about 10% faster than eruby.
* Multi-language support (Ruby/PHP/C/Java/Scheme/Perl/Javascript)
* Auto escaping support
* Auto trimming spaces around ”
* Embedded pattern changeable (default ”)
* Enable to handle Processing Instructions (PI) as embedded pattern (ex. ”)
* Context object available and easy to combine eRuby template with YAML datafile
* Print statement available
* Easy to extend and customize in subclass
* Ruby on Rails support

eventmachine EventMachine implements a fast, single-threaded engine for arbitrary network
communications. It’s extremely easy to use in Ruby. EventMachine wraps all
interactions with IP sockets, allowing programs to concentrate on the
implementation of network protocols. It can be used to create both network
servers and clients. To create a server or client, a Ruby program only needs
to specify the IP address and port, and provide a Module that implements the
communications protocol. Implementations of several standard network protocols
are provided with the package, primarily to serve as examples. The real goal
of EventMachine is to enable programs to easily interface with other programs
using TCP/IP, especially if custom protocols are required.
excon EXtended http(s) CONnections
ezcrypto Makes it easier and safer to write crypto code.
facade The facade library allows you to mixin singleton methods from classes
or modules as instance methods of the extending class.
factory_girl factory_girl provides a framework and DSL for defining and
using factories – less error-prone, more explicit, and
all-around easier to work with than fixtures.
fattr a “fatter attr” for ruby
ffi Ruby FFI library
fog The Ruby cloud services library. Supports all major cloud providers including AWS, Rackspace, Linode, Blue Box, StormOnDemand, and many others. Full support for most AWS services including EC2, S3, CloudWatch, SimpleDB, ELB, and RDS.
formatador STDOUT text formatting
gyoku Gyoku translates Ruby Hashes to XML
haml Haml (HTML Abstraction Markup Language) is a layer on top of HTML or XML that’s
designed to express the structure of documents in a non-repetitive, elegant, and
easy way by using indentation rather than closing tags and allowing Ruby to be
embedded with ease. It was originally envisioned as a plugin for Ruby on Rails,
but it can function as a stand-alone templating engine.
haml-rails Haml-rails provides Haml generators for Rails 3. It also enables Haml as the templating engine for you, so you don’t have to screw around in your own application.rb when your Gemfile already clearly indicated what templating engine you have installed. Hurrah.
handsoap Handsoap is a library for creating SOAP clients in Ruby
highline A high-level IO library that provides validation, type conversion, and more for
command-line interfaces. HighLine also includes a complete menu system that can
crank out anything from simple list selection to complete shells with just
minutes of work.
hike A Ruby library for finding files in a set of paths.
hirb Hirb provides a mini view framework for console applications and uses it to improve ripl(irb)’s default inspect output. Given an object or array of objects, hirb renders a view based on the object’s class and/or ancestry. Hirb offers reusable views in the form of helper classes. The two main helpers, Hirb::Helpers::Table and Hirb::Helpers::Tree, provide several options for generating ascii tables and trees. Using Hirb::Helpers::AutoTable, hirb has useful default views for at least ten popular database gems i.e. Rails’ ActiveRecord::Base. Other than views, hirb offers a smart pager and a console menu. The smart pager only pages when the output exceeds the current screen size. The menu is used in conjunction with tables to offer two dimensional menus.
hoe Hoe is a rake/rubygems helper for project Rakefiles. It helps you
manage, maintain, and release your project and includes a dynamic
plug-in system allowing for easy extensibility. Hoe ships with
plug-ins for all your usual project tasks including rdoc generation,
testing, packaging, deployment, and announcement..

See class rdoc for help. Hint: `ri Hoe` or any of the plugins listed
below.

For extra goodness, see: http://docs.seattlerb.org/hoe/Hoe.pdf

httparty Makes http fun! Also, makes consuming restful web services dead easy.
httpclient gives something like the functionality of libwww-perl (LWP) in Ruby
httpi Common interface for Ruby’s HTTP libraries
i18n New wave Internationalization support for Ruby.
inifile Although made popular by Windows, INI files can be used on any system thanks
to their flexibility. They allow a program to store configuration data, which
can then be easily parsed and changed. Two notable systems that use the INI
format are Samba and Trac.

More information about INI files can be found on the [Wikipedia Page](http://en.wikipedia.org/wiki/INI_file).

### Properties

The basic element contained in an INI file is the property. Every property has
a name and a value, delimited by an equals sign *=*. The name appears to the
left of the equals sign and the value to the right.

name=value

### Sections

Section declarations start with *[* and end with *]* as in `[section1]` and
`[section2]` shown in the example below. The section declaration marks the
beginning of a section. All properties after the section declaration will be
associated with that section.

### Comments

All lines beginning with a semicolon *;* or a number sign *#* are considered
to be comments. Comment lines are ignored when parsing INI files.

### Example File Format

A typical INI file might look like this:

[section1]
; some comment on section1
var1 = foo
var2 = doodle
var3 = multiline values \
are also possible

[section2]
# another comment
var1 = baz
var2 = shoodle

io-console add console capabilities to IO instances.
io-extra Adds the IO.closefrom, IO.fdwalk, IO.pread, IO.pread_ptr, IO.pwrite, and
IO.writev singleton methods as well as the IO#directio, IO#directio? and
IO#ttyname instance methods (for those platforms that support them).
journey Journey is a router. It routes requests.
json This is a JSON implementation as a Ruby extension in C.
json_pure This is a JSON implementation in pure Ruby.
linux_admin LinuxAdmin is a module to simplify management of linux systems.
It should be a single place to manage various system level configurations,
registration, updates, etc.
little-plugger LittlePlugger is a module that provides Gem based plugin management.
By extending your own class or module with LittlePlugger you can easily
manage the loading and initializing of plugins provided by other gems.
log4r See also: http://logging.apache.org/log4j
logging Logging is a flexible logging library for use in Ruby programs based on the
design of Java’s log4j library. It features a hierarchical logging system,
custom level names, multiple output destinations per log event, custom
formatting, and more.
mail A really Ruby Mail handler.
main a class factory and dsl for generating command line programs real quick
map the awesome ruby container you’ve always wanted: a string/symbol indifferent ordered hash that works in all rubies
mime-types The mime-types library provides a library and registry for information about
MIME content type definitions. It can be used to determine defined filename
extensions for MIME types, or to use filename extensions to look up the likely
MIME type definitions.

MIME content types are used in MIME-compliant communications, as in e-mail or
HTTP traffic, to indicate the type of content which is transmitted. The
mime-types library provides the ability for detailed information about MIME
entities (provided as an enumerable collection of MIME::Type objects) to be
determined and used programmatically. There are many types defined by RFCs and
vendors, so the list is long but by definition incomplete; don’t hesitate to to
add additional type definitions (see Contributing.rdoc). The primary sources
for MIME type definitions found in mime-types is the IANA collection of
registrations (see below for the link), RFCs, and W3C recommendations.

This is release 2.2, mostly changing how the MIME type registry is updated from
the IANA registry (the format of which was incompatibly changed shortly before
this release) and taking advantage of the extra data available from IANA
registry in the form of MIME::Type#xrefs. In addition, the {LTSW
list}[http://www.ltsw.se/knbase/internet/mime.htp] has been dropped as a
supported list.

As a reminder, mime-types 2.x is no longer compatible with Ruby 1.8 and
mime-types 1.x is only being maintained for security issues. No new MIME types
or features will be added.

mime-types (previously called MIME::Types for Ruby) was originally based on
MIME::Types for Perl by Mark Overmeer, copyright 2001 – 2009. It is built to
conform to the MIME types of RFCs 2045 and 2231. It tracks the {IANA Media
Types registry}[https://www.iana.org/assignments/media-types/media-types.xhtml]
with some types added by the users of mime-types.

minitest minitest provides a complete suite of testing facilities supporting
TDD, BDD, mocking, and benchmarking.

“I had a class with Jim Weirich on testing last week and we were
allowed to choose our testing frameworks. Kirk Haines and I were
paired up and we cracked open the code for a few test
frameworks…

I MUST say that minitest is *very* readable / understandable
compared to the ‘other two’ options we looked at. Nicely done and
thank you for helping us keep our mental sanity.”

— Wayne E. Seguin

minitest/unit is a small and incredibly fast unit testing framework.
It provides a rich set of assertions to make your tests clean and
readable.

minitest/spec is a functionally complete spec engine. It hooks onto
minitest/unit and seamlessly bridges test assertions over to spec
expectations.

minitest/benchmark is an awesome way to assert the performance of your
algorithms in a repeatable manner. Now you can assert that your newb
co-worker doesn’t replace your linear algorithm with an exponential
one!

minitest/mock by Steven Baker, is a beautifully tiny mock (and stub)
object framework.

minitest/pride shows pride in testing and adds coloring to your test
output. I guess it is an example of how to write IO pipes too. :P

minitest/unit is meant to have a clean implementation for language
implementors that need a minimal set of methods to bootstrap a working
test suite. For example, there is no magic involved for test-case
discovery.

“Again, I can’t praise enough the idea of a testing/specing
framework that I can actually read in full in one sitting!”

— Piotr Szotkowski

Comparing to rspec:

rspec is a testing DSL. minitest is ruby.

— Adam Hawkins, “Bow Before MiniTest”

minitest doesn’t reinvent anything that ruby already provides, like:
classes, modules, inheritance, methods. This means you only have to
learn ruby to use minitest and all of your regular OO practices like
extract-method refactorings still apply.

more_core_extensions MoreCoreExtensions are a set of core extensions beyond those provided by ActiveSupport.
multi_json A common interface to multiple JSON libraries, including Oj, Yajl, the JSON gem (with C-extensions), the pure-Ruby JSON gem, NSJSONSerialization, gson.rb, JrJackson, and OkJson.
multi_xml Provides swappable XML backends utilizing LibXML, Nokogiri, Ox, or REXML.
net-http-persistent Manages persistent connections using Net::HTTP plus a speed fix for Ruby 1.8.
It’s thread-safe too!

Using persistent HTTP connections can dramatically increase the speed of HTTP.
Creating a new HTTP connection for every request involves an extra TCP
round-trip and causes TCP congestion avoidance negotiation to start over.

Net::HTTP supports persistent connections with some API methods but does not
handle reconnection gracefully. Net::HTTP::Persistent supports reconnection
and retry according to RFC 2616.

net-ldap Net::LDAP for Ruby (also called net-ldap) implements client access for the
Lightweight Directory Access Protocol (LDAP), an IETF standard protocol for
accessing distributed directory services. Net::LDAP is written completely in
Ruby with no external dependencies. It supports most LDAP client features and a
subset of server features as well.

Net::LDAP has been tested against modern popular LDAP servers including
OpenLDAP and Active Directory. The current release is mostly compliant with
earlier versions of the IETF LDAP RFCs (2251–2256, 2829–2830, 3377, and 3771).
Our roadmap for Net::LDAP 1.0 is to gain full client compliance with
the most recent LDAP RFCs (4510–4519, plus portions of 4520–4532).

net-ping The net-ping library provides a ping interface for Ruby. It includes
separate TCP, HTTP, LDAP, ICMP, UDP, WMI (for Windows) and external ping
classes.
net-scp A pure Ruby implementation of the SCP client protocol
net-sftp A pure Ruby implementation of the SFTP client protocol
net-ssh Net::SSH: a pure-Ruby implementation of the SSH2 client protocol. It allows you to write programs that invoke and interact with processes on remote servers, via SSH2.
netrc This library can read and update netrc files, preserving formatting including comments and whitespace.
nokogiri Nokogiri (鋸) is an HTML, XML, SAX, and Reader parser. Among Nokogiri’s
many features is the ability to search documents via XPath or CSS3 selectors.

XML is like violence – if it doesn’t solve your problems, you are not using
enough of it.

nori XML to Hash translator
open4 open child process with handles on pid, stdin, stdout, and stderr: manage child processes and their io handles easily.
ovirt_metrics OvirtMetrics is an ActiveRecord-based gem for reading the oVirt History database.
parallel Run any kind of code in parallel processes
pdf-writer This library provides the ability to create PDF documents using only native Ruby libraries. There are several demo programs available in the demo/ directory. The canonical documentation for PDF::Writer is "manual.pdf", which can be generated using bin/techbook (just "techbook" for RubyGem users) and the manual file "manual.pwd".
pg Pg is the Ruby interface to the {PostgreSQL RDBMS}[http://www.postgresql.org/].

It works with {PostgreSQL 8.4 and later}[http://www.postgresql.org/support/versioning/].

A small example usage:

#!/usr/bin/env ruby

require ‘pg’

# Output a table of current connections to the DB
conn = PG.connect( dbname: ‘sales’ )
conn.exec( “SELECT * FROM pg_stat_activity” ) do |result|
puts ” PID | User | Query”
result.each do |row|
puts ” %7d | %-16s | %s ” %
row.values_at(‘procpid’, ‘usename’, ‘current_query’)
end
end

Platform Hopefully robust platform sensing
polyglot The Polyglot library allows a Ruby module to register a loader
for the file type associated with a filename extension, and it
augments ‘require’ to find and load matching files.
princely A wrapper for the PrinceXML PDF generation library.
prototype-rails Prototype, Scriptaculous, and RJS for Ruby on Rails
qpid_messaging Qpid is an enterprise messaging framework.
rack Rack provides a minimal, modular and adaptable interface for developing
web applications in Ruby. By wrapping HTTP requests and responses in
the simplest way possible, it unifies and distills the API for web
servers, web frameworks, and software in between (the so-called
middleware) into a single method call.

Also see http://rack.github.com/.

rack-cache Rack::Cache is suitable as a quick drop-in component to enable HTTP caching for Rack-based applications that produce freshness (Expires, Cache-Control) and/or validation (Last-Modified, ETag) information.
rack-ssl Rack middleware to force SSL/TLS.
rack-test Rack::Test is a small, simple testing API for Rack apps. It can be used on its
own or as a reusable starting point for Web frameworks and testing libraries
to build on. Most of its initial functionality is an extraction of Merb 1.0’s
request helpers feature.
rails Ruby on Rails is a full-stack web framework optimized for programmer happiness and sustainable productivity. It encourages beautiful code by favoring convention over configuration.
railties Rails internals: application bootup, plugins, generators, and rake tasks.
rake Rake is a Make-like program implemented in Ruby. Tasks and dependencies are
specified in standard Ruby syntax.

Rake has the following features:

* Rakefiles (rake’s version of Makefiles) are completely defined in
standard Ruby syntax. No XML files to edit. No quirky Makefile
syntax to worry about (is that a tab or a space?)

* Users can specify tasks with prerequisites.

* Rake supports rule patterns to synthesize implicit tasks.

* Flexible FileLists that act like arrays but know about manipulating
file names and paths.

* A library of prepackaged tasks to make building rakefiles easier. For example,
tasks for building tarballs and publishing to FTP or SSH sites. (Formerly
tasks for building RDoc and Gems were included in rake but they’re now
available in RDoc and RubyGems respectively.)

* Supports parallel execution of tasks.

rdoc RDoc produces HTML and command-line documentation for Ruby projects. RDoc
includes the +rdoc+ and +ri+ tools for generating and displaying documentation
from the command-line.
rest-client A simple HTTP and REST client for Ruby, inspired by the Sinatra microframework style of specifying actions: get, put, post, delete.
rspec BDD for Ruby
rspec-core BDD for Ruby. RSpec runner and example groups.
rspec-expectations rspec expectations (should[_not] and matchers)
rspec-mocks RSpec’s ‘test double’ framework, with support for stubbing and mocking
rspec-rails RSpec for Rails
ruby-graphviz Ruby/Graphviz provides an interface to layout and generate images of directed graphs in a variety of formats (PostScript, PNG, etc.) using GraphViz.
ruby-hmac This module provides common interface to HMAC functionality. HMAC is a kind of “Message Authentication Code” (MAC) algorithm whose standard is documented in RFC2104. Namely, a MAC provides a way to check the integrity of information transmitted over or stored in an unreliable medium, based on a secret key.

Originally written by Daiki Ueno. Converted to a RubyGem by Geoffrey Grosenbach

ruby-plsql ruby-plsql gem provides simple Ruby API for calling Oracle PL/SQL procedures.
It could be used both for accessing Oracle PL/SQL API procedures in legacy applications
as well as it could be used to create PL/SQL unit tests using Ruby testing libraries.
ruby-progressbar Ruby/ProgressBar is an extremely flexible text progress bar library for Ruby.
The output can be customized with a flexible formatting system including:
percentage, bars of various formats, elapsed time and estimated time remaining.
rubyforge A script which automates a limited set of rubyforge operations.

* Run ‘rubyforge help’ for complete usage.
* Setup: For first time users AND upgrades to 0.4.0:
* rubyforge setup (deletes your username and password, so run sparingly!)
* edit ~/.rubyforge/user-config.yml
* rubyforge config
* For all rubyforge upgrades, run ‘rubyforge config’ to ensure you have latest.

rubyrep This rubygem does not have a description or summary.
rubywbem This is a short description
rubyzip rubyzip is a ruby module for reading and writing zip files
rufus-lru LruHash class, a Hash with a max size, controlled by a LRU mechanism
rufus-scheduler job scheduler for Ruby (at, cron, in and every jobs).
ruport Ruby Reports is a software library that aims to make the task of reporting
less tedious and painful. It provides tools for data acquisition,
database interaction, formatting, and parsing/munging.
savon Heavy metal SOAP client
selenium-webdriver WebDriver is a tool for writing automated tests of websites. It aims to mimic the behaviour of a real user, and as such interacts with the HTML of the application.
shoulda-matchers Making tests easy on the fingers and eyes
simple-rss A simple, flexible, extensible, and liberal RSS and Atom reader for Ruby. It is designed to be backwards compatible with the standard RSS parser, but will never do RSS generation.
snmp A Ruby implementation of SNMP (the Simple Network Management Protocol).
soap4r An implementation of SOAP 1.1 for Ruby.
sprockets Sprockets is a Rack-based asset packaging system that concatenates and serves JavaScript, CoffeeScript, CSS, LESS, Sass, and SCSS.
state_machine Adds support for creating state machines for attributes on any Ruby class
syntax Syntax is Ruby library for performing simple syntax highlighting.
thin A thin and fast web server
thor Thor is a toolkit for building powerful command-line interfaces.
tilt Generic interface to multiple Ruby template engines
timecop A gem providing “time travel” and “time freezing” capabilities, making it dead simple to test time-dependent code. It provides a unified method to mock Time.now, Date.today, and DateTime.now in a single call.
transaction-simple Transaction::Simple provides a generic way to add active transaction support to
objects. The transaction methods added by this module will work with most
objects, excluding those that cannot be Marshal-ed (bindings, procedure
objects, IO instances, or singleton objects).

The transactions supported by Transaction::Simple are not associated with any
sort of data store. They are “live” transactions occurring in memory on the
object itself. This is to allow “test” changes to be made to an object before
making the changes permanent.

Transaction::Simple can handle an “infinite” number of transaction levels
(limited only by memory). If I open two transactions, commit the second, but
abort the first, the object will revert to the original version.

Transaction::Simple supports “named” transactions, so that multiple levels of
transactions can be committed, aborted, or rewound by referring to the
appropriate name of the transaction. Names may be any object except nil.

Transaction groups are also supported. A transaction group is an object wrapper
that manages a group of objects as if they were a single object for the purpose
of transaction management. All transactions for this group of objects should be
performed against the transaction group object, not against individual objects
in the group.

Version 1.4.0 of Transaction::Simple adds a new post-rewind hook so that
complex graph objects of the type in tests/tc_broken_graph.rb can correct
themselves.

Version 1.4.0.1 just fixes a simple bug with #transaction method handling
during the deprecation warning.

Version 1.4.0.2 is a small update for people who use Transaction::Simple in
bundler (adding lib/transaction-simple.rb) and other scenarios where having Hoe
as a runtime dependency (a bug fixed in Hoe several years ago, but not visible
in Transaction::Simple because it has not needed a re-release). All of the
files internally have also been marked as UTF-8, ensuring full Ruby 1.9
compatibility.

treetop A Ruby-based text parsing and interpretation DSL
trollop Trollop is a commandline option parser for Ruby that just
gets out of your way. One line of code per option is all you need to write.
For that, you get a nice automatically-generated help page, robust option
parsing, command subcompletion, and sensible defaults for everything you don’t
specify.
tzinfo TZInfo provides daylight savings aware transformations between times in different time zones.
uuidtools A simple universally unique ID generation library.
vcr VCR provides a simple API to record and replay your test suite’s HTTP interactions. It works with a variety of HTTP client libraries, HTTP stubbing libraries and testing frameworks.
wasabi A simple WSDL parser
websocket Universal Ruby library to handle WebSocket protocol
xml-simple A simple API for XML processing.
xpath XPath is a Ruby DSL for generating XPath expressions
ziya ZiYa allows you to easily create interactive charts, gauges and maps for your web applications. ZiYa leverages
flash which offload heavy server side processing to the client. At the root ZiYa allows you to easily generate an
XML files that will be downloaded to the client for rendering. Using this gem, you will be able to easily create great
looking charts for your application. You will also be able to use the charts, gauges and maps has a navigation scheme
by embedding various link in the graphical components thus bringing to the table an ideal scheme for reporting and dashboard
like applications. Your manager will love you for it !!

Sample site : http://ziya.liquidrail.com
Documentation : http://ziya.liquidrail.com/docs
Forum : http://groups.google.com/group/ziya-plugin
Repositories : git://github.com/derailed/ziya.git

Ravello with Cloudforms

ravello_systems_logo

Very excited about this one, if you have not seen Ravello, check it out http://www.ravellosystems.com, here is a summary…

“Run VMWare workload in AWS”

Whats even cooler, you can actually run hypervisors in AWS, which if you have been playing with EC2 you will know is impossible with them, without some sorcery!

Moving on….as I am not here to sell you Ravello,, though I can admit that we use them a LOT, very cheap and great API which is what this post is about.

I had a good friend ask if we could provision instances in Ravello using Cloudforms, there was no need to have visualisation of the instances after provision, just standard fire and forget from our service catalog.

So we first start with our service catalog item, here it is here;

Screen Shot 2014-04-09 at 09.52.20

 

The idea here is that we can take a blue print in Ravello and spin out as many of these as we wish. A blue print in Ravello is like a bunch of virtual machines in a design view, once you are happy with your design you publish it and those machines are available. So in our use case we have one blue print that we will want to instantiate many times. We also in the dialog collect some naming detail for the new application being instantiated from the blue print, we also give the choice of hostname prefix. This is because in our blue prints we may want to change the hostnames of the virtual machines when the application is instantiated, why? If we only instantiated one copy of the blue print then it would be fine, but if we do many then the hostnames would conflict with one another in each application instantiated from the blue print.

We also offer the drop down list box of what blue print to use as the template for this new Ravello application. Here is another graphic to show you the options of this;

Screen Shot 2014-04-09 at 09.52.01

This list of blue prints comes dynamically from Ravello.

After clicking submit we will get out applications; this is what it looks like in Ravello once Cloudforms has run the service install.

This run I entered 4 as the number of students (instantiations) .

Screen Shot 2014-04-09 at 10.18.33

Cloudforms shows in the service request queue the job being executed and processed, you could hook up approval and quota checks for this if you wanted.

Screen Shot 2014-04-09 at 10.04.48

Screen Shot 2014-04-09 at 10.00.43

 

So in Ravello finally we need to check did the hostname change for each instantiated application from the blueprint?

Screen Shot 2014-04-09 at 10.18.58

And if you look in the top right of the screen estate you can see the hostname field and the student number and prefix has been updated, this would be incremented for each instantiation of the blueprint for each application.

So if you want to integrate Cloudforms and Ravello, here is the method, its mega simple, a rest api thats was fairly simple to get together thought the documentation could do with more samples!

There is also a Ruby GEM for Ravello, though this can be good, we at Cloudforms tend to use RestAPI first, because then we do not have to worry about GEM management across the appliances, especially when the Ravello GEM is just an abstraction of the RestAPI anyway.

 

# Automate Method
#
$evm.log("info", "--------------------------RAVELLO APPLICATION START-----------------")
$evm.log("info", "--------------------------RAVELLO APPLICATION START-----------------")
$evm.log("info", "--------------------------RAVELLO APPLICATION START-----------------")</code>

#
# Method Code Goes here
#

require 'rubygems'
require 'rest_client'
require 'xmlsimple'
require 'json'
require 'cgi'

#how many students do you want?
$totalStudents = $evm.root['dialog_totalstudents'].to_i
@applicationNamePrefix = $evm.root['dialog_applicationnameprefix']
@applicationDescription = $evm.root['dialog_applicationdescription']
#hostname prefix, will result as hostnameX, example - hostname1, hostname2, hostname3...
@hostnamePrefix = $evm.root['dialog_hostnameprefix']
#Search String
@searchString = "NOreplaceNO"
#whats the bluePrint you want to use for the application(s)
@bluePrint = $evm.root['dialog_blueprint'].to_i
#whats your account username and password
rawUsername = $evm.root['dialog_username']
rawPassword = $evm.root['dialog_password']
@username = CGI::escape(rawUsername)
@password = CGI::escape(rawPassword)

$i = 1
while $i &lt;= $totalStudents do studentNumber = "%.3i" %$i
  $evm.log("info", "StudentNumber = #{studentNumber}")
  applicationName = "#{@applicationNamePrefix}-" + studentNumber.to_s
  applicationDescription = "#{@applicationDescription} - #{applicationName}"
  $evm.log("info", "Creating Application for #{applicationName}")
  #create the application from a known bluePrint
  rtr = RestClient.post "https://#{@username}:#{@password}@cloud.ravellosystems.com/api/v1/applications/", {:name =&gt; "#{applicationName}", :description =&gt; "#{applicationDescription}", :baseBlueprintId =&gt; @bluePrint.to_i}.to_json, :content_type =&gt; :json, :accept =&gt; 'application/json'

  #find the applicationID
  rtr_parsed = JSON.parse(rtr)
  applicationID = rtr_parsed['id']

  #fetch the newly created application
  data = RestClient.get "https://#{@username}:#{@password}@cloud.ravellosystems.com/api/v1/applications/#{applicationID}", :content_type =&gt; :json, :accept =&gt; 'application/json'
  entity = JSON.parse(data)
  new = entity

  x = 0
  new['design']['vms'].each do | a |
  $evm.log("info", "Processing #{a['hostnames']}")
  if a['hostnames'].to_s[/NOreplaceNO/].to_s == "NOreplaceNO"
    dooie = new['design']['vms'][x]['hostnames'][0]
    dooie.gsub!("NOreplaceNO",@hostnamePrefix.to_s + studentNumber.to_s)
    new['design']['vms'][x]['hostnames'][0] = dooie
    $evm.log("info", "New Hostname = #{new['design']['vms'][x]['hostnames']}")
  end
  x = x+1
end

#save the changes back to the application
yoyo = RestClient.put "https://#{@username}:#{@password}@cloud.ravellosystems.com/api/v1/applications/#{applicationID}", new.to_json, :content_type =&gt; :json, :accept =&gt; :json

#publish the application to Amazon
yoyo = RestClient.post "https://#{@username}:#{@password}@cloud.ravellosystems.com/api/v1/applications/#{applicationID}/publish", {:preferredCloud =&gt; "AMAZON", :optimizationLevel =&gt; "PERFORMANCE_OPTIMIZED" , :preferredRegion =&gt; "Virginia"}.to_json, :content_type =&gt; :json, :accept =&gt; 'application/json'

$i +=1
end

$evm.log("info", "--------------------------RAVELLO APPLICATION END-----------------")
$evm.log("info", "--------------------------RAVELLO APPLICATION END-----------------")
$evm.log("info", "--------------------------RAVELLO APPLICATION END-----------------")

#
#
#
$evm.log("info", "Automate Method Ended")
exit MIQ_OK

You can download this code https://github.com/jonnyfiveiq/CloudFORMSNOW/blob/master/Integrations/Provisioning/Ravello/ravello.rb

As a note to this, Ravello offers two cloud platforms as destination, we have ONLY ever used AWS as a destination platform, you can see this in the code as

PERFORMANCE_OPTIMIZED
AMAZON
Virginia

Thanks

AWS – Elastic Load Balancers and Cloudforms

First entry in ages, but those who know me will have seen I have been visiting a few places around the globe. Anyway, I promised some time ago the ability to add Amazon EC2 Instances to Amazon Elastic Load Balancers, here it is.

Problem was that whilst I have spent a couple of hours here and there on airplanes recently, its still quite difficult to connect to AWS from them! So any work on this had to wait until airport lounges. Its done now so here is the entry for it.

We start with the design, I wanted to deliver this as something that could be re-used and eat my own dog food when it comes to state-machines. Also because a load balancer best works with more than one workload the result of this was always going to include a n-tier service of instances. The following diagram show the high level design to the service.

Slide1

It is a service bundle that consists of two service items, each service item has some sort of service resource to it. The next diagram shows the actual service I am delivering in context to the first diagram.

Slide2

My service has a service bundle called “N-Tier AWS”, that has two service items called “Web Servers” and “Load Balancer”. The “Web Servers” delivers two AWS EC2 instances named “Web Server 1” and “Web Server 2”. There is nothing special about “Web Servers”, it’s a standard service item of AWS EC2 resources (AMI’s). The second service item is the new one I am blogging about. This is a generic item that connects to its own State Machine through a relationship connection. For this I had to edit the class of the out of the box state machine. Why? Because I wanted to nest my StateMachine with that of the out of the box one. Here is the change I made;

Edited /Factory / StateMachines / ServiceProvision_Template /

And added to the top of this class the following Field

  • Field Name – rel1
  • Field Type – Relationship

Accept all other defaults for the field.

It should look something like;

Screen Shot 2014-03-04 at 15.15.43

With the class change made, we can write into this relationship the call to the new statemachine. This means when ever you want to add the capability of AWS EC2 load balancers, all you need to do is wire the statemachine for loadbalancers to your service bundle state machine.

The state machine which is the download-able part of this, looks something like; Its simplicity is its power! If the state machine errors then we shall execute an error method called error_nlb otherwise, on entry to the state machine we shall run a method called add_to_nlb.

Screen Shot 2014-03-04 at 15.33.09

Download the state machine here -: https://github.com/jonnyfiveiq/CloudFORMSNOW/blob/master/Integrations/Provisioning/AWS_ELB/cfnow-aws-elb.xml

This will create a new namespace and class as follows;

Screen Shot 2014-03-04 at 16.09.57

You are then free to wire in my state machine to your service state machine, looking something like;

Screen Shot 2014-03-04 at 16.11.49

The service bundle is constructed with what ever instances you was instances are, in my case web server_a and web server_b , then a 3rd service item being the loadbalancer state machine, so this is a generic service item. Here is a screen shot of my bundle resources!

Screen Shot 2014-03-04 at 16.12.49

There service bundle detail is as follows, remember to create your own new state machine for the bundle and not just use default! In summary the state machine that this bundle points to does nothing at all other that what is configured in default!

Screen Shot 2014-03-04 at 16.12.41

And finally the generic item that is “AWS Load Balancer” and resides in automation location “/Factory/StateMachines/ServiceProvision_Template/aws_loadbalancers” is the state machine you will import from GitHub url on this page, here is the generic service item detail;

Screen Shot 2014-03-04 at 16.13.02

Notice no dialog, just the state machine to be called.

The Loadbalancer StateMachine

Some design considerations needed to be taken into account, for example when the instances first spin up will not immediately have IP addresses, the status checks will not be complete, as show here;

Screen Shot 2014-02-28 at 17.33.52

So we need to wait for the instances to be come available, the best way for this is to allow Cloudforms to tell the statemachine when an IP address turns up for each service resource in the service bundle. Now we can not actually have the message come to us, so we use the statemachine re-entrancy concept. Sounds complicated by easy to explain. We simply launch our loadbalncer statemachine at the same time as the AWS EC2 Instance statemachine. On initial run of the load balancer statemachine we check to the service resources for ip addresses, if blank we then put the load balancer statemachine into retry mode, with an interval of 30 seconds. The AWS EC2 instance statemachine will continue to process the service resources and insatiate the instances, 30 seconds later our loadbalancer statamachine will re-enter and run again. Repeating the same initial tests, if the instances now have IP addresses then the instances will be added to a load balancer, if the instances are still without IP addresses as with the first run, the loadbalancer statemachine will go into retry mode again.

The other features of the statamachine are to read what the user selected when ordering the service bundle. There are three parameters, shown here in the graphic;

Screen Shot 2014-02-28 at 16.55.59

The 3 elements we need are;

  • Elastic Load Balancer – Check Box
  • Elastic Load Balancer Name – Text Box
  • Existing Elastic Load Balancer – Dynamic Drop Down

The user must first select “Elastic Load Balancer – Check Box “ to enable our loadbalancer statemachine. Once this is enabled the user can choose either “Elastic Load Balancer Name – Text Box” to create a NEW elastic load balancer or select from the “Existing Elastic Load Balancer – Dynamic Drop Down” an existing Elastic Load Balancer. The dynamic drop down has behind it a method to populate the drop down with load balancers from a specified AWS account.

StateMachine Method Walk-Through

First we start with my bad habit of  “We are here” really loudly! Actually when I first wrote this method, I pumped my log entries to a completely new log file. It made it easy to debug whilst two statemachine are running synconlosy one of them delivering 2 instances or more, it can be tricky to read the automate log file real time.

As part of the starting code we have the gem statement, we are using the out of the box ruby gem for AWS-SDK, this is an awesome gem providing huge amounts of functions covering both S3 and EC2.

</pre>
require 'aws-sdk'

$evm.log("info", "------------------------------ADD TO AWS LOAD BALANCER------------------------------\n")

Next we have a section of code that deals with setting up the objects;

</pre>
stp_task = $evm.root["service_template_provision_task"]

miq_request_id = $evm.vmdb('miq_request_task', stp_task.get_option(:parent_task_id))

dialogOptions = miq_request_id.get_option(:dialog)

$evm.log("info", "ELB New ELB Name --> #{dialogOptions['dialog_elbName)']}\n")

$evm.log("info", "ELB Existing ELB --> #{dialogOptions['dialog_existingELB']}\n")

$evm.log("info", "Add to ELB Enabled --> #{dialogOptions['dialog_elb']}\n")

miqr = $evm.vmdb('service', stp_task.get_option(:parent_service_id))

Line 1, “stp_task”, is fetching the service_template_provision_task from the current statemachine. This would result in the service_template_provision_task for loadbalancers, which is not very useful to us by itself because its not the loadbalancers that have the detail about what is going to be insatiated, but the other service in the service bundle. Also the loadbalancer service did not collect the user input either that was the bundled server so Line 2, “miq_request_id” fetches from the VMDB the bundle service instance, the subsequent lines then extract from the bundle service the dialog options that were set. Lastly we fetch the live service instance for the bundle, this holds a very special helper attribute that will be populated by cloudforms as new VM’s are provisioned called “VMS”, at start its value is nil, but as instances are provisioned their object references are placed into an array in this attribute.

</pre>
i = 0

unless miqr.vms.nil?

miqr.vms.each do |vm|

i = i.to_i + 1

$evm.log("info", "------------ VM #{i} ----------\n")

$evm.log("info", "Instance Name --> #{vm.name}\n")

$evm.log("info", "Instnace ID --> #{vm.ems_ref}\n")

$evm.log("info", "DNS Name --> #{vm.location}\n")

$evm.log("info", "PowerState --> #{vm.power_state}\n")

$evm.log("info", "Active --> #{vm.active}\n")

instanceA << {:instance_id => vm.ems_ref}

$evm.log("info", "------------ VM #{i} ----------\n")

end

end

This code is fairly simple,  mainly logging so I can see the instances as they turn up, but most importantly its adding them to a new Ruby hash called instance as they turn up. Here is the segment that reports the number of instances (VMs) in the miqr objects (Service Request) vs that of the provisioned instances (isntanceCount)

</pre>
vmCount = miqr.vms.count

instanceCount = instanceA.size

$evm.log("info", "VMs Count --> #{vmCount}\n")

$evm.log("info", "instanceA Count --> #{instanceCount}\n")

Now we start the if block, to simply understand “did the user enable ELB?” and if they did, “did they want a new one or use an existing ELB”?

First if is “Have we the same number of instances as requested instances live and running?

</span>

if vmCount > 0 && vmCount = instanceCount

if dialogOptions['dialog_elb'].to_s == "1"

Now we deal with new vs old ELB,


if dialogOptions['dialog_existingELB'] != " None"
 $evm.log("info", "Using Existing ELB #{dialogOptions['dialog_existingELB']}\n")
 elbName = dialogOptions['dialog_existingELB']
 elsif dialogOptions['dialog_elbName'] != ""
 $evm.log("info", "Creating NEW ELB #{dialogOptions['dialog_elbName']}\n")
 elbName = dialogOptions['dialog_elbName']

#create the ELB

else
 $evm.log("info", "Neither New ELB or Existing ELB was set!\n")
 end

and finally the bit that actually adds the instances to the ELB.


$evm.log("info", "Using ELB #{elbName}\n")
 AWS.config(:access_key_id => '<your_user>', :secret_access_key => '<your_key>')

ec2 = AWS::EC2.new
 elb = AWS::ELB.new

miqr.vms.each do |vm|
 $evm.log("info", "Adding #{vm.ems_ref} to ELB #{elbName}\n")
 i = ec2.instances["#{vm.ems_ref}"]
 elb.load_balancers["#{elbName}"].instances.register(i)
 end
 end

I do admit I took a short cut in hardcoding the authentication details in the method, you should do better and get these from the provider!

Now here is the part of the method thats actually more important than adding the instances to the ELB! The re-entrancy part of the method.


else
 passNo = stp_task.get_option(:pass)
 passNo = passNo.to_i + 1
 stp_task.set_option(:pass, "#{passNo}")
 $evm.log("info", "Pass Number #{passNo}\n")
 $evm.root['ae_result'] = 'retry'
 $evm.root['ae_retry_interval'] = '30.seconds'
 $evm.log("info", "-------------------RETRY-----------------------\n")

exit MIQ_OK
end

Notice that in the block, we have check previously should we be here, else do this. Then we set the “ae_result” to retry and “ae_retry_interval” to every 30 seconds. This means that when we exit this method state, the state machine will put this into retry 30 seconds later, and we do the whole method again and again until we pass the check of should we be here or not, and add the instances to the ELB.

User Experience

Here is what the consumers or users would see when ordering an item in Cloudforms with ELB supports.

1. Select the item from the catalog

Screen Shot 2014-02-28 at 16.55.23

2. Complete the dialog for the catalog item, selecting the ELB configuration. (ignore the other detail to this dialog – that may form part two of this demo!)

Screen Shot 2014-02-28 at 16.55.38

3. Request enters queue.

Screen Shot 2014-02-28 at 16.58.19

Now if we have a look at AWS, (Not that we need to as we have Cloudforms!!!) The ELB’s during provisioning look empty;

Screen Shot 2014-02-28 at 16.57.11

And when Cloudforms has finished provisioning the instances, and added them to the ELB, AWS lists them as so;

Screen Shot 2014-03-04 at 15.29.30

This concludes this post, I hope its of value! Remember that the design goal here was to create a state machine that could be easily hooked to any AWS based service to provide ELB support.

inspectXML – Dump objects as XML

This is pretty simple but very useful. I have done a little research and whilst inspect is a way of seeing inside of an object its also hard to read and not very re-usable. Being somewhat old now (crazy thought) XML used to be the way we described things. Yes I know yaml, json etc have come to replace XML in languages such as Ruby, but I cannot get away from XML is far easier to read and self describing than the aforementioned.

If you have used InspectME that ships with the product then this is the same but in XML format, with a few advantages…

  • The code is self discovering, e.g. that it will traverse associations, virtual columns, attributes and options without the need to define the object type. This means as the product grows this routine requires no maintenance.
  • As stated, the traversal of the associations is cool, as the XML format allows for nesting of the various attributes, objects, etc… so you can see easily even in XML who owns what.

Download examples here <VMWARE_HOST> <KVM_HOST> <VM> <CLUSTER> <DATASTORE>

Here are the instructions for use.

Download the automate class here cfnow_inspectXML.xml

The import includes the following Namespace/Class/Instance/Method – cfnow/methods/inspectXML/inspectXML

Add a custom button to your desired area of the UI pointing to the inspectXML. When you execute your button you will have dumped to /tmp/ a file named after the object you have executed the button against. Example;

Add button to VM & Template, Click on vm named “Test01″ then you will have a new file /tmp/Test01.xml on your CloudFORMS appliance.

The routine does dump to the automation log but its a little unreadable, only in the exported file is it beautified.

The intention is that now I have the ability to dump any object to XML, and can transform the XML, graphically into SVG and show the object model for any area of the product. So keep following and that will come soon!

InfoBlox IPAM and CloudFORMS Integration Part 1

This subject is only the hottest on the block at the moment…why? I guess because InfoBlox is pretty cool, it provides DHCP, DNS and IP Address Management services for any size of network. Its fairly easy to configure and InfoBlox are a good company who allow you to try their software for 60 days eval license. You can integrate using simple RestAPI (there is a Ruby GEM, but I don’t advise using it, I found it problematic, and Rest is so easy anyway!)

Part 1 will show you how to extend the provisioning StateMachine to obtain an IP address from InfoBlox.

To work this solution you will need;

  • An InfoBlox IPAM server setup with at least one network.
  • A Virtual Machine template, I have forgotten Windows now! So for RHEL, remove the UDEV NIC information, and ensure you have VMware tools installed. Convert it to a template.
  • A LINUX customisation Specification within vSphere, that sets a Static IP Adress, Subnet and Getway, I set all of the values to 1.1.1.1 as they will be replaced later anyway.
  • CloudFORMS InfoBlox methods.
  • CloudFORMS VM Provisioning Statemachine edits to call InfoBlox Method.
  • A Sample Service Provision.

Adding the InfoBlox Methods

Part 1 supplies the first method, Get_IP. This will obtain an IP Address from InfoBlox. Part 2 will show you later how you can release the IP Address back to InfoBlox upon the retirement of a Virtual Machine.

Import the cfnow namespace from here https://github.com/jonnyfiveiq/CloudFORMSNOW/blob/master/Integrations/Provisioning/INFOBLOX/cfnow_ipam.xml

Screen Shot 2013-11-13 at 22.25.04

The new namespace includes a class called InfoBlox, with and instance and a method called get_ip. You will need to edit the get_ip method to specify your username, password and grid server address as follows;

username = “admin”
password = “Smartvm!23″
server = “10.16.132.47”

Adjusting the VM Provisioning State Machine

This is really quite easy, simply edit the Factory / StateMachines / VMProvision_VM / template as follows;

Screen Shot 2013-11-13 at 22.32.22

Take note to add /cfnow/infoblox/get_ip to the AcquireIPAddress state value.

That really concludes enabling the Get IP Address functions for CloudFORMS the following sections of this blog detail how to demonstrate it in a working use case.

Creating a Service to Leverage the Get_IP

The Get_IP method will be called on every provision of a VM, the code within the method does rely on something quite special to exist, which will only be found in a Service Deployment. So do NOT try to deploy VMs using “Provision VM” menu item after enabling this solution as it will not work any more, you can easily fork the Service Provision to use a custom VM Provision thats enabled for IPAM so not to break the old skool VM Provision. Anyway the item the code relies on  is a field in a dialog that will tell IPAM what network to fetch an IP address from. The use case here is that you have TEST, QA and PRODUCTION environments (networks) defined in IPAM. When deploying a new service in CloudFORMS you will have the ability to specify the destination environment to deploy/provision the service into, this will determine the network that should support the service. So you need to create a service dialog as follows;

Screen Shot 2013-11-13 at 22.42.21

The configuration behind the drop down MUST be as follows;

Screen Shot 2013-11-13 at 22.44.13

You can add other environments or change the names, if you do you need to do so in the dialog AND the method. case statement.

case environment

when "qa"
$evm.log("info","qa")

@gateway = "192.168.1.254"
@network = "192.168.1.0/24"
@dnsdomain = "acme.com"

when "test"
$evm.log("info","test")

@gateway = "192.168.2.254"
@network = "192.168.1.0/24"
@dnsdomain = "acme.com"

when "production"
$evm.log("info","production")

@gateway = "192.168.1.254"
@network = "192.168.1.0/24"
@dnsdomain = "acme.com"

else
$evm.log("info","NOTHING")

end

As you can see from this section of the method, you can list the network to search within IPAM for, the gateway to assign and the DNS Domain to register into for each environment. The selection you make in the dialog directly links to the case statement tot select the network, gateway and DNS domain, the method uses the network to ask InfoBlox IPAM to supply a subnet mask and IP address. The IP address is actually the next available it knows of in the network.

The next thing to do is to wrap this up into a service. This you need to create a standard service item or bundle. Give it a resource to deploy, and use a dialog that uses the drop down list box to provide the environment look up.

Note, when doing a catalog bundle you must order the resources in serial manner otherwise the IPAM system can deliver the same IP address to more than one system.

Putting it all together

With you service configured with a dialog that has the environment drop down, and you have the infoblox methods imported and the vm provisioning state machine calling the new get_ip method. The only last thing to do is ensure that the resource that your service is calling is configured correctly. The resource will need to be a operating system that supports either kickstart or sysprep customisation specifications. In my examples I will use RHEL and Kickstart.

My RHEL virtual machine was prepared as follows;

  • Installed RHEL 6.4 as basic server.
  • Removed NIC entries from /etc/udev/rules.d/70-persistent-net.rules
  • Installed VM tools.

I added a customisation specification to vSphere. Here are some screen shots to the specification customization.

Screen Shot 2013-11-13 at 23.31.41
Screen Shot 2013-11-13 at 23.32.12

Screen Shot 2013-11-13 at 23.30.46

Screen Shot 2013-11-13 at 23.31.04

So when we deploy within CloudFORMS it will look something like…

Screen Shot 2013-11-13 at 23.35.46
And when the service resource is provisioned, e.g. that the VM is created and the state machine calls the get_ip methods, the automate log will show something like;

[----] I, [2013-11-14T04:38:57.570595 #6938:a6f810] INFO -- : Q-task_id([miq_provision_1000000000072]) Processing State=[AcquireIPAddress]
[----] I, [2013-11-14T04:38:57.571350 #6938:a6f810] INFO -- : Q-task_id([miq_provision_1000000000072]) Following Relationship [miqaedb:/cfnow/infoblox/get_ip#create]
[----] I, [2013-11-14T04:38:57.598465 #6938:a6f810] INFO -- : Q-task_id([miq_provision_1000000000072]) Invoking [inline] method [cfnow/infoblox/get_ip] with inputs [{}]
[----] I, [2013-11-14T04:38:57.941651 #6938:659d598] INFO -- : Q-task_id([miq_provision_1000000000072]) &lt;User-Defined Method&gt; ********* InfoBlox - GetIP STARTED *********
[----] I, [2013-11-14T04:38:58.306802 #6938:659d598] INFO -- : Q-task_id([miq_provision_1000000000072]) &lt;User-Defined Method&gt; production
[----] I, [2013-11-14T04:38:58.307772 #6938:659d598] INFO -- : Q-task_id([miq_provision_1000000000072]) &lt;User-Defined Method&gt; GetIP --&gt; Network Search - 192.168.1.0/24
[----] I, [2013-11-14T04:38:58.366190 #6938:659d598] INFO -- : Q-task_id([miq_provision_1000000000072]) &lt;User-Defined Method&gt; GetIP --&gt; Network Found - network/ZG5zLm5ldHdvcmskMTkyLjE2OC4xLjAvMjQvMA:192.168.1.0/24/default
[----] I, [2013-11-14T04:38:58.367233 #6938:659d598] INFO -- : Q-task_id([miq_provision_1000000000072]) &lt;User-Defined Method&gt; GetIP --&gt; NextIP on - network/ZG5zLm5ldHdvcmskMTkyLjE2OC4xLjAvMjQvMA:192.168.1.0/24/default
[----] I, [2013-11-14T04:38:58.589829 #6938:659d598] INFO -- : Q-task_id([miq_provision_1000000000072]) &lt;User-Defined Method&gt; GetIP --&gt; NextIP is - 192.168.1.3
[----] I, [2013-11-14T04:38:58.709810 #6938:659d598] INFO -- : Q-task_id([miq_provision_1000000000072]) &lt;User-Defined Method&gt; GetIP --&gt; JH-Service_0006.acme.com with IP Address 192.168.1.3 created successfully
[----] I, [2013-11-14T04:38:58.713179 #6938:659d598] INFO -- : Q-task_id([miq_provision_1000000000072]) &lt;User-Defined Method&gt; GetIP --&gt; Netmask = 255.255.255.0
[----] I, [2013-11-14T04:38:58.716879 #6938:659d598] INFO -- : Q-task_id([miq_provision_1000000000072]) &lt;User-Defined Method&gt; GetIP --&gt; Hostname = JH-Service_0006
[----] I, [2013-11-14T04:38:58.717736 #6938:659d598] INFO -- : Q-task_id([miq_provision_1000000000072]) &lt;User-Defined Method&gt; GetIP --&gt; IP Address = 192.168.1.3
[----] I, [2013-11-14T04:38:58.719000 #6938:659d598] INFO -- : Q-task_id([miq_provision_1000000000072]) &lt;User-Defined Method&gt; GetIP --&gt; Netmask = 255.255.255.0
[----] I, [2013-11-14T04:38:58.719740 #6938:659d598] INFO -- : Q-task_id([miq_provision_1000000000072]) &lt;User-Defined Method&gt; GetIP --&gt; Gateway = 192.168.1.254
[----] I, [2013-11-14T04:38:59.029502 #6938:659d598] INFO -- : Q-task_id([miq_provision_1000000000072]) &lt;User-Defined Method&gt; ********* InfoBlox - GetIP COMPLETED *********
[----] I, [2013-11-14T04:38:59.042234 #6938:a6f810] INFO -- : Q-task_id([miq_provision_1000000000072]) Method exited with rc=MIQ_OK
[----] I, [2013-11-14T04:38:59.251708 #6938:a6f810] INFO -- : Q-task_id([miq_provision_1000000000072]) Followed Relationship [miqaedb:/cfnow/infoblox/get_ip#create]
[----] I, [2013-11-14T04:38:59.252109 #6938:a6f810] INFO -- : Q-task_id([miq_provision_1000000000072]) Processed State=[AcquireIPAddress] with Result=[ok]

And thats it! The Service is provisioning….

Screen Shot 2013-11-13 at 23.47.38

Last thing to do is show the IP address in the new Service resource VM.

Screen Shot 2013-11-13 at 23.51.13

Dialogs – Dynamic DropDowns

Its here and looks great. You can now populate a drop down list from an external source!

Simply create an Instance and Method, its advised you create a new Namespace to store dialog controls into rather than mixing them up into the general automation model.

The method requires the following code behind it;

dialog_field = $evm.object</code>

# sort_by: value / description / none
dialog_field["sort_by"] = "value"

# sort_order: ascending / descending
#dialog_field["sort_order"] = "ascending"

# data_type: string / integer
dialog_field["data_type"] = "integer"

# required: true / false
#dialog_field["required"] = "true"

#Return as a hash

dialog_field["values"] = {1 =&gt; "one", 2 =&gt; "two", 10 =&gt; "ten", 50 =&gt; "fifty"}

#Return as an array

dialog_field["values"] = [[1, "one"], [2, "two"], [10, "ten"], [50, "fifty"]]

dialog_field["default_value"] = 2

exit MIQ_OK

So the code is very simple, there are a number of settings you can control such as;

  • data_type
  • sort_order
  • sort_by
  • required

The the values are simply inserted into the drop down as EITHER a hash or array. The above example shows both, you would use either.

The next step is to create a dialog and add the element “Dynamic Dropdown List” to your form and set the NameSpace/Class/Instance to your Instance/Method you created in the previous step.

Screen Shot 2013-10-24 at 11.55.37

And what does it look like when finished?

Screen Shot 2013-10-24 at 11.56.29

The use cases for this are quite large, here are some;

  • Populate the list of Load Balancer Pools available from F5 Big IP, allow the consumer to choose which pool to place their service into.
  • Populate the list with the available AMI’s in Amazon you wish to share off based on a tag.
  • Populate the list with the OpenStack environments available, allowing the user to place their Instance based on that or even linked to department etc..
  • Populate the list with catalogs from a software distribution system to allow the user to have collections of software installed into the service. This could be SCCM, Puppet, Chef etc..