Pablo Iranzo Gómez's blog

feb 23, 2017

InfraRed for deploying OpenStack

InfraRed is tool that allows to install/provision OpenStack. You can find the documentation for the project at

Also, developers and users are online in FreeNode at #infrared channel.

Why InfraRed?

Deploying OSP with OSP-d (TripleO) requires several setup steps for preparation, deployment, etc. InfraRed simplifies them by automating with ansible most of those steps and configuration.

  • It allows to deploy several OSP versions
  • Allows to ease connection to installed vm roles (Ceph, Computes, Controllers, Undercloud)
  • Allows to define working environments so one InfraRed-running host can be used to manage different environments
  • and much more...

Setup of InfraRed-running host

Setting InfraRed is quite easy, at the moment the version 2 (branch on github) is working pretty well.

We'll start with:

  • Clone GIT repo: git clone
  • Create a virtual ENV so we can proceed with installation, later we'll need to source it before each use. cd infrared ; virtualenv .venv && source .venv/bin/activate
  • Proceed with upgrade of pip and setuptools (required) and installation of InfraRed
    • pip install --upgrade pip
    • pip install --upgrade setuptools
    • pip install .

Remote host setup

Once done, we need to setup the requirements on the host we'll use to virtualize, this includes, having the system registered against a repository providing required packages.

  • Register RHEL7 and update:
    • subscription-manager register (provide your credentials)
    • subscription-manager attach --pool= (check pool number first)
    • subscription-manager repos --disable=*
    • for canal in rhel-7-server-extras-rpms rhel-7-server-fastrack-rpms rhel-7-server-optional-fastrack-rpms rhel-7-server-optional-rpms rhel-7-server-rh-common-rpms rhel-7-server-rhn-tools-rpms rhel-7-server-rpms rhel-7-server-supplementary-rpms rhel-ha-for-rhel-7-server-rpms;do subscription-manager repos --enable=$canal; done


  • OSP7 did not contain RPM packaged version of images, a repo with the images needs to be defined like:
    • time infrared tripleo-undercloud --version $VERSION --images-task import --images-url $REPO_URL
    • NOTE: --images-task import and --images-url
  • Ceph failed to install unless --storage-backend ceph was provided (open bug for that)

Error reporting

  • IRC or github


Some bugs/RFE on the way to get implemented some day:

  • Allow use of localhost to launch installation against local host
  • Multi env creation, so several osp-d versions are deployed on the same hypervisor but one launched
  • Automatically add --storage-backend ceph when ceph nodes defined

Using Ansible to deploy InfraRed

This is something that I began testing to automate the basic setup, still is needed to decide version to use, and do deployment of infrastructure vm's but does some automation for setting up the hypervisors.

- hosts: all
  user: root

    - name: Install git
          - "git"
          - "python-virtualenv"
          - "openssl-devel"
        state: latest

    - name: "Checkout InfraRed to /root/infrared folder"
        dest: /root/infrared

    - name: Initialize virtualenv
        virtualenv: "/root/infrared/.venv"
        name: setuptools, pip

    - name: Upgrade virtualenv pip
        virtualenv: "/root/infrared/.venv"
        name: pip
        extra_args: --upgrade

    - name: Upgrade virtualenv setuptools
        virtualenv: "/root/infrared/.venv"
        name: setuptools
        extra_args: --upgrade

    - name: Install InfraRed
        virtualenv: "/root/infrared/.venv"
        name: file:///root/infrared/.

This playbook will do checkout of git repo, setup extra pip commands to upgrade virtualenv's deployed pip and setuptools, etc.

Deploy environment examples

This will show the commands that might be used to deploy some environments and some sample timings on a 64Gb RAM host.

Common requirements

export HOST_KEY=~/.ssh/id_rsa
export ANSIBLE_LOG_PATH=deploy.log


time infrared virsh --cleanup True --host-address $HOST --host-key $HOST_KEY

OSP 9 (3 + 2)

Define version to use

export VERSION=9

time infrared virsh --host-address $HOST --host-key $HOST_KEY --topology-nodes "undercloud:1,controller:3,compute:2"

real    11m19.665s
user    3m7.013s
sys     1m27.941s

time infrared tripleo-undercloud --version $VERSION --images-task rpm

real    48m8.742s
user    10m35.800s
sys     5m23.126s

time infrared tripleo-overcloud --deployment-files virt --version 9 --introspect yes --tagging yes --post yes

real    43m44.424s
user    9m36.592s
sys     4m39.188s

OSP 8 (3+2)

export VERSION=8

time infrared virsh --host-address $HOST --host-key $HOST_KEY --topology-nodes "undercloud:1,controller:3,compute:2"

real    11m29.478s
user    3m10.174s
sys     1m28.276s

time infrared tripleo-undercloud --version $VERSION --images-task rpm

real    40m47.387s
user    9m14.151s
sys     4m24.820s

time infrared tripleo-overcloud --deployment-files virt --version $VERSION --introspect yes --tagging yes --post yes

real    42m57.315s
user    9m2.412s
sys     4m25.840s

OSP 10 (3+2)

export VERSION=10

time infrared virsh --host-address $HOST --host-key $HOST_KEY --topology-nodes "undercloud:1,controller:3,compute:2"

real    10m54.710s
user    2m42.761s
sys     1m12.844s

time infrared tripleo-undercloud --version $VERSION --images-task rpm

real    43m10.474s
user    8m34.905s
sys     4m3.732s

time infrared tripleo-overcloud --deployment-files virt --version $VERSION --introspect yes --tagging yes --post yes

real    54m1.111s
user    11m55.808s
sys     6m1.023s

OSP 7 (3+2+3)

export VERSION=7

time infrared virsh --host-address $HOST --host-key $HOST_KEY --topology-nodes "undercloud:1,controller:3,compute:2,ceph:3"

real    13m46.205s
user    3m46.753s
sys     1m47.422s

time infrared tripleo-undercloud --version $VERSION --images-task import    --images-url $URLTOIMAGES

real    43m14.471s
user    9m45.479s
sys     4m53.126s

time infrared tripleo-overcloud --deployment-files virt --version $VERSION --introspect yes --tagging yes --post yes     --storage-backend ceph

real    86m47.471s
user    20m2.582s
sys     9m42.577s


Please do refer to the InfraRed documentation to get deeper in its possibilities and if interested, consider contributing!

Click to read and post comments

feb 20, 2017

Getting started with Ansible

I've started to get familiar with Ansible because, apart of getting more and more accepted for OSP-related tasks and installation, I wanted to automate some tasks we needed to setup some servers for the OpenStack group I work for.

First of all, it's recommended to get latest version of ansible (tested on RHEL7 and Fedora), but in order not to mess with the system python libraries, it's convenient to use python's virtual environments.

A virtual Environment allows to create a 'chroot'-like enviroment that might contain different library versions to the one installed with the system (but be careful as if it's not kept track as part of the usually system patching process, it might become a security concern).


For creating a virtualenv, we require the package python-virtualenv installed on our system and executing virtualenv and a target folder, for example:

[iranzo@iranzo ~]$ virtualenv .venv
New python executable in /home/iranzo/.venv/bin/python2
Also creating executable in /home/iranzo/.venv/bin/python
Installing setuptools, pip, wheel...done.

From this point, we've a base virtualenv installed, but as we would like to install more packages inside we'll first need to 'enter' into it:

. .venv/bin/activate

And from there, we can list the available/installed packages:

[iranzo@iranzo ~]$ pip list
DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning.
appdirs (1.4.0)
packaging (16.8)
pip (9.0.1)
pyparsing (2.1.10)
setuptools (34.2.0)
six (1.10.0)
wheel (0.30.0a0)

Now, all packages we install using pip will get installed to this folder, leaving system libraries intact.

Once we finished, to return back to system's environment, we'll execute deactivate.


In order to simplify the management we can make use of pipsi which not only allows to install Python packages as we'll normally do with pip, but also, takes care of doing proper symlinks so the installed packages are available directly for execution.

If our distribution provides it, we can install pipsi on our system:

dnf -y install pipsi

But if not, we can use this workaround (for example, on RHEL7)

# Use pip to install pipsi on the system (should be minor change not affecting other software installed)
pip install pipsi

From this point, we can use pipsi to take care of installation and maintenance (can do upgrades, removal, etc) of our python packages.

For example, we can install ansible by executing:

pipsi install ansible

This might fail, as ansible, does some compiling and for doing so, it might require some development libraries on your system, have care of that to satisfy requirements for the packages.

Prepare for ansible utilization

At this point we've the ansible binary available for execution as pipsi did take care of setting up the required symlinks, etc

Ansible uses an inventory file (can be provided on command line) so it can connect to the hosts listed there and apply playbooks which define the different actions to perform.

This file, for example, can consist of just a simple list of hosts to connect to like:

And for starting we create a simple playbook, for example a HW asset inventory:

- hosts: all
  user: root

    - name: Display inventory of host
        msg: "{{ inventory_hostname }} | {{ ansible_default_ipv4.address }} | | | {{ ansible_memtotal_mb }} | | | {{ ansible_bios_date }}"

This will act on all hosts, as user root and will run a task which prints a debug message crafted based on the contents of some of the facts that ansible gathers on the execution host.

To run it is quite easy:

[iranzo@iranzo labs]$ ansible-playbook -i, inventory.yaml

PLAY [all] *********************************************************************

TASK [setup] *******************************************************************
ok: []

TASK [Display inventory of host] ***********************************************
ok: [] => {
    "msg": " | | | | 14032 | | | 01/01/2011"

PLAY RECAP *********************************************************************             : ok=2    changed=0    unreachable=0    failed=0

This has connected to the target host, and returned a message with hostname, ip address, some empty fields, total memory and bios date.

This is a quite simple script, but for example, we can use ansible to deploy ansible binary on our target host using other modules available, in this case, for simplicity, we'll not be using pipsi for ansible installation.

- hosts: all
  user: root

    - name: Install git
          - "git"
          - "python-virtualenv"
          - "openssl-devel"
        state: latest

    - name: Install virtualenv
        virtualenv: "/root/infrared/.venv"
        name: pipsi

    - name: Upgrade virtualenv pip
        virtualenv: "/root/infrared/.venv"
        name: pip
        extra_args: --upgrade

    - name: Upgrade virtualenv setuptools
        virtualenv: "/root/infrared/.venv"
        name: setuptools
        extra_args: --upgrade

    - name: Install Ansible
        virtualenv: "/root/infrared/.venv"
        name: ansible

At this point, the system should have ansible available from within the virtualenv we've created and should be avialble when executing:

# Activate python virtualenv
. .venv/bin/activate
# execute ansible
ansible-playbook -i hosts ansible.yaml

Have fun!

Click to read and post comments

nov 05, 2016

Unit testing for stampy

Since my prior post on Contributing to OpenStack, I liked the idea of using some automated tests to validate functionality and specifically, the corner cases that could arise when playing with the code.

Most of the errors fixed so far on stampy, were related with some pieces of the code not properly handling UTF or some information returned, etc and still it has improved, the idea of ensuring that prior errors were not put back into the code when some other changes were performed, started to arise to be a priority.

For implementing them, I made use of nose, which can be executed with nosetests and are available on Fedora as 'python-nose' and to provide further automation, I've also relied on tox also inspired n what OpenStack does.

Let's start with tox: once installed, a new configuration file is created for it, defining the different environments and configuration in a similar way to:

minversion = 2.0
envlist = py27,pep8
skipsdist = True

passenv = CI TRAVIS TRAVIS_*
deps = -r{toxinidir}/requirements.txt
commands =
    /usr/bin/find . -type f -name "*.pyc" -delete
    nosetests \
commands = flake8

commands = {posargs}

commands =
  coverage report

show-source = True

This file, defines two environments, one for validating pep8 for the python formatting and another one for validating python 2.7.

The environment definition for the tests, also performs some commands like executed the forementioned nosetests to run the defined unit tests.

Above tox.ini also mentions requirements.txt and test-requirements.txt, which define the python packages required to validate the program, that will be automatically installed by tox on a virtualenv, so the alternate versions being used, doesn't interfere with the system-wide ones we're using.

About the tests themselves, as nosetests does automatic discovery of tests to perform, I've created a new folder named tests/ and placed there some files in alphabetically order:

ls -l tests
total 28
-rw-r--r--. 1 iranzo iranzo  709 nov  5 16:58
-rw-r--r--. 1 iranzo iranzo  739 nov  3 09:56
-rw-r--r--. 1 iranzo iranzo  456 nov  3 23:53
-rw-r--r--. 1 iranzo iranzo  581 nov  3 09:56
-rw-r--r--. 1 iranzo iranzo 3544 nov  5 18:19
-rw-r--r--. 1 iranzo iranzo  477 nov  3 23:15
-rw-r--r--. 1 iranzo iranzo  230 nov  3 09:56

First one test_00-setup takes the required commands to define the enviroment, as on each validation run of tox, a new environment should be available not to mask errors that could be overlooked.

#!/usr/bin/env python
# encoding: utf-8

from unittest import TestCase

from stampy.stampy import config, setconfig, createdb, dbsql

# Precreate DB for other operations to work

# Define configuration for tests
setconfig('token', '279488369:AAFqGVesZ-81n9sFafLQxUUCVO8_8L3JNEU')
setconfig('owner', 'iranzo')
setconfig('url', '')
setconfig('verbosity', 'DEBUG')

# Empty karma database in case it contained some leftover
dbsql('DELETE from karma')
dbsql('DELETE from quote')

class TestStampy(TestCase):
    def test_owner(self):
        self.assertEqual(config('owner'), 'iranzo')

This file creates the database if none is existing and defines some sample values, like DEBUG level, url for contacting telegram API servers, or even a token that can be used to test the functionality for sending messages.

Also, if the database is already existing, empties the karma table, quotes (and sets sequence to 0 to simulate TRUNCATE which is not available on sqlite)

An unittest is specified under the class inherited from TestCase imported from unittest, there for each one of the tests we want to performed, a new 'definition' is created and after it an assert is used, for example assertEqual validates that the function call returns the value provided as secondary argument, failing otherwise.

From that point, the tests are performed again in alphabetically order, so be careful in the naming of each tests or define a sequence number to use a top-to-bottom approach that will be probably easier to understand.

For example, for karma changes we've:

#!/usr/bin/env python
# encoding: utf-8

from unittest import TestCase

from stampy.stampy import getkarma, updatekarma, putkarma

class TestStampy(TestCase):
    def test_putkarma(self):
        putkarma('patata', 0)
        self.assertEqual(getkarma('patata'), 0)

    def test_getkarma(self):
        self.assertEqual(getkarma('patata'), 0)

    def test_updatekarmaplus(self):
        updatekarma('patata', 2)
        self.assertEqual(getkarma('patata'), 2)

    def test_updatekarmarem(self):
        updatekarma('patata', -1)
        self.assertEqual(getkarma('patata'), 1)

Which starts by putting a known karma on a word, validating, verifying the query, update the value by a positive number and later, decrease it with a negative one.

For the aliases, we use a similar aproach, as we also play with the karma changes when an alias is defined:

#!/usr/bin/env python
# encoding: utf-8

from unittest import TestCase

from stampy.stampy import getkarma, putkarma, createalias, getalias, deletealias

class TestStampy(TestCase):

    def test_createalias(self):
        createalias('patata', 'creilla')
        self.assertEqual(getalias('patata'), 'creilla')

    def test_getalias(self):
        self.assertEqual(getalias('patata'), 'creilla')

    def test_increasealiaskarma(self):
        updatekarma('patata', 1)
        self.assertEqual(getkarma('patata'), 1)

        # Alias doesn't get increased as the 'aliases' modifications are in
        # process, not in the individual functions
        self.assertEqual(getkarma('creilla'), 0)

    def test_removealias(self):
        self.assertEqual(getkarma('creilla'), 0)

    def test_removekarma(self):
        putkarma('patata', 0)
        self.assertEqual(getkarma('patata'), 0)

Where an alias is created, verified, karma in creased on the word with an alias, and then the aliased value.

As noted in the above example, the individual function for the karma doesn't take into consideration the aliases so this must be handled by processing a message set via process(messages) which has been also modified as well as other functions to allow the implementation of individual tests for them.

This will for sure end up with some more code rewriting so the functions can be fully tested individually and as a whole, to ensure that the bot behaves as intended... and many more tests to come to the code.

As an end, an example of the execution of tox and the results raised:

py27 installed: coverage==4.2,nose==1.3.7,prettytable==0.7.2
py27 runtests: PYTHONHASHSEED='604985980'
py27 runtests: commands[0] | /usr/bin/find . -type f -name *.pyc -delete
py27 runtests: commands[1] | nosetests
Ran 18 tests in 14.996s

pep8 installed: coverage==4.2,nose==1.3.7,prettytable==0.7.2
pep8 runtests: PYTHONHASHSEED='604985980'
pep8 runtests: commands[0] | flake8
WARNING:test command found but not installed in testenv
  cmd: /usr/bin/flake8
  env: /home/iranzo/DEVEL/private/stampython/.tox/pep8
Maybe you forgot to specify a dependency? See also the whitelist_externals envconfig setting.
__________________________________________________________________________ summary ___________________________________________________________________________
  py27: commands succeeded
  pep8: commands succeeded
  congratulations :)

If you're using a CI system, like 'Travis', which is also available to repos, a .travis.yml can be added to the repo to ensure those tests are performed automatically on each code push:

language: python
    - 2.7

    email: false

    - pip install pep8
    - pip install misspellings
    - pip install nose

    # Run pep8 on all .py files in all subfolders
    # (I ignore "E402: module level import not at top of file"
    # because of use case sys.path.append('..'); import <module>)
    - find . -name \*.py -exec pep8 --ignore=E402,E501 {} +
    - find . -name '*.py' | misspellings -f -
    - nosetests


Click to read and post comments

jul 21, 2016

Contributing to OpenStack

Contributing to an opensource project might take some time at the beginning, the good thing with OpenStack is that there are lot of guides on how to start and collaborate.

What I did is to look for a bug in the project tagged as low-hanging-fruit, this allows to browse a large list of bugs that are classified as easy, so they are the best place for new starters to get familiar with the workflow.

I did found an issue with weight which is supposed to be an integer, that was doing a conversion from float to integer (0.1 -> 0) which was considered invalid, and instead an error should be returned.

When I checked the Neutron-LBaaS I found out where the problem was, as the value provided, was being converted to integer instead of validating it.

Before contributing you need to:

Submitting a change is quite easy:

# Select the project, 'neutron-lbaas' for me
git clone$each.git
cd $each
# This setups git-review, getting required hooks, etc
git-review -s
# create a new branch so we can keep our changes separate
git branch new-branch

# Edit files with changes
git add $files
git commit -m "Descriptive message"
# send  to upstream for review:

git-review will output an url you can use to preview your change, and the hooks will automatically add a 'Change-ID' so subsequent changes are linked to it.

NOTE: full reference is available at the Developer's Guide

The biggest issue started here:

  • In order to not require a new function to validate integers, I've used the one for non-negative which already does this tests, but one of the reviewers suggested to write a function
  • Functions were imported from neutron-lib so I submitted a second change to neutron-lib project
  • As the change in neutron-lib couldn't be marked as dependent as neutron-lbaas uses the build the version already published, I had to define another interim version of the function so that neutron-lbaas can use it in the meantime and raise another bug, to later remove this interim function once than neutron-lib includes the validate_integer function
  • As part of the comments on neutron-lib review, it was found that it would be nice to validate values, so after some discussion, I moved to use the internal validate_values.
  • Of course, validate_values is just doing data in valid_values, so it fails if data or valid_values are not comparable and doesn't do conversion of depending on the values itselves, so this spin-off another review for improving the ´validate_values´ function.

At the moment, I'm trying to close the one to neutron-lib to use the function already defined, and have it merged, and then continue with the other steps, like removing the interim function in neutron-lbaas and work on enhancing validate_values and close all the dependant launchpad bugs I've created for tracking.

My experience so far, is that sometimes it might be a bit difficult, as git-review is a collaborative environment so different opinions are being shared with different approachs and some of them are 'easier' and some others 'pickier' like having an 'extra space', etc.

Of course, all the code is checked by some automation engines when submitted, which validates that the code still builds, no formatting errors, etc but many of them can be executed locally by using tox, which allows to perform part of the tests like:

  • tox -e pep8
  • tox -e py27
  • tox -e coverage

To respectively, validate pep8 formatting (line length, spaces around operators, docsstrings formatting, etc) and to run another set of tests like the ones you define.

After each set of changes performed to apply the feedback received, ensure to:

# Add the modified files to a commit

git add $files_modified

# Create the commit with the changes

git commit -m "whatever"

# This will show you the last two commits, so you can leave the first one and
# on the beginning of the second one,
# replace 'pick' for 'f' so the changes are merged with first one without
# keeping the commit message

git rebase -i HEAD~2

# Fix the commit message if needed  (like fixing formatting,
# set dependant commits, or bugs it closes, etc)

git commit --amend

# Submit changes again for review


Also, keep in mind that apart from submitting the code change is important to submit automated validation tests, which can be executed with tox -e py27 to view that the functions return the values we expect even if the input data is out of what it should be, or like coverage, to validate that the code is covered (check on tox.ini what is defined).

And last but not least, expect to have lot of comments on more serius changes like changes to stable libs, as lot of reviewers will come to ensure that everything looks good and might even discuss it on the regular meetings to ensure, that a change is a good fit for the product in the proposed approach.

Click to read and post comments

jun 03, 2016

New blog rendering engine: Pelican

As always, I don't usually find myself keen to write about things I do, until I later realize they might be helpful for others, and that's why in the past I decided to switch the place I was putting the information about why did to Github and also, take benefit of practicing markdown for writing the entries.

At that time, I moved my old blog posts to markdown to be used in conjunction with Jekyll and to use Octopress as the engine rendering the contents into a static website. The setup and migration was not difficult, but still require to use some ruby, while I was more familiar with Python.

Since some time ago, I was checking other platforms, following the same approach of rendering markdown files and sticker to Pelican, it's included in Fedora Repos (python-pelican). Pelican offers a similar behaviour, having also a server for allowing you to quickly test the new settings (plugins, themes, etc) and to publish the resulting website to a hosting provider.

As I did with Jekyll+Octopress, I'm still using for it, and I'm in the process of adapting some changes like additional plugins, theme tweaks and consider to develop one of my own.

Click to read and post comments
← Previous Next → Page 2 of 14