Red Hat’s virtualization strategy has redundancy – Quo vadis?

A couple of days there have been some reports that Red Hat will release a commercialized version of deltacloud, an abstraction layer for different kinds of virtualization technologies and clouds such as VMware, RHEV, Amazon EC2 etc.

Red Hat puts a lot of resources on virtualization, they maintain and/or sponsor multiple projects in parallel. The most important from my point of view is libvirt which is as well an abstraction layer for different virtulization technologies such as VMware, KVM, Xen and others. Libvirt and deltacloud are partially redundant.

It is not the only redundancy created by Red Hat. There is also O-virt “competing” with RHEV. Both are not tightly bound to RHN satellite or Spacewalk.

RHEV works with system templates similar to those at VMware. On the other hand: Koan, together with cobbler is a deployment software for virtual hosts and was recently bundled with RHN satellite.

Not all of those Red Hat virtualization projects are working well together. So the question arises: What is the strategy of having such redundancies of projects? Why not integrating all of this projects and glue them together?

Lots of questions…

Have fun!

A brief test of OTRS::ITSM Changemanagement and the insanity of ITIL compliant software

OTRS is known as best-of-breed in open source incident management systems. Since quite some time, OTRS made its product ITIL V3 compliant. Means: It also comes with a change management module.

At work we use a complex and extremely user-unfriedly software. This brought me to the idea to test the OTRS change management module in order to propose OTRS as a replacement for the currently used software for the Change- Incident- and Problem Management.

Incident- and Change Management integration
I was surprised how easy it is to change the ITIL type of a ticket from “Incident” to “RfC”. As soon as the ticket is of type RfC a new button appears: Create Change. The ticket gets automatically linked to the new change. Creating the change looks strait forward. Assigning people to the CAB was also quite easy. You also can generate a CAB-template with a few clicks. So far so good…

Where the trouble begins
The newly created change is now of status “Requested”. Whats next? Right! to approve it! But how? OTRS is using something they call “state engine”. You need to add “Workorders” and “Conditions” to your change. For a standard-change I made a template with a condition “If workorder-title=Standard-Change, set change to status approved“. In this case you just link a “workorder” to the change, call it “Standard-Change” and your change will be approved. Next Condition is to set the state of the change to “successful” when the “Workorder” is of state closed. At the end of the day three tasks and approx. 20 clicks for a simple standard-change. Not too bad.

Where it goes to insanity
Non-Standard-Changes usually have a CAB (Change Advisory Board). This makes sense because the change-requester usually does mot have the full overview about complex systems and services. Now, as I wrote further up, it is quite easy to create and assign a CAB to a change. So how works the process? Usually every single member of the CAB must approve a particular change. It should be easy to send all the CAB-Members a Email with a link where they can approve or reject the change. In OTRS this is a huge and very complex task.

The change manager or change creator has to create a “workworder” of type “Approval” for every single CAB-Member AND create a condition to it. If you plan a huge change such as upgrading Powerlines in a Datacenter, the CAB can grow to dozens of people. I tried with two CAB members and it was costing me about 20 minutes to create it (Without proper texts in the change and workorders). Think about a 20-people CAB. It will take hours just to create a proper change! This is so nuts!

Why are all ITIL compliant change mangement tools just crap?
ITIL processes are quite simple. One should think it is also easy to implement them in software and in companies. Wrong! The mind of People with ITIL-Roles such as “Change Manager”, “Problem Manager”, “Availability Manager” and “you-name-it-manager” works obviously different. It looks like they add as much complexity as possible even to every simple task. Obviously the ITIL-compliant software developers think the same way or got the orders to do so. I think this is the root cause of the completely unusable software OTRS::ITSM Changemanagement and others such as Remedy and Peregrine.

Conclusion
As there is no easy usable software on the market, companies should either write its own software or getting the less-crappiest software around. At the end of the day I’m tired of this and I’m not going to test similar software again.

IUS Community RPMs for Red Hats RHEL

I was criticizing that software in RHEL is too outdated for web servers quite soon after release, see my blog post http://blog.delouw.ch/2010/05/02/rhel6-as-a-web-server/. While this is true for a system fully supported by Red Hat, I learned an alternative from a comment on the post. This alternative is the so called IUS community repository.

About the IUS Community Project
The project was launched in September 2009. In spite of being a young project, it has a history. At Rackspace, a large hosting company which is operating thousands of production (web) servers, it was an internal project since 2006. They decided to build up a community around it, like Fedora is for RHEL, Quote: “IUS is The Fedora of Rackspace RPMS”

Support
Like for other community repositories out there, you cannot expect a “official” support neither from Red Hat nor from IUS or Rackspace. Of course there are the usual support sources for communities such as forums, IRC, bugtracker etc.

The difference to other repositories
While most community repositories such as EPEL, rpmforge etc. are focused on providing missing software, IUS focuses on providing upgrades for web server related software which is included in RHEL. This includes PHP, Python, MySQL and others.

Package conflicts with the stock distribution
One may think replace stock software with newer version is tricky and create conflicts. There is one way to find out: Lets give it a try…

The test
The server is a basic install of the yesterday released Centos 5.5. The following installation turns this machine in a lightweight LAMP server:

yum install httpd php-mysql php php-cli php-common php-pgsql php-dba php-pdo php-gd mysql-server perl-DBD-MySQL.

Now we have the situation like it exists in many companies: An outdated webserver. Now we want to upgrade PHP to 5.3.x. Lets see what happens.


[root@centos5 ~]# rpm -i http://dl.iuscommunity.org/pub/ius/stable/Redhat/5/x86_64/ius-release-1-4.ius.el5.noarch.rpm
warning: /var/tmp/rpm-xfer.o6JH6k: Header V3 DSA signature: NOKEY, key ID 9cd4953f
[root@centos5 ~]# rpm -i http://dl.iuscommunity.org/pub/ius/stable/Redhat/5/x86_64/epel-release-1-1.ius.el5.noarch.rpm
warning: /var/tmp/rpm-xfer.MRnuo8: Header V3 DSA signature: NOKEY, key ID 9cd4953f
package epel-release-5-3.noarch (which is newer than epel-release-1-1.ius.el5.noarch) is already installed
[root@centos5 ~]#

Hmm… no GPG key…
The second output is confusing me. Is the package just a clone of epel-release-5-3.noarch? Lets go forward to see if it is working.

“yum clean-all && yum check-update” did not show any pending updates, so far so good. Now lets try to upgrade php.


root@centos5 ~]# yum install php53
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* addons: mirror.netcologne.de
* base: mirror.netcologne.de
* epel: mirror.andreas-mueller.com
* extras: mirror.netcologne.de
* ius: ftp.astral.ro
* updates: mirror.netcologne.de
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package php53.x86_64 0:5.3.2-3.ius.el5 set to be updated
--> Processing Dependency: php53-common = 5.3.2-3.ius.el5 for package: php53
--> Processing Dependency: php53-cli = 5.3.2-3.ius.el5 for package: php53
--> Processing Dependency: php53-pear >= 1:1.8 for package: php53

[omitted output]

--> Processing Conflict: php53 conflicts php < 5.3 --> Finished Dependency Resolution
php53-5.3.2-3.ius.el5.x86_64 from ius has depsolving problems
--> php53 conflicts with php
Error: php53 conflicts with php
You could try using --skip-broken to work around the problem
You could try running: package-cleanup --problems
package-cleanup --dupes
rpm -Va --nofiles --nodigest
The program package-cleanup is found in the yum-utils package.

Correct behaviour, since it is a replacement package. After removing php (and only php) yum was complaining about more conflicts. After removing all php related packages installed to prepare for the test, needed to be removed. So the dependencies has been proper solved. Also the installation of related stock distribution packages such as “php-pgsql” has been successfully prevented.

Conclusion
The IUS community repositories are working as expected. With such a basic test I cannot promise if there are not hidden conflicts with packages between stock RHEL/CentOS packages and those from IUS. The experience on the long term will bring more clarity. I think is is sane to do some real-life tests with servers that are in an early project phase.

Further readings:
http://iuscommunity.org/
http://wiki.iuscommunity.org/
http://saferepo.iuscommunity.org/specification/

Have fun!

KVM supports live migration between CPUs with different features

The video is a bit old, it is from November 2008. But it is still quite interesting to see and discuss about it. With KVM you can upgrade your farm of servers easy, it does not matter if the new servers have CPUs with new features or not. I’m not sure if you can do this with ESX, I guess not, you probably need to migrate them shut down.

Have fun!