Archive for the ‘Virtualization’ Category

Experiences with RHEL6 Beta 2.1

Friday, July 23rd, 2010

Like promised I’ll keep you updated on the RHEL6b2.1. The “official name” is not Beta2.1, it is “Beta 2 refresh”. Why not calling it Beta3? Anyway: The good news first: In contrary to the first release of Beta 2, it works fine again! The first release of Beta2 was quite crappy, it was not installable as a KVM guest. This was obviously due to severe bugs in some virtio drivers.

So, what are the news?

1. The bugs in the virtio drivers have been fixed, you can deploy RHEL6 in KVM environments again.
2. The vmware_ballooning driver has been backported.
3. A lot of minor bugs have been fixed, see the announcement.

Especially point two is cool, running RHEL6 in a VMware ESX environment does not necessarily need the vmware-tools installed anymore. RHEL6 now provides all three important vm-ware related drivers: The vmxnet3, vmware_ballooning and pvscsi. At the end of the day, this means one can dismiss the always-hated vmware-tools. A test of the behavior w/o vmware-tools by a ESX specialist is pending.

The alternative of vmware-tools are the open-vm-tools. This would add the benefit of controlled shutdown of the ESX guest with the vCenter tools. Since VMware does not provide (yet) RHEL 6 packages of the open-vm-tools I was unable to test it.

I made the same brief tests as I reported here. It seems that Red Hat is back on track, RHEL6b2.1 is reliable and not far away from being ready for production.

When can we expect a Beta3? Will there even be a next beta, or is Red Hat release a RC1 soon? There is still no published release schedule, all we know is “later this year”.

Anyway: Download Beta2.1 and test it, its a pretty cool release. If you find bugs, report them.

Have fun!

What is possibly going into RHEL6 GA and what is not

Tuesday, June 1st, 2010

As I wrote different times before, RHEL6 is going to have a Kernel based on upstreams 2.6.32 Kernel. Meanwhile Linus Torvalds and his fellows released 2.6.34. Since then – from a System Engineers Point of view – there have some “minor” changes which are affecting the daily work in enterprise environments.

I think that Red Hat is aware that RHEL6 is one of its most important releases made so far. RHEL6 Beta-Testers have acknowledged that this is one of the best Linux distributions made so far.

So lets have a look to http://bit.ly/98yNsk (https://bugzilla.redhat.com search for RHEL6 select all states, sort by Bug-ID and having RFE (Request For Enhancement) in Summary).

Unrar
I requested to add “unrar” to RHEL, unfortunatly they refused because of the strange license of unrar. This is really not understandable, because *ALL* major Linux distros such as SLES, Debian, Ubuntu are providing a package for it. Red Hat think (and they are right) it is a “unfree” license. From my point of view it does not hurt because nobody is forced to use its libs in own software. Unfortunately SAP distributes a lot of software components in RAR-compressed files, this is a problem.

virtio net/vhost net speed enhancements from upstream kernel
This was reported as bug #593158 and later appeared as #595287. Since Red Hat is keen to improve virtualization things, I think this is going to GA.

DRBD
DRBD was getting into upstream Kernel 2.6.33. DRBD (Distributed Replicated Block Device) is some kind of RAID-1 over TCP/IP and is rock solid since years. From my point of view it is the best invention since sliced bread when it comes to cluster technologies. It is widely used, also on RHEL. Have a look to Florians Haas’ comment about support, and further to Alan Robertson’s comment. While Florian is working at Linbit (the developer company of DRBD) points to support problems existing on current releases on RHEL, Alan is a “Urgestein” (sorry, cant find a English word for it, it is meant in a very positive manner) of Linux clustering likes too to have DRBD in RHEL6. Quite a lot of people are included in the bugs CC list (as I’m writing 37 people). This brings quite some preasure on Red Hat to include DRBD in RHEL6. @Red Hat: Do it! include DRBD! If not as a “supported” product, deliver it and find a way with Linbit for the support.

Getting rid of the crappy VMware-tools
For people urged to use VMWares ESX stuff as virtalization technology, there is another important thing that changed: In 2.6.34 upstream Kernel, Linus Torvalds accepted VMWares ballooning driver (vmmemctl). In 2.6.33 Linus accepted VMWares vmxnet3 and pvscsci drivers which have been already backported to RH’s Kernel 2.6.32-EL. So, also backporting vmmemctl is *THE* chance to get rid of those crappy VMWare Tools. For companies relying on ESX this would be a *VERY* important feature. I’ll made a service request (SR 2021028) @Red Hat and will file a RFE-Bug at bugzilla ASAP. Please vote for it!

Other stuff
There are other RFE’s pending. Most of them are not really important for enterprise computing (my point of view). Mostly this RFE’s are about virtualization and bound to libvirt. Most of these RFE’s seems to be trivial and are on status “ON_QA” which means they are most probably included in RHEL6.

What is your favorit RFE-Bug? Please let me know…

Have fun!

Luc

Red Hat’s virtualization strategy has redundancy – Quo vadis?

Thursday, May 27th, 2010

A couple of days there have been some reports that Red Hat will release a commercialized version of deltacloud, an abstraction layer for different kinds of virtualization technologies and clouds such as VMware, RHEV, Amazon EC2 etc.

Red Hat puts a lot of resources on virtualization, they maintain and/or sponsor multiple projects in parallel. The most important from my point of view is libvirt which is as well an abstraction layer for different virtulization technologies such as VMware, KVM, Xen and others. Libvirt and deltacloud are partially redundant.

It is not the only redundancy created by Red Hat. There is also O-virt “competing” with RHEV. Both are not tightly bound to RHN satellite or Spacewalk.

RHEV works with system templates similar to those at VMware. On the other hand: Koan, together with cobbler is a deployment software for virtual hosts and was recently bundled with RHN satellite.

Not all of those Red Hat virtualization projects are working well together. So the question arises: What is the strategy of having such redundancies of projects? Why not integrating all of this projects and glue them together?

Lots of questions…

Have fun!

KVM supports live migration between CPUs with different features

Saturday, May 15th, 2010

The video is a bit old, it is from November 2008. But it is still quite interesting to see and discuss about it. With KVM you can upgrade your farm of servers easy, it does not matter if the new servers have CPUs with new features or not. I’m not sure if you can do this with ESX, I guess not, you probably need to migrate them shut down.

Have fun!

A brief test of RHEL 6 Beta 1

Friday, April 23rd, 2010

As promised yesterday, I publish the results of a brief test of RHEL6 Beta 1 and the most important findings. It is my point of view as a system guys daily business. If not stated, this overview is based on a default installation with no customization.

General

  • There are new package groups such as  “Minimal” with 228 Packages and “Basic Server” with 523 Packages. “Basic Server” is the default installation, which means the default click trough installation compared to RHEL5 is much less bloated.
  • The versions of the most important software is quite up-to-date but as expected not on the bleeding edge.
  • Postfix is the default MTA. Finally Red Hat managed to switch away from sendmail like other distributions did it years ago.
  • Bye bye SysV init: As I guesstimated in october 2009 RHEL6 comes with upstart instead of traditional SysV init. (See http://blog.delouw.ch/2009/10/31/ready-to-upstart/). The boot process is much faster compared to RHEL5. Upstart comes with legacy support for traditional runcontrol scripts in /etc/init.d.
  • Still too many services enabled after default install. Generally unneeded services like avahi/mDNS and NFS-related daemons such as  portmap are still enabled by default.

Virtualization

As expected, Xen was removed completely from RHEL6. These is being discussed controversial. Why not providing both virtualization solutions as before? Recently Citrix released Xen4 which works well together with Kernel 2.6.32, the same version as used by RHEL6.

KVM and its friends made a huge step forward. lib-virt, virt-manager and stuff is nearly up-to-date with the upstream versions. Means: The virtualization infrastructure made a lot of progress. Installing RHEL6 as a KVM guest works great. All drivers needed (virtio) are automatically installed.

A major good message to people which are using VMware vPhere 4 is that RHEL 6 comes which native support of vmxnet3 which was obviously backported from Kernel 2.6.33. Vmxnet3 is the driver for VMware’s para-virt NIC which brings quite some performance enhancements and lower CPU usage on the ESX host.

Certifications from ISVs

A quick check (not actually tested) for the requirements for SAP and Oracle shows that those are fulfilled already. We can expect the certification quite soon after GA of RHEL6. [update] Some compatibility RPMs from the mid 1990′s disappeared.  I now need to figure out if they are *really* needed by Oracle and/or SAP[/update]

Integration with Cobbler

Integration with cobbler works like expected, cobbler import –patch=/mnt –name=rhel6 and you are done. For a quick test I just copied the kickstart template from RHEL5 and I’m not sure if this is a good method. A test-install on ESX4 failed, the system hung at the creating of the root-VG. Not sure yet if it is a bug or something is incompatible in the kickstart. [update] The system hung was because of out-of-memory. The test-installation was on a ESX guest with 384Mbyte of memory which is enough according to the documentation but too little in real life. Growing the RAM of the test system to 512Mybte helped, but some packages needed by for SAP have changes names or disappeared.  After changing/removing those RPMs, the installation went smoothly[/update]

Bugs or features?

I detected some oddities where I’m not sure if it is a bug or a feature. We will see whats going on on http://bugzilla.redhat.com.

  • No network configured after default install. At the moment you need to configure it manually (considered a Bug)
  • I detected a major security issue during install, I’m not going to disclose it before a patch is available or more information from Red Hat is made available. I reported it 2010-04-23 ~12:00 on Red Hats bugzilla bugtracker. [update] The bug gots assigned to a Red Engineer after three hours, seems like Red Hat is acting very professional on the case[/update]

Conclusion

After this brief test one can say that RHEL6 will be a really great Linux Distribution for enterprise servers. The beta is already very stable with few bugs detected from my side. My guesstimate is that mid of May 2010 there will be a second public beta released, lets stay tuned, I’ll keep you up-to-date with further findings.

Have fun!

Roadmaps on the Red Hat Summit 2010 in Boston

Monday, March 15th, 2010

Finally Red Hat disclosed the agenda of its summit in 2010. For more informations see http://www.redhat.com/promo/summit/2010/agenda/.

RHEL6?

Tim Burke of Red Hat will talk about the new features of RHEL6. It sounds like the present, not the future. Does this mean I’m right with my guess that RHEL6 will be released end of June like I wrote in earlier blog article?

Roadmaps

Count how many times the word “Roadmap” appears in the agenda. It seems to get even more interesting what Red Hat plans to do. But it is still unsure what kind of new features we can expect in RHEL6. Red Hat just disclosed some snippets of RHEL6 again, this is called Salami-Tactic.

Where is the commitment?

We (the RHEL community) are still missing a clear commitment to us as customers. Only little is known about RHEL 6

Love or hate?

Should the RHEL community love or hate Red Hat? At the end of the day I like Red Hat, they do a lot for the progress of Linux in general and Linux in enterprises in particular. Anyway: Not providing a roadmap makes me and possibly others too very angry. Such a roadmap does not need to necessarily be in detail.

Have fun! Really? Soon we will have!

Kernel questions about RHEL6, ESX support and experiences with F13a3

Monday, March 15th, 2010

Still no official informations

Red Hat is still refusing any questions about the features of RHEL 6 and its Linux Kernel. However: Since Vanilla Kernel 2.6.33 vmxnet3 and pvscsi is supported. Fedora 13 Alpha 3 is shipped with a derivate of Kernel 2.6.33.

I still hope that Red Hat is switching to 2.6.33 or back-porting the VMWare code to its 2.6.32 derivative Kernel as known by RHEL 6 Alpha 3.

Experiences with F13a3 so far

Installing F13a3 on a ESX guest – with RHEL5 as “supported Guest OS”  – and enabled vmxnet “enhanced” plus pvscsci as HBA was a smooth experience. No driver disk was needed, no dirty fixes. Just selecting vmxnet3 as NIC and PVSCSCI as disk HBA. Thats the way RHEL6 should work from my point of view.

RHEV vs. VMWare ESX

Since Red Hat released its visualization solution “RHEV”, VMWare and Red Hat are competitors. Is Red Hat willing to include ESX support in its Enterprise Products? My guess is to not to do so, but I’m open for surprises.

The goals

The goal on the long term is to switch from ESX to KVM. However, if you deployed a large ESX farm already and the management members are members of the “ESX-Church” it will be hard.

The mid-term goal is to get rid of those crappy VMWare tools. The current state of this “Tools” definitively proves that VMWare is a Windows shop and  does not take care about Linux virtualization.

Will we have fun? Depends on EMC and Red Hat….

Where the heck is RHEL6?

Sunday, February 28th, 2010

Release cycle slowed down

In the past Red Hat has released a new version of its Red Hat Enterprise Linux (RHEL) roughly every two years. RHEL5 was released on march 2007. Compared to the past release cycle, RHEL6 is overdue since one year.

Official information

There is only little known about the upcoming features of RHEL6. On the Red Hat Summit 2009, there was a presentation held by Tim Burke which gives just some hints that RHEL6 is actually approaching, see http://www.redhat.com/f/pdf/summit/tburke_1050_rhel_roadmap.pdf. Quoting a note on the slide about RHEL6: Note: this information is high level planning projection and does not constitute formal product commitment.

My conclusion is that Red Hat seems to be unsure about the features planned for its upcoming Enterprise Product.

Another interesting quote from the same presentation is: RHEL6 feature previews – appearing in Fedora 11 & 12. Meanwhile, almost a half year later, Fedora 13 is approaching and still no sign of RHEL6, no schedule, no official feature list. Looking at the feature list if Fedora 13 https://fedoraproject.org/wiki/Releases/13/FeatureList, nothing special so far. It seems that the pace of development has been slowed down a bit to put more energy into stabilizing F11/F12 to RHEL6.

Inofficial information

When carefully watching git commit logs and bugzilla entries, there are some small traces of RHEL6.

There is almost no information leaking for the topic. The only valuable unofficial information is from bug #562766 which was reported by a Red Hat employee on 2010-02-08.  This bug states RHEL6 Alpha3!  Quoting a comment from the same employee: Upgrading rhel6.0 kernel to 2.6.32-14.el6 fixes the issue.

this brings me to a wild guess for a release schedule:

  • February 2010: Alpha3
  • March 2010: Beta1
  • April 2010: Beta2
  • May or June 2010: GA [Update: End of June/Early July seems to be more likely, since the Red Hat Summit will be held June 22-25 2010]

My wish list for RHEL6

  • Kernel based on version 2.6.33 instead of 2.6.32 as in Alpha3, since there are a lot of improvements when using RHEL as a VMware ESX guest.
  • Default installation with a smaller footprint
  • Cleanup of insane package dependencies
  • BusLogic drivers included as the vanilla Kernel ships it since years

The question remains

Where the heck is RHEL6? One reason could be that the focus on RHEL6 seems to be virtualization and system management. Since approximately two years, in this domain the pace of the development had increased a lot, maybe too much. Think about KVM, libvirt, virt-manager, o-virt. All of those projects are sponsored by Red Hat and included in F12. So one of the reason of the late release of RHEL6 can be problems in stabilizing those virtualization products to be enterprise-ready.

Why Red Hat makes its customers angry with late releases and no roadmap

First of all, RHEL products have a life-cycle of seven years. RHEL5 was released on march 2007. Assuming RHEL6 will be GA on May 2010. Add a few months before it is supported by ISVs such as SAP, Oracle etc. Customers can begin with deploying RHEL6 on lets say August 2010. Until then, RHEL5 has almost reached half of its life-cycle: 3 1/2 years. Means: A SAP system deployed on July 2010 is out of support some 3 years and nine months later. For an enterprise product this not acceptable! Red Hat should think about a life-cycle like “Next-Release plus five years“, this would make system deployment and company-internal life-cycle management easier.

Keeping its customers in the dark with no official roadmap at all is just bad behavior and indeed not customer friendly.