Upgrading RHN Satellite from 5.3 to 5.4, experiences and hints

As I wrote in my previous post I’ll let you know about my experiences. The most important message is: It is easy to upgrade your Satellite from 5.3 to 5.4, if you have an eye on certain things. Everyone that plans to use RHEL6 and manage it with a Satellite server needs this upgrade, due to the fact that RHEL6 comes with SHA-256 checksums on its RPM packages.

Who needs this upgrade?
Any company that plans to use RHEL6 which is released when quoting Red Hat: “later this year”. Any company with more than 50 managed systems which is annoyed by BZ #629543. Every company with more than 500 managed systems to able to manage 501+ systems because of the same bug.

The odd thing
RHN Satellite 5.4 was released at the end of October 2010. There no press release, no announcement, nothing (yet). This upgrade is probably one of the most important ones, because of RHEL6. @Red Hat: Please explain.

Before you begin
Obtain the new Satellite certificate from Red Hat. Open a support case at rhn.redhat.com. It takes about two days. Provide them the version of the new Satellite to avoid further loss of time.

Do I need to say that you need to download the ISO-image from rhn.redhat.com?

If your Satellite has SELinux enabled and it is in enforcing mode, you need to put it into permissive mode by issuing setenforce 0. At the moment there is are two open bugs regading SELinux: BZ #646863 and BZ #646862.

The protocol for syncing has changed from version 5.3 to 5.4. You need to keep this in mind when using ISS (Inter Satellite Sync). At the moment a 5.3 Satellite cannot be the master of a 5.4 Satellite. The other way round it works perfectly. A bug was filed, read BZ #644239 for more information, a fix will be released quite soon.

It is important to backup /etc/rhn/rhn.conf since this file will get overwritten. A Bugzilla issue is filed (BZ #650987).

If you did not backed up your database, do it now!

Another thing you need to know is that Red Hats recommendations on extending the Oracle embedded DB’s table space is not enough. Please ensure you have at least 2 Gbyte on DATA_TBS and 1GByte of UNDO_TBS. To be sure of that, fire up db-control report as user “oracle”. If one of the TBS’ does not have the expected free space, fire up db-control extend you-name-it-TBS.

Further, you need to have a closer look to the file system size – depending on on your file system layout – to /var/cache/rhn. I made some odd observations that the space needed has quite exploded from about 2Gbyte to about 6Gybte. So ensure you’re having some free space on /var resp. /var/cache/rhn.

If you using cobbler and kickstarting for provisioning or monitoring, please install the package rhn-upgrade and have a look to the files installed in /etc/sysconfig/rhn/satellite-upgrade/ for more information about the procedure.

First you need to delete some stuff…
Before upgrading your Satellite you need to clean your caches located in /var/cache/rhn. It will be rebuilt later.

… and rebuild it from scratch later
The different tasks you need to conduct will take, depending on how many custom channels you have, between four and 14 hours. More to read further below, after the upgrade process is finished.

Lets start

• Mount the ISO-image downloaded: mount satellite-embedded-oracle-5.4.0-20101025-rhel-5-x86_64.iso /mnt -o loop && cd /mnt.
• Fire up the installer: ./install.pl –upgrade It will ask you some questions, anwer then with “Y” for yes.
• Fire up su – oracle -c db-control gather-stats
• Update your database scheme with spacewalk-schema-upgrade and depending on the hour, take a break for a long lunch/dinner or go home and continue the following day.
• When finished, check if the schema was applied successfully by issuing rhn-schema-version. It should read as 5.4.0.8-1el5sat.
• Activate your freshly upgraded Satellite: rhn-satellite-activate –rhn-cert /path/to/the/cert/you/got/from/redhat.
• You would like to rebuild the search indexes: service rhn-search cleanindex
• Check /etc/rhn/rhn.conf and compare it to the previously backed up version. Change it accordingly.
• Restart your satellite: rhn-satellite restart and have another break.

Rebuild your satsync cache
As written further up, you need to rebuild some more meta data caches: /var/cache/rhn/repodata will be rebuilt when restarting the Satellite. The /var/cache/rhn/satsync will we rebuilt on the first satellite-sync. Keep in mind that satellite-sync still does not remember previously synced custom channels. Please vote on this bug.

Cool stuff to do with RHN Satellite 5.4
Who is using EPEL? IUS? someone? I think a lot of people do so. Until now, the best method was probably wget -m and rhn-push all RPMs in the output directory. This was time consuming and created some extra traffic on the net. Now you can add yum repositories to your satellite and link them to a custom channel. Log in and go to Channels -> Manage Software Channels -> Manage Repositories -> create new repository.

Add a label, i.e. “epel5-x86_64” and add the repository URL. I.e. http://download.fedora.redhat.com/pub/epel/5/x86_64/. Save and go back to “Manage Software Channels” and select a channel, or create a new one. Base channel is mostly a Red Hat Channel. Go to “Repositories” and select the formerly created repository. Click on “Update Repositories”, Click in “sync” and confirm by clicking the “Sync” button and you’re done.

What else? Staging content sounds nice. Unfortunately this only works with the upcoming RHEL5.6 (Beta was announced today) and RHEL 6.1 (Why not RHEL 6.0?). It means that if enabled, every enabled system downloads the to-be-updated packages before the actual maintenance window. This greatly helps keeping downtimes short.

At the end of the day…
Companies using RHN Satellite are strongly encouraged to upgrade. Not only because of the support for RHEL6, there also have been a lot of bugfixes, performance improvements and enhancements. I can encourage every Satellite user to upgrade, Red Hat (and in particular the Spacewalk developer team) made a great job, thanks a lot!

Having fun?
Sure, Red Hat Satellite 5.4 makes my daily work more efficient 🙂