Upgrading Redhat Satellite 5.7 to 5.8

Couple of days ago, Redhat released its latest and last major upgrade for Satellite 5.x. Its a rather important upgrade, you are advised to upgrade soon.

This upgrade contains some major improvements like stated in an earlier article

Disclaimer

I’m not responsible for any damage caused by the procedure provided here. Always create a backup before even thinking about upgrading a Satellite server.

Preparation

As always when you plan to upgrade your Satellite server to the latest version, you need to do some preparations first.

Ensure you will have enough disk space free in /var/opt/rh. The upgrade to 5.8 will also install a new PostgreSQL version located at /var/opt/rh/rh-postgresql95/lib/pgsql. The new version will roughly use the same diskspace as the old version 9.2 in /opt/rh/postgresql92/root/var/lib/pgsql.

Download the ISO

Visit https://access.redhat.com/downloads/content/250/ver=5.8/rhel—6/5.8/x86_64/product-software and make sure you select 5.8 and the architecture fitting you system (x86_64 or S390)

Get a new Satellite Certificate Manifest

Satellite 5.8 switches from Certificates to Manifests like Satellite 6. You need a Manifest to get it activated. You can create it by your own at the Subscription Management Application site, ensure you attach enough subscriptions to your Satellite server(s).

Backup

Usually an upgrade runs smooth, but just in case… it is recommended practice to have a recent backup ready. If your Satellite is running on a virtual machine, power off, snapshot and power on to have a consistent backup ready. For physical systems, db-control and the choice.

Backup the rest of your Satellite:

Create a copy of your rhn configuration directory as we need some information from the old files after the upgrade.

rhnsat:~# cp -rp /etc/rhn/ /etc/rhn-$(date +"%F")

Update your OS and Satellite 5.7

First step is to update the operating system and the Satellite 5.7 and apply the latest database schema updates as well.

rhnsat:~# yum -y update && reboot

Update the database schema if needed

To update the database schema, run the following command. Ideally it looks as follows:

rhnsat:~# spacewalk-schema-upgrade 
Schema upgrade: [satellite-schema-5.7.0.27-1.el6sat] -> [satellite-schema-5.7.0.27-1.el6sat]
Your database schema already matches the schema package version [satellite-schema-5.7.0.27-1.el6sat].
rhnsat:~# 

Switch from RHN to Subscription Manager

It is important to ensure you switched from RHN to subscription manager before doing the upgrade. You can check if this is the case with:

rhnsat:~# subscription-manager repos --list-enabled
+----------------------------------------------------------+
    Available Repositories in /etc/yum.repos.d/redhat.repo
+----------------------------------------------------------+
Repo ID:   rhel-6-server-rpms
Repo Name: Red Hat Enterprise Linux 6 Server (RPMs)
Repo URL:  https://cdn.redhat.com/content/dist/rhel/server/6/$releasever/$basearch/os
Enabled:   1

Repo ID:   rhel-6-server-satellite-5.7-rpms
Repo Name: Red Hat Satellite 5.7 (for RHEL 6 Server) (RPMs)
Repo URL:  https://cdn.redhat.com/content/dist/rhel/server/6/$releasever/$basearch/satellite/5.7/os
Enabled:   1

rhnsat:~# 

If this is not yet done, have a look at the knowledge base article located here: https://access.redhat.com/articles/2884191

Functionality Check with the old version 5.7

It is recommended to restart and check a software functionality before upgrading to be able to pinpoint problems if there are some.

rhnsat:~# rhn-satellite restart

Cleanup

Review the software channels in use and delete unused channels as this can free up quite some disk space and reduces the size of the database significantly.

rhnsat:~# spacewalk-remove-channel -c rhel-i386-rhev-agent-6-server
Deleting package metadata (20):
                  ________________________________________
Removing:         ######################################## - complete
rhnsat:~# 

Delete old system snapshots which are not used anymore. The following example deletes all snapshots which are older than one month:

rhnsat:~#  sw-system-snapshot --delete --all --start-date 200001010000 --end-date $(date -d "-1 months" "+%Y%m%d0000") 

Check for old MD5 user passwords and certificates

Check if there are still some users with an md5 hashed password. The same applies to certificates.

rhnsat:~# spacewalk-report users-md5
rhnsat:~# spacewalk-report system-md5-certificates

If there are any, please have a look to https://access.redhat.com/documentation/en-us/red_hat_satellite/5.8/html/installation_guide/ch10s03

RTFM

If not done yet, install or update the rhn-upgrade package which contains the instructions how to proceed.

rhnsat:~# yum -y install rhn-upgrade

The package contains not only SQL- and other useful scripts needed for the upgrade but also important documents to read. The are located in /etc/sysconfig/rhn/satellite-upgrade/doc.

For most users, the document satellite-upgrade-postgresql.txt applies.

Do not forget to read the updated product documentation as well:

Performing the upgrade

rhnsat:~# mount satellite-5.8-rhel-6-x86_64-dvd.iso /mnt  -o loop
rhnsat:~#  cd /mnt
rhnsat:/mnt# ./install.pl --upgrade
* Starting Red Hat Satellite installer.
* Performing pre-install checks.
* Pre-install checks complete.  Beginning installation.
* RHSM Registration.
** Registration: System is already registered with RHSM.  Not re-registering.
* RHSM Subscriptions.
** Subscriptions: Subscription providing 'Red Hat Satellite' already attached.
** Subscriptions: Subscription providing 'Red Hat Enterprise Linux Server' already attached.
** Subscriptions: Disabling all RHSM repositories (rhel-6-server-rpms, rhel-6-server-satellite-5.7-rpms).
** Subscriptions: All repositories disabled.
** Subscriptions: Enabling RHEL repository.
** Subscriptions: RHEL repository enabled.
* Upgrade flag passed.  Stopping necessary services.
* Purging conflicting packages.
* Checking for uninstalled prerequisites.
** Checking if yum is available ...
There are some packages from Red Hat Enterprise Linux that are not part
of the @base group that Satellite will require to be installed on this
system. The installer will try resolve the dependencies automatically.
However, you may want to install these prerequisites manually.
Do you want the installer to resolve dependencies [y/N]? y
* Installing Satellite packages.
Warning: more packages were installed by yum than expected:
        python-backports
        python-backports-ssl_match_hostname
        python-chardet
        python-requests
        python-urllib3
* Now running spacewalk-setup.
* Setting up SELinux..
** Database: Setting up database connection for PostgreSQL backend.
*** Upgrading embedded database.
** Database: Populating database.
** Database: Skipping database population.
* Configuring tomcat.
* Setting up users and groups.
** GPG: Initializing GPG and importing key.
* Performing initial configuration.
* Activating Red Hat Satellite.
** Manifest not activated.
** Upgrade process requires the manifest to be activated after the schema is upgraded.
* Configuring apache SSL virtual host.
Should setup configure apache's default ssl server for you (saves original ssl.conf) [Y]? 
* Configuring jabberd.
* Creating SSL certificates.
** Skipping SSL certificate generation.
* Deploying configuration files.
* Update configuration in database.
* Setting up Cobbler..
Cobbler requires tftp and xinetd services be turned on for PXE provisioning functionality. Enable these services [Y]? 
This portion of the Red Hat Satellite upgrade process has successfully completed.
Please refer to appropriate upgrade document in /etc/sysconfig/rhn/satellite-upgrade
for any remaining steps in the process.
rhnsat:/mnt# 

Active and Updating the Satellite

Since the ISO image is always a bit outdated, you need to activate and update the Satellite after its installation.

Upgrading the Database schema

rhnsat:~# rhn-satellite stop
rhnsat:~# /etc/init.d/rh-postgresql95-postgresql start
rhnsat:~# spacewalk-schema-upgrade
rhnsat:~# rhn-satellite-activate --manifest=/root/manifest.zip --ignore-version-mismatch

Some more work to do

After the upgrade succeed there is some work work to do.

Initial Sync with CDN

Unfortunately cdn-sync does not inherit the history which channels have been synced previously with satellite-sync. You need to once sync each channel again. Only missing data will be downloaded.

rhnsat:~# for i in $(spacecmd -u admin -p secret -q softwarechannel_listbasechannels); do cdn-sync -c $i; done
rhnsat:~# for i in $(spacecmd -u admin -p secret -q softwarechannel_listchildchannels); do cdn-sync -c $i; done

If you have custom channels, this will produce errors as the custom channels are not available in CDN. Just ignore them.

Rebuild the search index

rhnsat:~# service rhn-search cleanindex

Have fun 🙂

New features in Satellite 5.8

Redhat Satellite 5 was released in version 5.8. based on Spacewalk 2.5. It will probably be the last upgrade available, Support ends in January 2019.

New features and enhancements

  • The major new feature is the introduction of support for the CDN for both, Satellite activation and content sync. The key benefit is a massively enhanced performance for content sync. It’s now called cdn-sync, not satellite-sync anymore. Be aware that some custom scripts as well as cronjobs must be updated as well.This change also introduces the usage of Satellite Manifests instead of the old Certificates
  • Introduction of the new CLI tool taskotop which allows you to watch the activity of the Taskomatic Daemon.
  • PostgreSQL is upgraded to 9.5 which brings, compared to 9.2, some performance improvements as well.
  • More Perl bits have been rewritten to Java
  • Java JRE is now IBM’s Version 1.8
  • A few new commands in the spacecmd CLI
  • Lots of bugfixes and small enhancements

Removed features

There are some features that have been dropped with this release.

  • Support for Patch management of Solaris Systems. Who was using that? I can not remember that I’ve seen a company using that feature.
  • Monitoring is gone as well, I only know one organization that have used that feature. Most companies are using Icinga or Nagios.

Usange of cdn-sync

Populate Repository Metadata

The listing of available channels is working off-line. To be able to see the number of packages assigned to which channels you need to download the repository metadata first.

[root@sat58 ~]# cdn-sync --count-packages
14:05:25 Number of channels: 1271
14:05:25 Number of repositories: 1456
Downloading repomd:   |##################################################| 100.0% 
Comparing repomd:     |##################################################| 100.0% 
Downloading metadata: |##################################################| 100.0% 
Counting packages:    |##################################################| 100.0% 
14:42:21 Total time: 0:36:56
[root@sat58 ~]# 

Be aware that this will take a while, depending on how many entitlements are defined in the Satellite Manifest.

To keep that data up to date you should add a cronjob to do so.

[root@sat58 ~]# echo '0 1 * * * perl -le "sleep rand 9000" && /usr/bin/cdn-sync --count-packages' >> /etc/cron.d/cdn-sync-populate-metadata

Initial content sync

Similar to the old satellity-sync you provide the parameter -c repeatedly for all repos to be synced.

[root@sat58 ~]# cdn-sync -c rhel-x86_64-server-7              
11:16:20 ======================================
11:16:20 | Channel: rhel-x86_64-server-7
11:16:20 ======================================
11:16:20 Sync of channel started.
11:16:20 Repo URL: https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/os
11:16:28 Packages in repo:             14275
11:16:41 Packages already synced:          0
11:16:41 Packages to sync:             14275
11:16:42 New packages to download:     14275
11:16:43 1/14275 : 389-ds-base-1.3.5.10-20.el7_3.x86_64.rpm
11:16:43 2/14275 : 389-ds-base-1.3.4.0-26.el7_2.x86_64.rpm
11:16:43 3/14275 : 389-ds-base-1.3.5.10-11.el7.x86_64.rpm
11:16:43 4/14275 : 389-ds-base-1.3.3.1-16.el7_1.x86_64.rpm
[.. output ommited ..]
11:57:03 14275/14275 : zsh-5.0.2-14.el7_2.2.x86_64.rpm
Importing packages:     |##################################################| 100.0% 
13:10:05 Linking packages to channel.
13:10:19 Repo https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/os has comps file 730c62cc7600c7518e4920f800cb9af6b73d75ba-comps.xml.
13:10:20 Repo https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/os has 1885 errata.
13:10:50 Kickstartable tree not detected (no valid treeinfo file)
13:10:50 Repo URL: https://cdn.redhat.com/content/dist/rhel/server/7/7.3/x86_64/kickstart
13:10:55 Packages in repo:              4751
13:12:32 No new packages to sync.
13:12:32 Linking packages to channel.
13:12:44 Repo https://cdn.redhat.com/content/dist/rhel/server/7/7.3/x86_64/kickstart has comps file c542e4cf37dd210de68877b53f41d92dc7686c6e1b35ca4b1852f2e62fca2c72-comps-Server.x86_64.xml.gz.
13:12:44 Repo https://cdn.redhat.com/content/dist/rhel/server/7/7.3/x86_64/kickstart has 0 errata.
13:12:44 Added new kickstartable tree ks-rhel-x86_64-server-7-7.3. Downloading content...
13:12:44 Gathering all files in kickstart repository...
Downloading kickstarts: |##################################################| 100.0%
[.. output ommited ..]
13:24:53 Sync of channel completed in 2:08:33.
13:24:54 Total time: 2:08:33
[root@sat58 ~]# 

A subsequent run of cdn-sync without any parameters behaves like satellite-sync, its syncing previously synced channels.

You probably want to schedule a cronjob for daily syncing new content

[root@sat58b ~]# echo '0 1 * * * perl -le "sleep rand 9000" && /usr/bin/cdn-sync' >> /etc/cron.d/cdn-sync

The output of the sync actions are logged to /var/log/rhn/cdnsync.log

Clearing the cache

Remember rm -rf /var/cache/rhn/satsync/* when something went wrong? That’s gone :-). You just use cdn-sync –clear-cache.

cdn-sync --clear-cache

Upgrading from Satellite 5.7

I’ve not found the time yet to test the upgrade, I’ll let you know about my experience and thoughts in a few days.

Conclusion

After approximately 15 years, old school Redhat Satellite 5 will finally be replaced with Satellite 6 which is built on base of completely different technologies such as The Foreman, Pulp, Katello etc.

Satellite 5.8 is a very mature release no major bugs are known.

Satellite users are encouraged to discover Satellite 6 now, to be ready for the transition to be made in 2020.

Have fun 🙂

Upgrading RHN Satellite 5.6 to 5.7

This post guides you trough the upgrade procedure for a Satellite 5.6 using the embedded database on RHEL6-x86_64. Further it guides you to setup of Kerberos authentication of Satellite users with IPA.

Recently Redhat released Satellite Server 5.7. Despite Satellite 5.x will be outphased in the next few years, there are plenty of new features. The most significant new features are:

  • Upgraded PostgreSQL to 9.2
  • Authentication via IPA/SSSD/Kerberos
  • IPMI support
  • Renewed WebUI
  • Readonly API users

And finally… drum roll…. formal support for spacecmd 🙂

As always when you plan to upgrade your Satellite server to the latest version, you need to do some preparations first.

Download the ISO
As usual, visit the Download site and make sure you select 5.7 and the architecture fitting you system (x86_64 or S390)

Get a new Satellite Certificate
Satellite 5.7 needs a new certificate to get it activated. You can create it by your own at the Subscription Management Application site, ensure you attach enough subscriptions to your Satellite server(s). Alternatively open a support case.

Backup
Usually an upgrade runs smooth, but just in case… it is recommended practice to have a recent backup ready. If your Satellite is running on a virtual machine, power off, snapshot and power on to have a consistent backup ready. For physical systems, db-control and the choice of your backup software need to be visited.

Backup the rest of your Satellite:

Create a copy of your rhn configuration directory as we need some information from the old files after the upgrade.

[root@rhnsat ~]# cp -rp /etc/rhn/ /etc/rhn-$(date +"%F")

Update your OS and Satellite
First step is to update the operating system and the Satellite 5.6 and apply the latest database schema updates as well.

yum -y update && reboot

To update the database schema, run the following command. Ideally it looks as follows:

root@rhnsat ~]# spacewalk-schema-upgrade 

You are about to perform upgrade of your satellite-schema.

For general instructions on Red Hat Satellite schema upgrade, please consult
the following article:

    https://access.redhat.com/knowledge/articles/273633

Hit Enter to continue or Ctrl+C to interrupt: 
Schema upgrade: [satellite-schema-5.6.0.21-1.el6sat] -> [satellite-schema-5.6.0.21-1.el6sat]
Your database schema already matches the schema package version [satellite-schema-5.6.0.21-1.el6sat].
[root@rhnsat ~]# 

Functionality Check
It is recommended to restart and check a softwares functionality before upgrading to be able to pinpoint problems if there are some.

[root@rhnsat ~]# rhn-satellite restart

Cleanup
Its a good idea to review the software channels in use and delete unused channels as this can free up quite some diskspace and reduces the size of the database significantly.

[root@rhnsat ~]# spacewalk-remove-channel -c rhel-i386-rhev-agent-6-server
Deleting package metadata (20):
                  ________________________________________
Removing:         ######################################## - complete
[root@rhnsat ~]#

Delete old system snapshots not used anymore. The following example deletes all snapshots which are older than one month:

[root@rhnsat ~]# sw-system-snapshot --delete --all --start-date 200001010000 --end-date $(date -d "-1 months" "+%Y%m%d0000" 

Remove spacecmd from EPEL
Most Satellite users have spacecmd installed from EPEL. Its a good idea to remove it to avoid conflicts. It is also important to disable the EPEL repositories on Satellite servers as a simple yum update can bring your Satellite server into trouble.

RTFM
If not done yet, install the rhn-upgrade package which contains the instructions how to proceed.

yum -y install rhn-upgrade

The package contains not only SQL- and other useful scripts needed for the upgrade but also important documents to read. The are located in /etc/sysconfig/rhn/satellite-upgrade/doc.

For most users, the document satellite-upgrade-postgresql.txt applies.

Do not forget to read the updated product documentation as well:

Changing your file system layout
As there will be an updated PostgreSQL version needed which is part of the Software Collection and not installable from the base channel, you need to add a new file system in /opt/rh.
The new database is about the same size as before. Check your used disk space at /var/lib/pgsql

[root@rhnsat ~]# lvcreate /dev/vg_data -n lv_opt_rh -L 17G 
[root@rhnsat ~]# mkfs.ext4 /dev/vg_data/lv_opt_rh
[root@rhnsat ~]# tune2fs -c0 -i0  /dev/vg_data/lv_opt_rh

Exit your /etc/fstab accordingly and mount the file system with mount -a to check if it working as expected.

Lets do it
Mount the ISO image and run the installer.

[root@rhnsat ~]# mount satellite-5.7.0-20150108-rhel-6-x86_64.iso /mnt -o loop
[root@rhnsat ~]# cd /mnt
[root@rhnsat mnt]# 

If you are using a proxy to sync your satellite, provide the --diconnected flag.

[root@rhnsat mnt]# ./install.pl --upgrade --disconnected
* Starting Red Hat Satellite installer.
* Performing pre-install checks.
* Pre-install checks complete.  Beginning installation.
* RHN Registration.
** Registration: Disconnected mode.  Not registering with RHN.
* Upgrade flag passed.  Stopping necessary services.
* Purging conflicting packages.
* Checking for uninstalled prerequisites.
** Checking if yum is available ...
There are some packages from Red Hat Enterprise Linux that are not part
of the @base group that Satellite will require to be installed on this
system. The installer will try resolve the dependencies automatically.
However, you may want to install these prerequisites manually.
Do you want the installer to resolve dependencies [y/N]? y
* Installing RHN packages.
* Now running spacewalk-setup.
* Setting up SELinux..
** Database: Setting up database connection for PostgreSQL backend.
*** Upgrading embedded database.
** Database: Populating database.
** Database: Skipping database population.
* Setting up users and groups.
** GPG: Initializing GPG and importing key.
* Performing initial configuration.
* Activating Red Hat Satellite.
** Certificate not activated.
** Upgrade process requires the certificate to be activated after the schema is upgraded.
* Enabling Monitoring.
* Configuring apache SSL virtual host.
Should setup configure apache's default ssl server for you (saves original ssl.conf) [Y]? y
* Configuring tomcat.
* Configuring jabberd.
* Creating SSL certificates.
** Skipping SSL certificate generation.
* Deploying configuration files.
* Update configuration in database.
* Setting up Cobbler..
task started: 2015-02-08_154708_sync
task started (id=Sync, time=Sun Feb  8 15:47:08 2015)
running pre-sync triggers
cleaning trees
removing: /var/www/cobbler/images/ks-rhel-x86_64-es-4-u6
[..]
running shell triggers from /var/lib/cobbler/triggers/change/*
*** TASK COMPLETE ***
Cobbler requires tftp and xinetd services be turned on for PXE provisioning functionality. Enable these services [Y]? y
This portion of the Red Hat Satellite upgrade process has successfully completed.
Please refer to appropriate upgrade document in /etc/sysconfig/rhn/satellite-upgrade
for any remaining steps in the process.
[root@rhnsat mnt]#

The next step is having a look at diff /etc/rhn/rhn.conf /etc/rhn-$(date +”%F”)/rhn.conf
and edit /etc/rhn/rhn.conf accordingly. You will probably see missing things such as proxy, server.satellite.rhn_parent etc. Also change the setting disconnected to 0.

Upgrade database scheme
Before doing anything, first update the database schema to 5.7.

[root@rhnsat ~]# spacewalk-schema-upgrade

Activate the updated Satellite server
To subscribe the Satellite server to the appropriate software channels, it must be activated. Since it was activated before, the --ignore-version-mismatch parameter must be provided.

[root@rhnsat ~]# rhn-satellite-activate --rhn-cert=rhn-satellite57-2015-02-08.xml --ignore-version-mismatch

Initial Update of Software and database schema
There is a good chance that there are updates available for the Satellite Server as the ISO image will not be updated that often.

[root@rhnsat ~]# yum -y update

Even if no update was installed, there is a schema update available:

[root@rhnsat ~]# spacewalk-schema-upgrade 
Schema upgrade: [satellite-schema-5.6.0.21-1.el6sat] -> [satellite-schema-5.7.0.11-1.el6sat]
Searching for upgrade path: [satellite-schema-5.6.0.21-1] -> [satellite-schema-5.7.0.11-1]
Searching for upgrade path: [satellite-schema-5.6.0.21] -> [satellite-schema-5.7.0.11]
Searching for upgrade path: [satellite-schema-5.6.0] -> [satellite-schema-5.7.0]
Searching for upgrade path: [satellite-schema-5.6] -> [satellite-schema-5.7]
The path: [satellite-schema-5.6] -> [satellite-schema-5.7]
Planning to run spacewalk-sql with [/var/log/spacewalk/schema-upgrade/20150208-155657-script.sql]

Plase make sure you have a valid backup of your database before continuing.

Hit Enter to continue or Ctrl+C to interrupt: 
Executing spacewalk-sql, the log is in [/var/log/spacewalk/schema-upgrade/20150208-155657-to-satellite-schema-5.7.log].
The database schema was upgraded to version [satellite-schema-5.7.0.11-1.el6sat].
[root@rhnsat ~]# 

Cleanup
After startarting the Satellite Server, the package meta data should be automatically recreated. If not, run
/etc/sysconfig/rhn/satellite-upgrade/scriptsregenerate-repodata manually.

Rebuild the search index:

[root@rhnsat ~]# service rhn-search cleanindex

You don’t need to remove the old PostgreSQL version, this is done automatically.

Using IPA and Kerberos for authentication
Before configure the Satellite Server to use IPA, make sure it is enrolled and the HTTP service principal exists. If not, add it with the following command:

[root@ipa1 ~]# ipa-addservice HTTP/rhnsat.example.com@EXAMPLE.COM

Next will be getting a Kerbros Ticket of a user allowed to create Keytabs. In this example it is the user admin.

[root@rhnsat ~]# kinit admin
Password for admin@EXAMPLE.COM: 
[root@rhnsat ~]# 

Afterwards, run the setup script:

[root@rhnsat ~]# spacewalk-setup-ipa-authentication
Enabling authentication against [ipa2.example.com].
Retrieving HTTP/ service keytab into [/etc/httpd/conf/http.keytab] ...
Keytab successfully retrieved and stored in: /etc/httpd/conf/http.keytab
changed ownership of `/etc/httpd/conf/http.keytab' to apache
Configuring PAM service [spacewalk].
Will install additional packages ...
Loaded plugins: product-id, rhnplugin, security, subscription-manager
This system is receiving updates from RHN Classic or RHN Satellite.
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package mod_auth_kerb.x86_64 0:5.4-13.el6 will be installed
---> Package mod_authnz_pam.x86_64 0:0.9.2-1.el6 will be installed
---> Package mod_intercept_form_submit.x86_64 0:0.9.7-1.el6 will be installed
---> Package mod_lookup_identity.x86_64 0:0.9.2-1.el6 will be installed
---> Package sssd-dbus.x86_64 0:1.11.6-30.el6_6.3 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

============================================================================================================================================================
 Package                                      Arch                      Version                               Repository                               Size
============================================================================================================================================================
Installing:
 mod_auth_kerb                                x86_64                    5.4-13.el6                            rhel-x86_64-server-6                     30 k
 mod_authnz_pam                               x86_64                    0.9.2-1.el6                           rhel-x86_64-server-6                     13 k
 mod_intercept_form_submit                    x86_64                    0.9.7-1.el6                           rhel-x86_64-server-6                     17 k
 mod_lookup_identity                          x86_64                    0.9.2-1.el6                           rhel-x86_64-server-6                     19 k
 sssd-dbus                                    x86_64                    1.11.6-30.el6_6.3                     rhel-x86_64-server-6                    122 k

Transaction Summary
============================================================================================================================================================
Install       5 Package(s)

Total download size: 201 k
Installed size: 0  
Downloading Packages:
(1/5): mod_auth_kerb-5.4-13.el6.x86_64.rpm                                                                                           |  30 kB     00:00     
(2/5): mod_authnz_pam-0.9.2-1.el6.x86_64.rpm                                                                                         |  13 kB     00:00     
(3/5): mod_intercept_form_submit-0.9.7-1.el6.x86_64.rpm                                                                              |  17 kB     00:00     
(4/5): mod_lookup_identity-0.9.2-1.el6.x86_64.rpm                                                                                    |  19 kB     00:00     
(5/5): sssd-dbus-1.11.6-30.el6_6.3.x86_64.rpm                                                                                        | 122 kB     00:00     
------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                        41 kB/s | 201 kB     00:04     
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing : mod_authnz_pam-0.9.2-1.el6.x86_64                                                                                                        1/5 
  Installing : mod_intercept_form_submit-0.9.7-1.el6.x86_64                                                                                             2/5 
  Installing : mod_auth_kerb-5.4-13.el6.x86_64                                                                                                          3/5 
  Installing : mod_lookup_identity-0.9.2-1.el6.x86_64                                                                                                   4/5 
  Installing : sssd-dbus-1.11.6-30.el6_6.3.x86_64                                                                                                       5/5 
  Verifying  : mod_intercept_form_submit-0.9.7-1.el6.x86_64                                                                                             1/5 
  Verifying  : sssd-dbus-1.11.6-30.el6_6.3.x86_64                                                                                                       2/5 
  Verifying  : mod_lookup_identity-0.9.2-1.el6.x86_64                                                                                                   3/5 
  Verifying  : mod_authnz_pam-0.9.2-1.el6.x86_64                                                                                                        4/5 
  Verifying  : mod_auth_kerb-5.4-13.el6.x86_64                                                                                                          5/5 

Installed:
  mod_auth_kerb.x86_64 0:5.4-13.el6                  mod_authnz_pam.x86_64 0:0.9.2-1.el6            mod_intercept_form_submit.x86_64 0:0.9.7-1.el6          
  mod_lookup_identity.x86_64 0:0.9.2-1.el6           sssd-dbus.x86_64 0:1.11.6-30.el6_6.3          

Complete!
** /etc/sssd/sssd.conf has been backed up to sssd.conf-swsave
Updated sssd configuration.
Turning SELinux boolean [httpd_dbus_sssd] on ...
        ... done.
Turning SELinux boolean [allow_httpd_mod_auth_pam] on ...

        ... done.
Configuring Apache modules.
** /etc/tomcat6/server.xml has been backed up to server.xml-swsave.ipa
Stopping sssd:                                             [  OK  ]
Starting sssd:                                             [  OK  ]
Stopping tomcat6:                                          [  OK  ]
Starting tomcat6:                                          [  OK  ]
Stopping httpd:                                            [  OK  ]
Starting httpd:                                            [  OK  ]
Waiting for tomcat to be ready ...
Authentication against [ipa2.example.com] sucessfully enabled.
As admin, at Admin > Users > External Authentication, select
          Default organization to autopopulate new users into.
[root@rhnsat ~]# 

Next, point your browser to https://rhnsat.example.com/rhn/admin/multiorg/ExternalAuthentication.do to finalize the setup.

Configure your browser for Kerberos
If you did not yet configured your browser to use Kerberos authentication, do so. Assuming you are using an IPA invironment, follow the instructions provided on the IPA servers. http://ipa2.example.com/ipa/config/browserconfig.html

Disclaimer
I take no responsibility about damaged Satellites, lost of data etc. in doubt, stick on the official product documentation at http://access.redhat.com

Upgrading RHN Satellite 5.5 to 5.6

Redhat released version 5.6 of the Redhat Satellite. Time to have a closer look to it and how to upgrade from version 5.5.

New features

  • Finally PostgreSQL support is mature enough for Enterprise usage. No need of a closed source data base anymore. This also brings a lot of new capabilities such as online backups which before was only available using an external Oracle Database which needs the availability of a DBA.

    PostgreSQL also brings some performance benefits over the embedded Oracle database as delivered with 5.5 and earlier. Disclaimer: I did not made any benchmarks, but it “feels” much faster.

  • If you are using the multi-org feature, you may be happy about enhancements for Inter-Satellite-Sync (ISS). Now you can define access rights for different software channels for different organizations.
  • It is not a new feature, but now it is supported: cobbler buildiso. It is a handy solution if you can not use PXE boot in your environment. cobbler buildiso generates a small boot image which allows you to select the installation of a system from a boot menu.
  • Intergrated System Asset Manager (SAM) which is based on Candlepin and allows you assess your system landscape for subscription compliance.
  • Upgrading from RHN Satellite 5.5
    The first thing that you probably would ask: Is it possible and supported to migrate from the Embedded Oracle Database to PostgreSQL? Is it hassle free and bullet-proof? Yes it is.

    Keep in mind

  • As always: Have a look to the product documentation before doing anything on a production Satellite.
  • Create a new RHN Satellite Certificate at access.redhat.com
  • Download the ISO image for 5.6
  • ensure having a recent database backup
  • ensure having a recent backup of your /etc/rhn directory as well as /var/lib/cobbler
  • Update your existing Satellite 5.5 with the latest available patches
  • Delete unnecessary software channels from the Satellite for faster DB migration
  • Delete old Snapshots to minimize database data to be migrated
  • Make enough storage available to migrate from embedded Oracle to PostgreSQL. It takes roughly about the same amount of storage for the data. The PostgreSQL database stores its data in /var/lib/pgsql.
  • Install the latest available package rhn-upgrade: yum install rhn-upgrade

    Lets do it, Perparation work

    First of all, create a database backup of your embedded Oracle Database:

    [root@rhnsat ~]# rhn-satellite stop
    [root@rhnsat ~]# su - oracle -c "db-control backup /path/to/your/backup/directory"
    [root@rhnsat ~]# su - oracle -c "db-control verify /path/to/your/backup/directory"
    [root@rhnsat ~]# rhn-satellite start
    

    Backup the rest of your Satellite:

    [root@rhnsat ~]# cp -rp /etc/rhn/ /etc/rhn-$(date +"%F")
    [root@rhnsat ~]# cp -rp /var/lib/cobbler /var/lib/cobbler-$(date +"%F")
    [root@rhnsat ~]# cp -rp /etc/cobbler /etc/cobbler-$(date +"%F")
    

    Update your RHN Satellite 5.5 with the latest available patches and reboot:

    [root@rhnsat ~]# yum -y update && reboot
    

    Ensure the latest schema updates have been applied. The output should read as follow:

    [root@rhnsat ~]# spacewalk-schema-upgrade 
    
    You are about to perform upgrade of your satellite-schema.
    
    For general instructions on Red Hat Satellite schema upgrade, please consult
    the following article:
    
        https://access.redhat.com/knowledge/articles/273633
    
    Hit Enter to continue or Ctrl+C to interrupt: 
    Schema upgrade: [satellite-schema-5.6.0.10-1.el6sat] -> [satellite-schema-5.6.0.10-1.el6sat]
    Your database schema already matches the schema package version [satellite-schema-5.6.0.10-1.el6sat].
    [root@rhnsat ~]#
    

    It is always a good idea to restart a software and check if all is working as expected *before* doing an upgrade. So you can pinpoint problems better if there are some.

    [root@rhnsat ~]# rhn-satellite restart
    

    Review your list of software channels and delete unused ones. This example will delete the channel rhel-i386-rhev-agent-6-server:

    [root@rhnsat ~]# spacewalk-remove-channel -c rhel-i386-rhev-agent-6-server
    Deleting package metadata (20):
                      ________________________________________
    Removing:         ######################################## - complete
    [root@rhnsat ~]#  
    

    Delete old system snapshots not used anymore. The following example deletes all snapshots which are older than one month:

    [root@rhnsat ~]# sw-system-snapshot --delete --all --start-date 200001010000 --end-date $(date -d "-1 months" "+%Y%m%d0000")
    

    Update the rhn-update package to the latest available:

    yum install rhn-upgrade
    

    After installing the the rhn-upgrade package, the SQL scripts needed for the DB migration are installed as well as some documentation you should read. They are located in /etc/sysconfig/rhn/satellite-upgrade/doc.

    Upgrade Procedure

    Mount the downloaded ISO image:

    [root@rhnsat ~]# mount satellite-5.6.0-20130927-rhel-6-x86_64.iso /mnt -o loop && cd /mnt
    [root@rhnsat mnt]# 
    

    If you operate your Satellite behind a proxy, you need to upgrade it in disconnected mode, if not, ignore the –disconneded parameter.

    [root@rhnsat mnt]# ./install.pl --disconnected --upgrade
    * Starting the Spacewalk installer.
    * Performing pre-install checks.
    * Pre-install checks complete.  Beginning installation.
    * RHN Registration.
    ** Registration: Disconnected mode.  Not registering with RHN.
    * Upgrade flag passed.  Stopping necessary services.
    * Purging conflicting packages.
    * Checking for uninstalled prerequisites.
    ** Checking if yum is available ...
    There are some packages from Red Hat Enterprise Linux that are not part
    of the @base group that Satellite will require to be installed on this
    system. The installer will try resolve the dependencies automatically.
    However, you may want to install these prerequisites manually.
    Do you want the installer to resolve dependencies [y/N]? y
    * Installing RHN packages.
    * Now running spacewalk-setup.
    * Setting up Selinux..
    ** Database: Setting up database connection for PostgreSQL backend.
    ** Database: Installing the database:
    ** Database: This is a long process that is logged in:
    ** Database:   /var/log/rhn/install_db.log
    *** Progress: #
    ** Database: Installation complete.
    ** Database: Populating database.
    *** Progress: ###################################
    * Database: Starting Oracle to PostgreSQL database migration.
    ** Database: Starting embedded Oracle database.
    ** Database: Trying to connect to Oracle database: succeded.
    ** Database: Migrating data.
    *** Database: Migration process logged at: /var/log/rhn/rhn_db_migration.log
    ** Database: Data migration successfully completed.
    ** Database: Stoping embedded Oracle database.
    * Setting up users and groups.
    ** GPG: Initializing GPG and importing key.
    * Performing initial configuration.
    * Activating Red Hat Satellite.
    ** Certificate not activated.
    ** Upgrade process requires the certificate to be activated after the schema is upgraded.
    * Enabling Monitoring.
    * Configuring apache SSL virtual host.
    Should setup configure apache's default ssl server for you (saves original ssl.conf) [Y]? y
    * Configuring tomcat.
    ** /etc/sysconfig//tomcat6 has been backed up to tomcat6-swsave
    ** /etc/tomcat6//tomcat6.conf has been backed up to tomcat6.conf-swsave
    Reversed (or previously applied) patch detected!  Skipping patch.
    1 out of 1 hunk ignored -- saving rejects to file web.xml.rej
    * Configuring jabberd.
    * Creating SSL certificates.
    ** Skipping SSL certificate generation.
    * Deploying configuration files.
    * Update configuration in database.
    * Setting up Cobbler..
    cobblerd does not appear to be running/accessible
    Cobbler requires tftp and xinetd services be turned on for PXE provisioning functionality. Enable these services [Y]? 
    This portion of the Red Hat Satellite upgrade process has successfully completed.
    Please refer to appropriate upgrade document in /etc/sysconfig/rhn/satellite-upgrade
    for any remaining steps in the process.
    [root@rhnsat mnt]# 
    

    Depending on the size of your database and the speed of your disks, the upgrade procedure can take many hours.

    The next step is having a look at diff /etc/rhn/rhn.conf /etc/rhn-$(date +”%F”)/rhn.conf
    and edit /etc/rhn/rhn.conf accordingly. You will probably see missing things such as proxy, server.satellite.rhn_parent etc. Also change the setting disconnected to 0.

    After checking and correcting the config file you can activate the Satellite:

    [root@rhnsat ~]# rhn-satellite-activate --rhn-cert=/root/rhns-cert56.cert --ignore-version-mismatch
    

    After the activation the System is subscribed to the Softwarechannel “redhat-rhn-satellite-5.6-server-x86_64-6”, now bring the Satellite to latest available patchlevel:

    [root@rhnsat ~]# yum -y update 
    

    Stop and disable Oracle
    Bofore doing any Database related actions its better to stop the old Oracle Database to be sure all is now running on PostgreSQL.

    [root@rhnsat ~]# service oracle stop
    Shutting down Oracle Net Listener ...                      [  OK  ]
    Shutting down Oracle DB instance "rhnsat" ...              [  OK  ]
    [root@rhnsat ~]# chkconfig oracle off
    [root@rhnsat ~]# rhn-satellite restart
    

    Aftermath

    Check if your database schema is up-to-date:

    root@rhnsat ~]# spacewalk-schema-upgrade 
    
    You are about to perform upgrade of your satellite-schema.
    
    For general instructions on Red Hat Satellite schema upgrade, please consult
    the following article:
    
        https://access.redhat.com/knowledge/articles/273633
    
    Hit Enter to continue or Ctrl+C to interrupt: 
    Schema upgrade: [satellite-schema-5.6.0.10-1.el6sat] -> [satellite-schema-5.6.0.10-1.el6sat]
    Your database schema already matches the schema package version [satellite-schema-5.6.0.10-1.el6sat].
    [root@rhnsat ~]# 
    

    Rebuild the search index:

    [root@rhnsat ~]# service rhn-search cleanindex
    Stopping rhn-search...
    Stopped rhn-search.
    Starting rhn-search...
    [root@rhnsat ~]# 
    

    Recreate the software channel meta data:

    [root@rhnsat doc]# /etc/sysconfig/rhn/satellite-upgrade/scripts/regenerate-repodata -a
    Scheduling repodata creation for 'rhel-x86_64-server-supplementary-6'
    Scheduling repodata creation for 'rhel-x86_64-server-6'
    Scheduling repodata creation for 'rhn-tools-rhel-x86_64-server-6'
    [root@rhnsat doc]# 
    

    Check functionality
    Before removing the Oracle Database, run your tests to validate the Satellites functionality. Please proceed as stated in /etc/sysconfig/rhn/satellite-upgrade/doc/verification.txt

    This is an important point, as we are getting rid of the Oracle database later on. To be sure all is working as expected, do a complete functionality test for the important things.

    To be on the safe side, let the Satellite run for a few days with Oracle still installed.

    Getting rid of Oracle

    Please read /etc/sysconfig/rhn/satellite-upgrade/doc/satellite-upgrade-postgresql.txt first!

    [root@rhnsat ~]# yum remove *oracle*
    

    Getting rid of the last Oracle bits:

    [root@rhnsat ~]# rm -rf /rhnsat /opt/apps/oracle /usr/lib/oracle/
    

    Result:
    Having fun with a faster Satellite with an open source database 🙂

    Disclaimer
    I take no responsibility about damaged Satellites, lost data etc. in doubt, stick on the official product documentation at http://access.redhat.com