Archive for the ‘Fedora’ Category

Updating Fedora to version 23 – how to workaround some issues

Sunday, November 8th, 2015

After upgrading two machines from Fedora 22 to 23 I stumbled upon some severe issues. Most of them are easy to solve.

This weekend I’ve found some time to upgrade my headless router and one of my workstations. Unfortunately is did not went that smooth like the past few upgrades.

No initrd created and grub config lacks initrd reference
This seems to be connected to the Plymouth issue as described here: Common F23 bugs. On my headless machine I only had “details” and “text” themes installed, the result is that the machine can not access the root fs and the Kernel panics.

Solution: before upgrading, ensure you have the Plymouth theme “charge” active.

plymouth-set-default-theme charge && dracut -f && reboot

If you already upgraded and the machine fails to boot, select the Fedora 22 Kernel to boot from, create a new initrd and update the grub config as follows:

dracut /boot/initramfs-4.2.5-300.fc23.$(uname -i).img 4.2.5-300.fc23.$(uname -i)
grub2-mkconfig -o /boot/grub2/grub.cfg

Renamed network interfaces
On one machine I ended up having no network connectivity at all because both interfaces got a new name. I.e. “p135p1” was now known as “enp2s0”. If you do not use NetworkManager, just rename your ifcfg scripts in /etc/sysconfig/network-scripts/ and edit it them accordingly (DEVICE=new-interface-name). No clue how to manage that issue with NetworkManager, probably in a similar way.

No keyboard and mouse available
After the upgrading the machine I’m using as a workstation, keyboard and mouse have not been working anymore which makes the usage as a Workstation “a bit” problematic. The reason is a missing package due to a broken dependency.

dnf -y install  xorg-x11-drv-evdev
# Restart X11
systemctl isolate
systemctl isolate

See also Bugzilla #1212833

KDM does not show any user
Since I do not have any local users (I’m using IPA for centralized identity management), its unclear if this is reason. I was not able to enter a username manually. I was also unable to track down the problem.

Solution: Switch to GDM:

system-switch-displaymanager GDM
# Restart X11
systemctl isolate
systemctl isolate

KDE Plasma 5 garbled graphics
When I logged in to Plasma, the desktop was quite odd. All graphic stuff (the whole desktop) was garbled. No clue why, in a virtual machine this is working (somehow). Can not be the Nouveau driver because then also XFCE and Gnome would be affected as well.

Solution: Switch to XFCE (or Gnome if you like)

While the major part of the upgrade went really smooth, there are some issues which I did not expected to see. From my point of view the Plymouth issue and the keyboard/mouse issue should have been a blocker for the release. The rest of the issues I’ve stumbled on was probably just bad luck.

A word to KDE Plasma: Its available in the KDE spin since Fedora 22 but it is still not yet usable for daily work. I personally consider Plasma be a pre-alpha software. KDE repeats the same mistake they made when switching from KDE3 to KDE4. This will cause more users to switch away from KDE which is a petty.

Have fun :-)

2FA with (Free) IPA. The good, the bad and the ugly

Thursday, April 9th, 2015

Two factor authentication (2FA) is more and more emerging which is good to enhance security. Since the release of IPA4 it comes with 2FA included.

Over time I made a lot of experiments and experience I wanted to share with you. Its is easy to set up and maintain as long as you use it only for system authentication. If you are using such things as webmail, it fails. This post shows you the capabilities as they are of today. Almost all bad issues apply not only to Fee(IPA) but 2FA in general.

The good
All your systems are Fedora 21, RHEL 7.1 or Ubuntu 14.02 all is working fine as the included SSSD is new enough to handle 2FA. All kerberized services can be used with 2FA w/o logging in again during the validity of your Kerberos ticket. Very convenient, very secure.

3rd Party applications can use LDAP authentication (Depending on the usecase)

The bad
Systems with older distributions such as RHEL6.6 come with a SSSD version which is to outdated to handle kerberized 2FA at all. This will probably change soon.


  • Use LDAP authentication (See later on)
  • Use a Jump host with a recent Linux distribution

If you are logging in to your workstation with a local user, you can not grab a Kerberos ticket with kinit and use this ticket further on. (i.e for ssh logins on remote server, mail etc.)


  • Switch to a IPA managed user if your workstation is recent enough.
  • Use a Jump host with a recent Linux distribution
  • Wait until krb5-PAKE is in place, software is being developed, see and
    • The ugly

      Looks like most mobile applications such as the IMAP client in Android do not prompt for the password, they expect it configured. Needless to say that you can not reconfigure the password each time you want to check your emails with your phone.


      • 3rd party email app? One that prompts for the password if needed
      • Configure IPA to accepts password and 2FA which lets the user choose to either use the password only or 2FA. Needless to say that this makes 2FA less useful as people tend to be lazy
      • Turn off 2FA in IPA and use a Yubikey with a static password (spit password). This is not a real 2FA it is a single password split in two. Password change is a horror.
      • Accessing Webmail clients (I tested roundcube mail) causes headaches as well. They authenticate the users with IMAP and use this credentials to access the mail storage. As the second factor is a one time password (OTP) this will result in failure to retrieve mails after logging in.

        Workaround: Same as for mobile applications. I would appreciate if someone can point me to a webmail software which can handle this.

        Offline usage

        One sentence: Offline usage does not work because it can not work.


        • Create a local user and use a Yubikey and configure it with a static password (split password). This is not a real 2FA it is a single password split in two. Password change is a horror.
        • Install a IPA server on your Notebook 😉 This will scale up to 18 Notebooks (plus two replicas in the datacenter) but introduce a lot of other problems, so: Not seriously to be considered.

        LDAP Authentication as a Workaround
        Configure PAM/SSSD to use LDAP authentication for your users. IPA comes with a very nice feature called ipa-advise.

        [root@ipa1 ~]# ipa-advise config-redhat-nss-pam-ldapd
        # ----------------------------------------------------------------------
        # Instructions for configuring a system with nss-pam-ldapd as a IPA
        # client. This set of instructions is targeted for platforms that
        # include the authconfig utility, which are all Red Hat based platforms.
        # ----------------------------------------------------------------------
        # Schema Compatibility plugin has not been configured on this server. To
        # configure it, run "ipa-adtrust-install --enable-compat"
        # Install required packages via yum
        yum install -y wget openssl nss-pam-ldapd pam_ldap authconfig
        # NOTE: IPA certificate uses the SHA-256 hash function. SHA-256 was
        # introduced in RHEL5.2. Therefore, clients older than RHEL5.2 will not
        # be able to interoperate with IPA server 3.x.
        # Please note that this script assumes /etc/openldap/cacerts as the
        # default CA certificate location. If this value is different on your
        # system the script needs to be modified accordingly.
        # Download the CA certificate of the IPA server
        mkdir -p -m 755 /etc/openldap/cacerts
        wget -O /etc/openldap/cacerts/ipa.crt
        # Generate hashes for the openldap library
        command -v cacertdir_rehash
        if [ $? -ne 0 ] ; then
         wget "" -O cacertdir_rehash ;
         chmod 755 ./cacertdir_rehash ;
         ./cacertdir_rehash /etc/openldap/cacerts/ ;
         cacertdir_rehash /etc/openldap/cacerts/ ;
        # Use the authconfig to configure nsswitch.conf and the PAM stack
        authconfig --updateall --enableldap --enableldapauth --ldapserver=ldap:// --ldapbasedn=cn=compat,dc=example,dc=com
        [root@ipa1 ~]#

        The output actually reflects your environment, will be replaced with your domain, its copy-paste ready. I love this feature :-) For other Linux systems, run ipa-advise without parameters to see which advises are available.

        2FA works well, convenient and secure in a datacenter and office environment. Notebooks are fine as well as long as there is a network connection available. The mobile world (Smartphones and Tablets) is not yet ready for 2FA. Some issues can be worked around (with some drawbacks) while others render 2FA not usable at all (offline usage).

        Hopefully there will be some smart solutions available for mobile usage soon, as mobile usage causes the most of the security headaches.

Migrating legacy servers to FreeIPA authentication using ID-views

Monday, April 6th, 2015

ID-Views are a new feature of FreeIPA4 which allows you to map UID/GID user/group names to another. This is a very handy solution when migrating legacy servers.

There are legacy servers in the field with a lot of history. They have been migrated from one operating system to another since the last decade(s). It is unfortunately also not uncommon on those legacy servers to find software with hardcoded UID/GID and/or user/group names. Along with an unknown number of scripts installed on such servers, its always problematic to migrate such systems when it comes to users and authentication. Another issue is that in the early years it was very common to have regular users with UID of >=500 while it is >=1000 as of today.

Unfortunately, almost nobody has the time to clean up the mess. Here is solution: ID-views. ID-Views can be applied to single hosts or group of hosts.

At the moment ID-Views are only working with newer SSSD versions as it is available with RHEL 7.1.

Creating a view

[root@ipa1 ~]# ipa idview-add --desc "Old servers with legacy users" oldservers
Added ID View "oldservers"
  ID View Name: oldservers
  Description: Old servers with legacy users
[root@ipa1 ~]# 

Override a group

[root@ipa1 ~]# ipa idoverridegroup-add --desc "Old group" --gid=500 --group-name=users oldservers users
Added Group ID override "users"
  Anchor to override: users
  Description: Old group
  Group name: users
  GID: 500
[root@ipa1 ~]#

Override a user
If you ommit the --login parameter (or any other) then the value in question is not overridden. Ususally you just override the numeric UID and/or GID.

[root@ipa1 ~]# ipa idoverrideuser-add --desc="John Doe is actually Hans Tester" --login=jdoe --uid=500 --gidnumber=500 --homedir=/home/jdoe --shell=/bin/csh oldservers tester
Added User ID override "tester"
  Anchor to override: tester
  Description: John Doe is actually Hans Tester
  User login: jdoe
  UID: 500
  GID: 500
  Home directory: /home/jdoe
  Login shell: /bin/csh
[root@ipa1 ~]# 

Apply the ID-View to a server

[root@ipa1 ~]# ipa idview-apply oldservers
Applied ID View "oldservers"
Number of hosts the ID View was applied to: 1
[root@ipa1 ~]# 

To enable the view on the client side, clean the SSSD cache and restart the sssd service. Login to

[root@legacy ~]# sss_cache -E
[root@legacy ~]# systemctl restart sssd

You also need to change the PAM configuration to accept logins with UID &lt1000.

Now do some tests. Both users, “jdoe” and “tester” have UID 500.

[root@legacy ~]# getent passwd jdoe
jdoe:*:500:500:Hans Tester:/home/jdoe:/bin/csh
[root@legacy ~]# getent passwd tester
jdoe:*:500:500:Hans Tester:/home/jdoe:/bin/csh
[root@legacy ~]# 

On other servers, the “jdoe” login is unknown, and “tester” has the normal UID assigned by IPA

[root@ipa1 ~]# getent passwd jdoe
[root@ipa1 ~]# echo $?
[root@ipa1 ~]# getent passwd tester
tester:*:1225800004:1225800004:Hans Tester:/home/tester:/bin/bash
[root@ipa1 ~]# 

Please keep in mind that not cleaning up a messy system is just a workaround :-)

Using IPA to provide automount maps for NFSv4 home directories

Saturday, March 14th, 2015

Since the invention of NFSv4, automount NFS home directories is secure. Since the invention of IPA, its easier to set up and maintain. This article guides you trough the steps needed to set it up. The procedures have been tested on RHEL7.1 for the IPA servers, RHEL6.6 and 7.1 as clients but should work on Fedora and CentOS. Unfortunately it seems not to work (yet) for Debian Sid and Ununtu. [Update] Works in Ubuntu 14.04[/Update]


  • Your Domain is
  • Your Kerberos Realm is EXAMPLE.COM
  • The NFS server is
  • The exported home directories are on /exports/home
  • The client is
  • A few words about security and kerbrized NFS
    There are basically three different modes: krb5, krb5i and krb5p.

    • krb5 means that the server and client authenticate each other, traffic can be intercepted.
    • krb5i the same as krb5 but providing integrity. It verifies that the data has not been tampered with, but traffic still can be intercepted.
    • krb5p like the two above, plus privacy protection, all traffic is encrypted.

    Depending on the sensitivity of the data to be transferred krb5i or krb5p should be used. Also keep in mind that the higher the security the lower the throughput is.

    Work to do on one of the IPA replicas

    Add the NFS service principal for the server and client to Kerberos.

    [root@ipa1 ~]# ipa service-add nfs/
    [root@ipa1 ~]# ipa service-add nfs/

    Assume you are only using one location, you can use the default one.

    Add the auto.home map

    [root@ipa1 ~]# ipa automountmap-add default auto.home
    Added automount map "auto.home"
      Map: auto.home
    [root@ipa1 ~]# 

    And add the auto.home map to auto.master

    [root@ipa1 ~]# ipa automountkey-add default --key "/home" --info auto.home auto.master
    Added automount key "/home"
      Key: /home
      Mount information: auto.home
    [root@ipa1 ~]# 

    Finally add the key to the auto.home map

    [root@ipa1 ~]# ipa automountkey-add default --key "*" --info "-fstype=nfs4,rw,sec=krb5,soft,rsize=8192,wsize=8192" auto.home
    Added automount key "*"
      Key: *
      Mount information: -fstype=nfs4,rw,sec=krb5i,soft,rsize=8192,wsize=8192
    [root@ipa1 ~]# 

    Configure the NFS server
    Create a Kerberos Keytab for your NFS server

    [root@nfs ~]# kinit admin
    [root@nfs ~]# ipa-getkeytab -s -p nfs/ -k /etc/krb5.keytab

    Tell your NFS service to use NFSv4

    [root@nfs ~]# perl -npe 's/#SECURE_NFS="yes"/SECURE_NFS="yes"/g' -i /etc/sysconfig/nfs

    Create your NFS share and start the NFS server

    [root@nfs ~]# mkdir /exports/home
    [root@nfs ~]# echo "/exports/home  *(rw,sec=sys:krb5:krb5i:krb5p)" >> /etc/exports
    [root@nfs ~]# service nfs start
    [root@nfs ~]# chkconfig nfs on

    Configure your clients

    Get the Kerberos keytab

    [root@ipaclient1 ~]# ipa-getkeytab -s -p nfs/ -k /etc/krb5.keytab

    Finally you need to configure your client systems to map use of the automount maps provided by IPA

    [root@login ~]# ipa-client-automount --location=default
    Searching for IPA server...
    IPA server: DNS discovery
    Location: default
    Continue to configure the system with these values? [no]: yes
    Configured /etc/nsswitch.conf
    Configured /etc/sysconfig/nfs
    Configured /etc/idmapd.conf
    Started rpcidmapd
    Started rpcgssd
    Restarting sssd, waiting for it to become available.
    Started autofs
    [root@login ~]# 

    Strange problems you can run into

    If you run into troubles, enable debugging in the related daemons. In /etc/sysconfig/autofs, add a line LOGGING=debug.
    Add debug_level = 9 in the [autofs] stanza.

    If you have something like this in /var/log/messages

    lookup(file): failed to read included master map auto.master

    Then probably your nsswitch.conf does not point to sss. Ensure you have

    automount:  files sss

    In your nsswitch.conf. This should actually be configured by ipa-client-automount but it seems that it is not 100% reliable to do so.

    If you have something like this in /var/log/messages:

    Mar 14 20:02:37 ipaclient nfsidmap[3039]: nss_getpwnam: name '' does not map into domain 'localdomain'

    Then check your /etc/hosts file if all is correct. Also ensure that the short hostname is not in front of the FQHN. Another mistake can trigger the same error: DNS. Ensure you have a working DNS setup for both A (and/or AAAA) and PTR records.

    Read further
    There are plenty of docs available, there is a choice

    Have fun! :-)

hostapd can not find the wlan interface but interface is ready

Wednesday, February 25th, 2015

Have you ever got an error when using hostapd complaining a network interface not be found but its actually there and ready? You probably have a space at the end of the line “interface”. Hostapd does not work when having a space in that line (and probably in other lines as well) in /etc/hostapd/hostapd.conf.

ap:/etc/hostapd# cat -vet hostapd.conf|grep ^interface
interface=wlp0s29f7u7 $

This shows that there is a space at the lines end. Remove it and it will work as expected. Remove it and it works. If it does not work you probably have an error in another layer of your WLAN setup.