Configure SSSD to work on IPv6-only Hosts

SSSD is used for the client side of IPA and other centralized Identity Management Services. Unfortunately it does not behave as it should. The default is to look up first IPv4 addresses and if that fails IPv6 should be used. Well, if IPv4 fails, the whole request fails and you got weird error messages when joining an IPA domain.

As the pool for IPv4 addresses is depleted, IPv6 is getting more and more important. Thus, IPv6-only hosts are on the rise.

Here is an example error message from the IPA client.

[root@ipv6host ~]# ipa-client-install
[output ommited] 
SSSD enabled
Configured /etc/openldap/ldap.conf
Unable to find 'admin' user with 'getent passwd admin@example.com'!
Unable to reliably detect configuration. Check NSS setup manually.
[output ommited]

The host itself gets properly joined to the IPA domain and authentication works with Kerberos but you can not log in because SSSD fails.

Workaround

Configure SSSD to only use IPv6. This is done in /etc/sssd/sssd.conf

[domain/example.com]
lookup_family_order = ipv6_only
cache_credentials = True
krb5_store_password_if_offline = True
ipa_domain = example.com
id_provider = ipa
auth_provider = ipa
access_provider = ipa
ipa_hostname = ipv6host.example.com
chpass_provider = ipa
ipa_server = _srv_, ipa1.example.com
ldap_tls_cacert = /etc/ipa/ca.crt
[sssd]
services = nss, sudo, pam, ssh

domains = example.com
[nss]
homedir_substring = /home

[pam]

[sudo]

[autofs]

[ssh]

[pac]

[ifp]

Solution

At the moment there is no solution yet (just the workaround described), but its addressed at the SSSD project team, as you can see in https://pagure.io/SSSD/sssd/issue/2128 and https://bugzilla.redhat.com/show_bug.cgi?id=1021435

Happy IPv6-ing 🙂

Secure your system with SELinux

SELinux Logo

SELinux Logo

Introduction to SELinux

SELinux is well known as the most sophisticated Linux Mandatory Access Control (MAC) System. If you install any Fedora or Redhat operating System it is enabled by default and running in enforcing mode. So far so good.

Its available for many years and its not rocket science to use it. This article is supposed to give you some hints how to make your system even more secure and how to solve some troubles SELinux may have on your system.

DAC vs. MAC

Linux and traditional Unix systems are using DAC (Discretionary Access Control). Every user can change access rights to its own files. SELinux is a MAC (Mandatory Access Control) System where access rights are ruled by system wide policies. This can cause confusion when access is denied to a resource. Be aware that DAC will kick in before SELinux policies do. So if access to a resource is denied, please check access rights first. In such a case you will not see any AVC denials in your logs. The return code (EACCES) is the same.

RTFM

There is plenty of information available in the man pages. Some of the configuration file examples also contains additional information.

server:~# man -k selinux

Gives a good overview

Stick to Standards

Sofware installed from a RHEL or Fedora repository is usually not a problem at all, as long as you are using standards for config files, data, ports etc. Stick to the standards wherever possible. I.e. It does not make any sense to store websites in /opt instead of /var/www/html

Standards do not work for you?

If you can not stick to the standards for whatever reason, you can adjust a lot of settings with semanage.

Adding an allowed TCP Port for Apache

If you want to run your Apache httpd on port 8010, Apache will not start and a SELinux AVC denial is filed. To check which ports are allowed for Apache run:

server:~# semanage port -l|grep http_port_t
http_port_t                    tcp      80, 81, 443, 488, 8008, 8009, 8443, 9000
server:~# 

There is nothing like 8010

You can simply add port 8010 to the allowed ports by running

server:~# semanage port -a -t http_port_t 8010 -p tcp

Check again:

server:~# semanage port -l|grep http_port_t
http_port_t                    tcp      8010, 80, 81, 443, 488, 8008, 8009, 8443, 9000

VoilĂ !

Using a non-standard location for HTML files

Lets assume you want to store your HTML files in /opt/srv. To do so, you need to change the file context of that path and restore the file context afterwards.

server:~# semanage fcontext -a -t httpd_sys_rw_content_t '/opt/srv(/.*)?'
server:~# restorecon -R -v /opt/srv

Make use of Boolean variables

There are plenty of bool variables which simple allows to turn on or off a particular protection.

To get a list of defined bools, run

server:~# getsebool -a

You may want to pipe it to less or grep for a search pattern.

As an example, the default behavior is that a web application running in the httpd_t context will not be allowed to send emails. That helps greatly to prevent a vulnerable web application to send out SPAM. Well, if you want to operate a web mail service Apache must be able to send emails. No big deal:

server:~# setsebool -P httpd_can_sendmail on

Troubleshooting

The are some CLI (and GUI) tools available to troubleshoot AVC denials. The most important is sealert. Here is an example of an AVC because of a mislabled file in /var/www/html

sealert -a /var/log/audit/audit.log
SELinux is preventing /usr/sbin/httpd from getattr access on the file /var/www/html/test.html
*****  Plugin restorecon (99.5 confidence) suggests   ************************
If you want to fix the label. 
/var/www/html/test.html default label should be
httpd_sys_content_t.
Then you can run restorecon.
Do
# /sbin/restorecon -v /var/www/html/test.html

As you can see, sealert already provides you a hint how to fix the problem. In more complex cases, audit2why and audit2allow will help you. You simply grep for the misbehaving process:

server:~# grep httpd /var/log/audit/audit.log |audit2allow -m my_local_module

Review the result to check if it makes sense (ensure your grep statement does not catch too much). If you’re confident its okay, run the same command again with a capital M as parameter. It will create you a Local Policy Module which can be inserted:

server:~# grep httpd /var/log/audit/audit.log |audit2allow -M my_local_module
server:~# semodule -i my_local_module.pp

Temporary mitigation of SELinux troubles

If sealert and audit2allow can not immediately solve your problems and you quickly need to get your service up and running again, temporary put your system in permissive mode.

server:~# setenforce permissive

It will stay in pemissive mode until you reboot your system.

Permissive mode does not enforce the SELinux policies, it just logs AVC denials and help you to solve the problems without any service interruption. Be aware: This is a temporary quick fix, not a solution.

Put the affected domain only into permissive mode

If all your investigation did not help, all answers from support did not helped (very unlikely) you can put a particular domain into permissive mode. The rest of the policies are still in enforcing mode, your system still have some protection.

As an example, you can put the Apache module into permissive mode:

server:~# semanage permissive -a http_t

Hardening your System

Most people are not aware of the fact that when a system is in enforcing mode a malicious user with root access can manipulate policies or put SELinux into permissive mode.

There is a method to prevent this: Lock down your system

server:~# setsebool -P secure_mode_policyload on

Be aware: Once active nothing can not be changed during runtime, you need to reboot your system and provide selinux=1 enforcing=0 as grub boot parameter to be able to change any SELinux settings.

Have some fun!

Download “The SELinux Coloring Book” and learn more 🙂

Further reading

Have fun 🙂

Updating Fedora to version 23 – how to workaround some issues

After upgrading two machines from Fedora 22 to 23 I stumbled upon some severe issues. Most of them are easy to solve.

This weekend I’ve found some time to upgrade my headless router and one of my workstations. Unfortunately is did not went that smooth like the past few upgrades.

No initrd created and grub config lacks initrd reference
This seems to be connected to the Plymouth issue as described here: Common F23 bugs. On my headless machine I only had “details” and “text” themes installed, the result is that the machine can not access the root fs and the Kernel panics.

Solution: before upgrading, ensure you have the Plymouth theme “charge” active.

plymouth-set-default-theme charge && dracut -f && reboot

If you already upgraded and the machine fails to boot, select the Fedora 22 Kernel to boot from, create a new initrd and update the grub config as follows:

dracut /boot/initramfs-4.2.5-300.fc23.$(uname -i).img 4.2.5-300.fc23.$(uname -i)
grub2-mkconfig -o /boot/grub2/grub.cfg
reboot

Renamed network interfaces
On one machine I ended up having no network connectivity at all because both interfaces got a new name. I.e. “p135p1” was now known as “enp2s0”. If you do not use NetworkManager, just rename your ifcfg scripts in /etc/sysconfig/network-scripts/ and edit it them accordingly (DEVICE=new-interface-name). No clue how to manage that issue with NetworkManager, probably in a similar way.

No keyboard and mouse available
After the upgrading the machine I’m using as a workstation, keyboard and mouse have not been working anymore which makes the usage as a Workstation “a bit” problematic. The reason is a missing package due to a broken dependency.

dnf -y install  xorg-x11-drv-evdev
# Restart X11
systemctl isolate multi-user.target
systemctl isolate graphical.target

See also Bugzilla #1212833

KDM does not show any user
Since I do not have any local users (I’m using IPA for centralized identity management), its unclear if this is reason. I was not able to enter a username manually. I was also unable to track down the problem.

Solution: Switch to GDM:

system-switch-displaymanager GDM
# Restart X11
systemctl isolate multi-user.target
systemctl isolate graphical.target

KDE Plasma 5 garbled graphics
When I logged in to Plasma, the desktop was quite odd. All graphic stuff (the whole desktop) was garbled. No clue why, in a virtual machine this is working (somehow). Can not be the Nouveau driver because then also XFCE and Gnome would be affected as well.

Solution: Switch to XFCE (or Gnome if you like)

Conclusion
While the major part of the upgrade went really smooth, there are some issues which I did not expected to see. From my point of view the Plymouth issue and the keyboard/mouse issue should have been a blocker for the release. The rest of the issues I’ve stumbled on was probably just bad luck.

A word to KDE Plasma: Its available in the KDE spin since Fedora 22 but it is still not yet usable for daily work. I personally consider Plasma be a pre-alpha software. KDE repeats the same mistake they made when switching from KDE3 to KDE4. This will cause more users to switch away from KDE which is a petty.

Have fun 🙂

Using OTP Tokens and 2FA with FreeIPA 4.0

On 2014-07-08 FreeIPA 4.0 was released. One of the most interesting new features is the support of two factor authentication (2FA). I was curious about how to set it up and get it running. Unfortunately the documentation does not tell much about the OTP setup.

What is OTP and 2FA? An overview
OTP stands for One Time Password and 2FA for two factor authentication. OTP is available since long time, in the beginning usually as a list of passwords printed on paper. It was enhancing security gradually but was an operational nightmare.

RSA then came up with harware tokens somewhere in the 1990this which made it much more usable. Also 2FA was introduced. the two factors are ownership (or possession) and knowledge. One needs to obtain a piece of hardware (Hardware Token or a smart phone with a software token) and knowledge (knowing the password).

Meanwhile a lot of competing tokens are on the market, as well as so called soft-tokens. Most (or all?) of the hardware tokens are proprietary, making system configuration a nightmare (RSA PAM modules and stuff). On the other hand, every proprietary solution comes with the support of Radius. There is a quite new definition of using a Radius proxy to use those tokens with Kerberos and connect them with IPA.

However, hardware tokens and Radius proxies have been out of scope for my initial test. Lets go for the simpler soft token way.

Installing FreeIPA 4.0
It is planed to include FreeIPA 4.0 in Fedora 21 which will be released later this year. For testing you can either use Fedora Rawhide 21 or Fedora 20 with an external Yum repository. I was choosing the later way.

wget https://copr.fedoraproject.org/coprs/pviktori/freeipa/repo/fedora-20-i386/pviktori-freeipa-fedora-20-i386.repo -O /etc/yum.repos.d/pviktori-freeipa-fedora-20-i386.repo

The rest of the installation is the same as with (Free)IPA2 and (Free)IPA3. Please have a look at my earlier Post

Enabling OTP
You can either enable OTP on a global scope or per user. At the moment I recommend it on a per-user base.

ipa user-mod username --user-auth-type=otp

If you want to enable users to authenticate with more than one method, user –user-auth-type={otp,password}

Adding a new user with OTP enabled will probably be possible in the future. There seems to be a bug, according to ipa user-add –help it is supposed to be working.

ipa user-add hwurst --first="Hans" --last="Wurst" --user-auth-type=otp

Adding a token
The best way for a user to add a token is probably the web interface. Lets call it self-service. The user first authenticates with username and the initial password set by the admin to set a new one. The OTP field can be ignored for the moment.

After authentication, the user can navigate to “OTP Tokens” on the top navigation bar and add a new token. This looks as following:

ipa-otpThe ID needs to be unique, this can case problems when users are adding the tokens by themself as people would tend to provide a simple ID by themself. When not providing an ID, one will be generated. The field Unique ID should IMHO not be available for ordinary users.

After adding the token, login via password only is not possible anymore (unless explicitly enabled with the user-auth-type).

After hitting “Add”, a QR code will be shown. This allows users to scan the code with the Smartphone app, such as FreeOTP and Google Authenticator.

The next step users needs to do is to sync the token. This can be done by returning to the login screen and clicking on “Sync OTP Token” right left to the Login button.

ipa-otp2With a generated Unique ID (=Token ID) its quite annoying to enter that ID. However, usually this only needs to be one once 🙂

 

 

 

 

Limitations

The release notes mentions that there are concerns about the scalability when using HOTP, where TOTP has a known issue that tokens can be reused, but only within a short timeframe.

I see another issue which is a kind of a chicken-and-egg problem: After adding a user, this user is able to login with its password only until a token has been added. This ability is needed to log in to the IPA WebUI to add the token at the first place. However, password-only access should be limited to the token add facility.

Conclusion

I’m pretty amazed how well it works as this is a brand new feature for FreeIPA. The involved engineers made a brilliant job! I’m looking forward to see this feature in Redhat IPA/IdM somewhere in the future as 2FA is an often requested killer feature in enterprise environments.

Read more

Have fun! 🙂