Using Unbound for recursive DNS lookup

Some organizations decide to use its internal authoritative DNS servers as recursive DNS because of easiness and reverse lookup of internal RFC 1918 networks works out of the box. That should be avoided for (at least) two reasons:

  • Cache poisoning can cause security nightmares
  • Authoritative answers are never cached and can cause a high load on the DNS servers.

Cache poisoning is a problem that can lead to severe problems, as more and more information is stored in DNS. Examples:

  • SSHFP entries for SSH fingerprint of servers
  • SRV entries for LDAP and Kerberos server autodetection

If an attacker can manipulate those kind of entries it can potentially be abused for redirecting users to fake authentication services.

There are some protective measures to avoid this kind of problems:

  • The usage of a separate recursive DNS infrastructure
  • Setting up DNSSEC and sign your DNS zones
  • The use of TLS for LDAP queries

This article is about how to set up recursive DNS servers, DNSSEC will be covered in a follow-up article.

Turning off recursion in authoritative DNS servers

In the option section of the bind DNS configuration make sure you have the following line in /etc/named.conf:

allow-recursion { none; };

If you are using a different DNS server software, check the vendor manual. After a restart, check if it is working as expected.

[luc@bond ~]$ dig www.example.com @ipa2.delouw.ch

; <<>> DiG 9.10.4-P6-RedHat-9.10.4-4.P6.fc25 <<>> www.example.com @ipa2.delouw.ch
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 58272
;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;www.example.com.               IN      A

;; Query time: 0 msec
;; SERVER: 192.168.100.106#53(192.168.100.106)
;; WHEN: Sat Apr 15 12:10:15 CEST 2017
;; MSG SIZE  rcvd: 44

[luc@bond ~]$ 

Using Unbound as recursive DNS

Unbound is very secure, lightweight and high performance DNS server for validating, recursion, and caching of queries. Its astonishing how easy it is to configure Unbound.

Installation on RHEL7, Fedora and probably other Linux and BSD distributions is easy:

recursor1:~# yum -y install unbound

For this example, all configuration is made in /etc/unbound/unbound.conf
First you must define on which IPs Unbound should listen. The default is localhost only.

interface: 0.0.0.0
interface: ::0

The next default that needs to be changed is the access control. Default to refuse all but localhost. In this example you will allow access from two of your RFC 1918 subnets and the RFC 3849 IPv6 range.

access-control: 0.0.0.0/0 refuse
access-control: 127.0.0.0/8 allow
access-control: 192.168.1.0/24 allow
access-control: 192.168.2.0/24 allow
access-control: ::1 allow
access-control: 2001:DB8::/32 allow

Forward PTR queries to your RFC 1918 zones

Unbound has a nice default setting: It ignores any queries to RFC 1918 PTR queries to avoid sending queries to the blackhole servers.

In this example, we need to change the behavior to allow queries for our internal networks 192.168.1.0 and 192.168.2.0.

local-zone: "1.168.192.in-addr.arpa." transparent
local-zone: "2.168.192.in-addr.arpa." transparent

Next up: Forward this queries to our internal DNS server infrastructure (i.e IPA or MS-DNS or simply bind)

forward-zone:
        name: "1.168.192.in-addr.arpa."
        forward-host: ipa1.example.com
        forward-host: ipa2.example.com
        forward-host: ipa3.example.com

forward-zone:
        name: "2.168.192.in-addr.arpa."
        forward-host: ipa1.example.com
        forward-host: ipa2.example.com
        forward-host: ipa3.example.com

This will forward queries at random to DNS servers ipa1,ipa2 and ipa3.example.com. Add more servers as needed.

The final step is to (re)configure your clients to use the newly set up recursive DNS servers.

Have fun 🙂

Disabling NetworkManager on Servers and Workstations

Why not using NetworkManager in some cases

NetworkManager is a great tool for managing connectivity on Notebooks and other mobile devices, On server or desktop machines with a complex network setup such as a combination of bonding, bridging and VLAN its probably not the best choice, at least I was not able to configure it that way. This was some time ago (approx 1y), meanwhile it may have changed.

Removing NetworkManager

Unfortunately on a desktop system its impossible to get rid of NetworkManager, there are too many really weird dependencies. On servers without a GUI it is very easy to uninstall it, IIRC no drawbacks so far.

To remove NetworkManager run

system:~# yum remove NetworkManager

Be careful, there can a a lot of dependencies getting uninstalled as well. Handle with care.

Solution w/o removing NetworkManager

Disabling the NetworkManager itself is easy,

system:~# systemctl stop NetworkManager
system:~# systemctl disable NetworkManager
system:~# systemctl mask NetworkManager

Unfortunately the NetworkManager-wait-online.service Systemd unit file can not be disabled, its enabled even when systemctl status says its disabled. At the end this means that the boot process will take 30 seconds longer than needed, that is the timeout defined for /usr/bin/nm-online.

You can check the boot process which step is to blame for the long boot time with systemd-analyze blame.

system:~# systemd-analyze blame|grep NetworkManager
          30.060s NetworkManager-wait-online.service
system:~# 

Changing the Systemd unit file

Never ever edit a systemd unit file in /usr/lib/systemd/system/ as they get overwritten with the next software update (in this case NetworkManager).

You can simply copy the unit file to the systemd local config directory /etc/systemd/system.

system:~# cp NetworkManager-wait-online.service /etc/systemd/system

You now replace the /usr/bin/nm-online with /usr/bin/true which always exits with 0.

system:~# sed -i "s|/usr/bin/nm-online -s -q --timeout=30|/usr/bin/true|g" /etc/systemd/system/NetworkManager-wait-online.service

Reload the Systemd daemon

system:~# systemctl daemon-reload 

Ensure the Symlink is correct

system:~# systemctl disable NetworkManager-wait-online.service
system:~# systemctl enable NetworkManager-wait-online.service

Further reading

Have fun 🙂

Configure SSSD to work on IPv6-only Hosts

SSSD is used for the client side of IPA and other centralized Identity Management Services. Unfortunately it does not behave as it should. The default is to look up first IPv4 addresses and if that fails IPv6 should be used. Well, if IPv4 fails, the whole request fails and you got weird error messages when joining an IPA domain.

As the pool for IPv4 addresses is depleted, IPv6 is getting more and more important. Thus, IPv6-only hosts are on the rise.

Here is an example error message from the IPA client.

[root@ipv6host ~]# ipa-client-install
[output ommited] 
SSSD enabled
Configured /etc/openldap/ldap.conf
Unable to find 'admin' user with 'getent passwd admin@example.com'!
Unable to reliably detect configuration. Check NSS setup manually.
[output ommited]

The host itself gets properly joined to the IPA domain and authentication works with Kerberos but you can not log in because SSSD fails.

Workaround

Configure SSSD to only use IPv6. This is done in /etc/sssd/sssd.conf

[domain/example.com]
lookup_family_order = ipv6_only
cache_credentials = True
krb5_store_password_if_offline = True
ipa_domain = example.com
id_provider = ipa
auth_provider = ipa
access_provider = ipa
ipa_hostname = ipv6host.example.com
chpass_provider = ipa
ipa_server = _srv_, ipa1.example.com
ldap_tls_cacert = /etc/ipa/ca.crt
[sssd]
services = nss, sudo, pam, ssh

domains = example.com
[nss]
homedir_substring = /home

[pam]

[sudo]

[autofs]

[ssh]

[pac]

[ifp]

Solution

At the moment there is no solution yet (just the workaround described), but its addressed at the SSSD project team, as you can see in https://pagure.io/SSSD/sssd/issue/2128 and https://bugzilla.redhat.com/show_bug.cgi?id=1021435

Happy IPv6-ing 🙂

Secure your system with SELinux

SELinux Logo

SELinux Logo

Introduction to SELinux

SELinux is well known as the most sophisticated Linux Mandatory Access Control (MAC) System. If you install any Fedora or Redhat operating System it is enabled by default and running in enforcing mode. So far so good.

Its available for many years and its not rocket science to use it. This article is supposed to give you some hints how to make your system even more secure and how to solve some troubles SELinux may have on your system.

DAC vs. MAC

Linux and traditional Unix systems are using DAC (Discretionary Access Control). Every user can change access rights to its own files. SELinux is a MAC (Mandatory Access Control) System where access rights are ruled by system wide policies. This can cause confusion when access is denied to a resource. Be aware that DAC will kick in before SELinux policies do. So if access to a resource is denied, please check access rights first. In such a case you will not see any AVC denials in your logs. The return code (EACCES) is the same.

RTFM

There is plenty of information available in the man pages. Some of the configuration file examples also contains additional information.

server:~# man -k selinux

Gives a good overview

Stick to Standards

Sofware installed from a RHEL or Fedora repository is usually not a problem at all, as long as you are using standards for config files, data, ports etc. Stick to the standards wherever possible. I.e. It does not make any sense to store websites in /opt instead of /var/www/html

Standards do not work for you?

If you can not stick to the standards for whatever reason, you can adjust a lot of settings with semanage.

Adding an allowed TCP Port for Apache

If you want to run your Apache httpd on port 8010, Apache will not start and a SELinux AVC denial is filed. To check which ports are allowed for Apache run:

server:~# semanage port -l|grep http_port_t
http_port_t                    tcp      80, 81, 443, 488, 8008, 8009, 8443, 9000
server:~# 

There is nothing like 8010

You can simply add port 8010 to the allowed ports by running

server:~# semanage port -a -t http_port_t 8010 -p tcp

Check again:

server:~# semanage port -l|grep http_port_t
http_port_t                    tcp      8010, 80, 81, 443, 488, 8008, 8009, 8443, 9000

VoilĂ !

Using a non-standard location for HTML files

Lets assume you want to store your HTML files in /opt/srv. To do so, you need to change the file context of that path and restore the file context afterwards.

server:~# semanage fcontext -a -t httpd_sys_rw_content_t '/opt/srv(/.*)?'
server:~# restorecon -R -v /opt/srv

Make use of Boolean variables

There are plenty of bool variables which simple allows to turn on or off a particular protection.

To get a list of defined bools, run

server:~# getsebool -a

You may want to pipe it to less or grep for a search pattern.

As an example, the default behavior is that a web application running in the httpd_t context will not be allowed to send emails. That helps greatly to prevent a vulnerable web application to send out SPAM. Well, if you want to operate a web mail service Apache must be able to send emails. No big deal:

server:~# setsebool -P httpd_can_sendmail on

Troubleshooting

The are some CLI (and GUI) tools available to troubleshoot AVC denials. The most important is sealert. Here is an example of an AVC because of a mislabled file in /var/www/html

sealert -a /var/log/audit/audit.log
SELinux is preventing /usr/sbin/httpd from getattr access on the file /var/www/html/test.html
*****  Plugin restorecon (99.5 confidence) suggests   ************************
If you want to fix the label. 
/var/www/html/test.html default label should be
httpd_sys_content_t.
Then you can run restorecon.
Do
# /sbin/restorecon -v /var/www/html/test.html

As you can see, sealert already provides you a hint how to fix the problem. In more complex cases, audit2why and audit2allow will help you. You simply grep for the misbehaving process:

server:~# grep httpd /var/log/audit/audit.log |audit2allow -m my_local_module

Review the result to check if it makes sense (ensure your grep statement does not catch too much). If you’re confident its okay, run the same command again with a capital M as parameter. It will create you a Local Policy Module which can be inserted:

server:~# grep httpd /var/log/audit/audit.log |audit2allow -M my_local_module
server:~# semodule -i my_local_module.pp

Temporary mitigation of SELinux troubles

If sealert and audit2allow can not immediately solve your problems and you quickly need to get your service up and running again, temporary put your system in permissive mode.

server:~# setenforce permissive

It will stay in pemissive mode until you reboot your system.

Permissive mode does not enforce the SELinux policies, it just logs AVC denials and help you to solve the problems without any service interruption. Be aware: This is a temporary quick fix, not a solution.

Put the affected domain only into permissive mode

If all your investigation did not help, all answers from support did not helped (very unlikely) you can put a particular domain into permissive mode. The rest of the policies are still in enforcing mode, your system still have some protection.

As an example, you can put the Apache module into permissive mode:

server:~# semanage permissive -a http_t

Hardening your System

Most people are not aware of the fact that when a system is in enforcing mode a malicious user with root access can manipulate policies or put SELinux into permissive mode.

There is a method to prevent this: Lock down your system

server:~# setsebool -P secure_mode_policyload on

Be aware: Once active nothing can not be changed during runtime, you need to reboot your system and provide selinux=1 enforcing=0 as grub boot parameter to be able to change any SELinux settings.

Have some fun!

Download “The SELinux Coloring Book” and learn more 🙂

Further reading

Have fun 🙂