Using modern Protocols like HTTP/2 and QUIC

First there was HTTP, then HTTP/2 and now HTTP/2 over the QUIC protocol. Lets have a look at the available HTTP Clients and Servers that support HTTP/2 and the experimental QUIC protocol.


The Hypertext Transfer Protocol (HTTP) was invented in 1991. Up to 2015 then there was only little to no evolution. In 2015 the HTTP/2 protocol was defined as a standard. HTTP/2 is much more efficient that its ancestors.

It features multiplexing, stream prioritization, binary transmission and much more. Its a huge step forward.

Nevertheless, there is a need for something more efficient. HTTP/2 is using TCP (Transmission Control Protocol) which was created in the early days of the Internet to have a reliable connection over unreliable networks. Today’s networks are much more reliable which allows the usage of the unreliable but very efficient UDP (User Datagram Protocol) to transmit data. As a consequence, QUIC was born. It is using UDP instead of TCP.

QUIC includes the crypto layer, so there is no need of a separate TLS layer. The goal is to use TLS 1.3 which is not ready as of writing this post.

Both, QUIC and TLS 1.3 are currently being defined as standards, the current state of the TLS Working group is publish here, the work of the QUIC working group is vailable here.

A good overview about QUIC can be found here.

Client Software

As of writing this post, all major Browsers are supporting HTTP/2 over TCP. When it comes to QUIC, there is little left. At the moment only Chrome and Opera are capable to access web sites with QUIC.

It is expected that this will change as soon as the standard is finalized.

Web sites

I’m not aware of any prominent Website using QUIC beside of google. HTTP/2 is used by a lot of prominent sites such as facebook, google and many others.

Server Software

The situation for HTTP/2 looks good, most webservers such as Apache HTTPD, NGINX etc. come with support for HTTP/2. Well, Apache does not work with the prefork MPM, that means you can not use mod_php with HTTP/2. You can make use of FastCGI but this means that Apache will be the slowest webserver available on the market. Better use NGINX.

If it comes to QUIC support, there is an experimental NGINX module available.

An option could be the commercial LiteSpeed Server.

From my point of view, the only usable Webserver for both, HTTP/2 and QUIC is Caddy. Its a relatively new open source project implementing a lot of new and experimental technologies. A nice feature is automatic HTTPS with Letsencrypt.

Caddy Webserver

Lets have a closer look to Caddy on Fedora 27. Its quite straight forward to install and configure.


[root@f27 ~]# dnf install caddy certbot


cat > /etc/caddy/caddy.conf << EOF
:80 {
    root /usr/share/caddy

Get a Letsencrypt Certficate

[root@f27 ~]# certbot certonly
Saving debug log to /var/log/letsencrypt/letsencrypt.log

How would you like to authenticate with the ACME CA?
1: Spin up a temporary webserver (standalone)
2: Place files in webroot directory (webroot)
Select the appropriate number [1-2] then [enter] (press 'c' to cancel): 2
Plugins selected: Authenticator webroot, Installer None
Please enter in your domain name(s) (comma and/or space separated)  (Enter 'c'
to cancel):
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for
Input the webroot for (Enter 'c' to cancel): /usr/share/caddy/
Waiting for verification...
Cleaning up challenges

 - Congratulations! Your certificate and chain have been saved at:
   Your key file has been saved at:
   Your cert will expire on 2018-05-31. To obtain a new or tweaked
   version of this certificate in the future, simply run certbot
   again. To non-interactively renew *all* of your certificates, run
   "certbot renew"

Configure TLS

cat >> /etc/caddy/caddy.conf << EOF
:443 {
    root /usr/share/caddy
    tls /etc/letsencrypt/live/ /etc/letsencrypt/live/


Give the caddy user access to the cert and key

[root@f27 ~]# setfacl -m u:caddy:r-X /etc/letsencrypt/live

Enable QUIC

[root@f27 ~]# cp /usr/lib/systemd/system/caddy.service /etc/systemd/system/
[root@f27 ~]# sed -i 's#ExecStart=/usr/bin/caddy -conf /etc/caddy/caddy.conf -log stdout -root /tmp -agree#ExecStart=/usr/bin/caddy -conf /etc/caddy/caddy.conf -log stdout -root /tmp -agree -quic#g' /etc/systemd/system/caddy.service
[root@f27 ~]# systemctl daemon-reload
[root@f27 ~]# systemctl restart caddy

Checking the Result

Enabling QUIC in your brower

Point Chrome to chrome://flags/ and search for QUIC. Enable it and relaunch the browser.

Open Chrome and a second tab with chrome://net-internals/#quicType the URL, i.e. Switch the to chrome tab and see the Result.

QUIC Screenhot

QUIC Screenhot

Blueborne – How to disable Bluetooth in Fedora

Yesterday 2017-09-13 Redhat released infomation about the mitigation of the Blueborne vulnerability in RHEL:

For Fedora the new updates are probably still in the build queue and/or being QAed by the community. For a quick fix, you can disable Bluetooth similar than in RHEL:

Stopping Bluetooth related service

systemctl stop bluetooth.service
systemctl disable bluetooth.service
systemctl mask bluetooth.service

Disable the Kernel modules

echo "install bnep /bin/true" >> /etc/modprobe.d/disable-bluetooth.conf
echo "install bluetooth /bin/true" >> /etc/modprobe.d/disable-bluetooth.conf
echo "install btusb /bin/true" >> /etc/modprobe.d/disable-bluetooth.conf
echo "install btintel /bin/true" >> /etc/modprobe.d/disable-bluetooth.conf
echo "install btrtl /bin/true" >> /etc/modprobe.d/disable-bluetooth.conf
echo "install btbcm /bin/true" >> /etc/modprobe.d/disable-bluetooth.conf

Removing the Kernel Modules from a running System

  rmmod bnep
  rmmod btusb
  rmmod btintel
  rmmod btrtl
  rmmod btbcm
  rmmod bluetooth

Configure SSSD to work on IPv6-only Hosts

SSSD is used for the client side of IPA and other centralized Identity Management Services. Unfortunately it does not behave as it should. The default is to look up first IPv4 addresses and if that fails IPv6 should be used. Well, if IPv4 fails, the whole request fails and you got weird error messages when joining an IPA domain.

As the pool for IPv4 addresses is depleted, IPv6 is getting more and more important. Thus, IPv6-only hosts are on the rise.

Here is an example error message from the IPA client.

[root@ipv6host ~]# ipa-client-install
[output ommited] 
SSSD enabled
Configured /etc/openldap/ldap.conf
Unable to find 'admin' user with 'getent passwd'!
Unable to reliably detect configuration. Check NSS setup manually.
[output ommited]

The host itself gets properly joined to the IPA domain and authentication works with Kerberos but you can not log in because SSSD fails.


Configure SSSD to only use IPv6. This is done in /etc/sssd/sssd.conf

lookup_family_order = ipv6_only
cache_credentials = True
krb5_store_password_if_offline = True
ipa_domain =
id_provider = ipa
auth_provider = ipa
access_provider = ipa
ipa_hostname =
chpass_provider = ipa
ipa_server = _srv_,
ldap_tls_cacert = /etc/ipa/ca.crt
services = nss, sudo, pam, ssh

domains =
homedir_substring = /home








At the moment there is no solution yet (just the workaround described), but its addressed at the SSSD project team, as you can see in and

Happy IPv6-ing 🙂

Secure your system with SELinux

SELinux Logo

SELinux Logo

Introduction to SELinux

SELinux is well known as the most sophisticated Linux Mandatory Access Control (MAC) System. If you install any Fedora or Redhat operating System it is enabled by default and running in enforcing mode. So far so good.

Its available for many years and its not rocket science to use it. This article is supposed to give you some hints how to make your system even more secure and how to solve some troubles SELinux may have on your system.


Linux and traditional Unix systems are using DAC (Discretionary Access Control). Every user can change access rights to its own files. SELinux is a MAC (Mandatory Access Control) System where access rights are ruled by system wide policies. This can cause confusion when access is denied to a resource. Be aware that DAC will kick in before SELinux policies do. So if access to a resource is denied, please check access rights first. In such a case you will not see any AVC denials in your logs. The return code (EACCES) is the same.


There is plenty of information available in the man pages. Some of the configuration file examples also contains additional information.

server:~# man -k selinux

Gives a good overview

Stick to Standards

Sofware installed from a RHEL or Fedora repository is usually not a problem at all, as long as you are using standards for config files, data, ports etc. Stick to the standards wherever possible. I.e. It does not make any sense to store websites in /opt instead of /var/www/html

Standards do not work for you?

If you can not stick to the standards for whatever reason, you can adjust a lot of settings with semanage.

Adding an allowed TCP Port for Apache

If you want to run your Apache httpd on port 8010, Apache will not start and a SELinux AVC denial is filed. To check which ports are allowed for Apache run:

server:~# semanage port -l|grep http_port_t
http_port_t                    tcp      80, 81, 443, 488, 8008, 8009, 8443, 9000

There is nothing like 8010

You can simply add port 8010 to the allowed ports by running

server:~# semanage port -a -t http_port_t 8010 -p tcp

Check again:

server:~# semanage port -l|grep http_port_t
http_port_t                    tcp      8010, 80, 81, 443, 488, 8008, 8009, 8443, 9000

VoilĂ !

Using a non-standard location for HTML files

Lets assume you want to store your HTML files in /opt/srv. To do so, you need to change the file context of that path and restore the file context afterwards.

server:~# semanage fcontext -a -t httpd_sys_rw_content_t '/opt/srv(/.*)?'
server:~# restorecon -R -v /opt/srv

Make use of Boolean variables

There are plenty of bool variables which simple allows to turn on or off a particular protection.

To get a list of defined bools, run

server:~# getsebool -a

You may want to pipe it to less or grep for a search pattern.

As an example, the default behavior is that a web application running in the httpd_t context will not be allowed to send emails. That helps greatly to prevent a vulnerable web application to send out SPAM. Well, if you want to operate a web mail service Apache must be able to send emails. No big deal:

server:~# setsebool -P httpd_can_sendmail on


The are some CLI (and GUI) tools available to troubleshoot AVC denials. The most important is sealert. Here is an example of an AVC because of a mislabled file in /var/www/html

sealert -a /var/log/audit/audit.log
SELinux is preventing /usr/sbin/httpd from getattr access on the file /var/www/html/test.html
*****  Plugin restorecon (99.5 confidence) suggests   ************************
If you want to fix the label. 
/var/www/html/test.html default label should be
Then you can run restorecon.
# /sbin/restorecon -v /var/www/html/test.html

As you can see, sealert already provides you a hint how to fix the problem. In more complex cases, audit2why and audit2allow will help you. You simply grep for the misbehaving process:

server:~# grep httpd /var/log/audit/audit.log |audit2allow -m my_local_module

Review the result to check if it makes sense (ensure your grep statement does not catch too much). If you’re confident its okay, run the same command again with a capital M as parameter. It will create you a Local Policy Module which can be inserted:

server:~# grep httpd /var/log/audit/audit.log |audit2allow -M my_local_module
server:~# semodule -i my_local_module.pp

Temporary mitigation of SELinux troubles

If sealert and audit2allow can not immediately solve your problems and you quickly need to get your service up and running again, temporary put your system in permissive mode.

server:~# setenforce permissive

It will stay in pemissive mode until you reboot your system.

Permissive mode does not enforce the SELinux policies, it just logs AVC denials and help you to solve the problems without any service interruption. Be aware: This is a temporary quick fix, not a solution.

Put the affected domain only into permissive mode

If all your investigation did not help, all answers from support did not helped (very unlikely) you can put a particular domain into permissive mode. The rest of the policies are still in enforcing mode, your system still have some protection.

As an example, you can put the Apache module into permissive mode:

server:~# semanage permissive -a http_t

Hardening your System

Most people are not aware of the fact that when a system is in enforcing mode a malicious user with root access can manipulate policies or put SELinux into permissive mode.

There is a method to prevent this: Lock down your system

server:~# setsebool -P secure_mode_policyload on

Be aware: Once active nothing can not be changed during runtime, you need to reboot your system and provide selinux=1 enforcing=0 as grub boot parameter to be able to change any SELinux settings.

Have some fun!

Download “The SELinux Coloring Book” and learn more 🙂

Further reading

Have fun 🙂