Audit your systems for security compliance with OpenSCAP

OpenSCAP logoIntroduction to (Open)SCAP

SCAP stands for Security Content Automation Protocol. It is an open standard which defines methods for security policy compliance, vulnerability management and measurement etc. This article focuses on the operating system compliance part of SCAP.

It comes originally from the US National Institute of Standards and Technology (NIST) to provide a way for US government agencies to audit its systems for regulatory compliance.

OpenSCAP is a NIST validated open source implementation of SCAP.

Why should I make use of OpenSCAP anyway?

Lot of people will ask this question to them self, in particular System Administrators and Engineers since they are not IT Security Officers.

The simple answer is that you just sit down with the IT Security Officer once and define which systems need to be compliant to what regulatory, With OpenSCAP you can always ensure the systems are configured according the the policy (or policies).

Organizations that need to be compliant according to a official policy will sooner or later facing an external security audit. I experienced that several times, its a nightmare. If you can proof that your systems are scanned regularly with the SCAP standard, you will be very well prepared, an external auditor will not bug you for a long time.

Abbreviations, abbreviations, abbreviations

Its obvious, government agencies love abbreviations 😉 Lets explain the two most important ones.

XCCDF

Extensible Configuration Checklist Description Format. This files, i.e. /usr/share/xml/scap/ssg/content/ssg-rhel7-xccdf.xml contain descriptions used for auditing a system against compliance to a policy.

This files are usually included in your distribution and are updated if needed.

OVAL

Open Vulnerability and Assessment Language. Its used to detect vulnerabilities and patches.

Since vulnerabilities and patches are popping up very quickly they need to be downloaded and distributed to all systems to be audited on a regular base (i.e. daily).

OVAL files can be downloaded as listed below:

Organizations using System Management Tools such as Red Hat Satellite or SUSE Magager will not profit from OVAL patch scans as those products will report which patches have been applied or not by themself. Nevertheless, additional OVAL scans add the benefit of vulnerability scanning regardless of installed patches.

More Abbreviations

More abbreviations and a short description of them can be found here: https://www.open-scap.org/resources/acronyms/

OpenSCAP Scap Security Guide (SSG)

There are a lot of regulations out there. Government of some countries releases policies and sometimes SCAP content for some Operating Systems, mostly RHEL and Windows. The SSG Project works on collecting and implementing content for this policies for the operating systems as well as for some other software such as JBoss. Included in the scap-security-guide are the most important US Government and PCI-DSS for RHEL. Only available for Debian at the moment is the content for the French ANSSI DAT-NT28.

The only Linux distributions I’m aware of that provides packages for scap-security-guide are RHEL and Fedora. However, upstream there is some content for more distributions available. I really hope that all important and fine distributions such as SLES, Debian and Ubuntu will jump on the bandwagon.

Regulations covered by OpenSCAP SSG

Here a list of what is available for the most important Linux distributions.

Red Hat Enterprise Linux 7

  • PCI-DSS (Payment Card Industry – Data Security Standard), Commercial – USA
  • C2S (Commercial Cloud Services), Government – USA
  • USGCB/STIG (United States Government Configuration Baseline/Security Technical Implementation Guide), Government – USA
  • CNSSI 1253 (Committee on National Security Systems), Government – USA
  • CJIS (Criminal Justice Information Services), Government – USA

Debian and Ubuntu

Officially there is nothing available. Its is currently under development, see https://github.com/OpenSCAP/scap-security-guide/tree/master/Ubuntu/16.04 and https://github.com/OpenSCAP/scap-security-guide/tree/master/Debian/8.

As of 2017-03-04 compiling fails.

  • ANSSI DAT-NT28 (Agence nationale de la sĂ©curitĂ© des systĂšmes d’information), Government – France

Suse Linux Entrprise Server

Suse does not provide the scap-security-guide package and there is no XCCDF content for regulatory compliance checks delivered by Suse. However, some basic tests are available. It is not clear if Suse has some plans to join the scap-security-guide community, would be nice to see that. SLES customers can open a support case at https://scc.suse.com/login and ask for enhancement.

Using SCAP content without scap-security-guide

You can make use of SCAP content without the OpenSCAP security guide. Its rather complex and not covered in this article.

Installing the required packages

RHEL 7

[root@server ~]# yum -y install scap-security-guide

All required dependencies will be installed as well

Debian and Ubuntu

root@ubuntu:~# aptitude install python-openscap

All required dependencies will be installed as well

SLES12sp2

sles12sp2:~ # zypper install openscap openscap-content openscap-extra-probes openscap-utils

All required dependencies will be installed as well

Tailoring profiles

For most users it is probably too much to secure its systems according to military standards which includes turning off USB support and the like.

The most important civil regulatory by far is PCI-DSS. Each company handling kind of Credit- or Debitcard data must obey the current standard. As of writing this article this is version 3.2.

PCI-DSS is a de-facto standard in Enterprise Linux environments.

Of course it makes sense for all kind of companies to secure its systems. On systems which are not exposed security policies can be more relaxed.

Also good to know is that some tests simply do not apply to your system. I.e. if you are using a centralized identity management software such as Redhat IdM with IPA or Microsoft Active Directory then the central instance will take care about the password policies, not the particular system to be audited.

Installation of the SCAP Workbench

The Scap Workbench is available in RHEL to be installed by yum, a binary for Windows and Mac OS is available as well. Needless to say that the source code is available.

Downloads: https://github.com/OpenSCAP/scap-workbench/releases

Usage

In the following examples, we disable the check for AIDE.

SCAP-Workbench Screencast

SCAP-Workbench Screencast

You can save the tailoring file as a single XML file or even better safe it as an RPM for easy distribution to all your systems.

Scanning

The usage is the same on all tested Linux distributions. Be aware, XCCDF scanning makes no sense w/o any SCAP content. If your distribution does not provide you the necessary data, 3rd party providers may.

RHEL 7 comes with the scap-workbench which is GUI that allows you to scan the local or remote systems via SSH. The scap-workbench is a nice tool to scan a handful of servers manually but not to scan a whole zoo of servers.

You also can scan your systems with the CLI on the host itself. Kind of automation can be done with i.e with Ansible.

Manual Scan

The oscap info command gives you an overview which profiles are available.

[root@server ~]# oscap info /usr/share/xml/scap/ssg/content/ssg-rhel7-xccdf.xml
Document type: XCCDF Checklist
Checklist version: 1.1
Imported: 2017-02-14T13:33:08
Status: draft
Generated: 2017-02-14
Resolved: true
Profiles:
        standard
        pci-dss
        C2S
        rht-ccp
        common
        stig-rhel7-workstation-upstream
        stig-rhel7-server-gui-upstream
        stig-rhel7-server-upstream
        ospp-rhel7-server
        nist-cl-il-al
        cjis-rhel7-server
Referenced check files:
        ssg-rhel7-oval.xml
                system: http://oval.mitre.org/XMLSchema/oval-definitions-5
        ssg-rhel7-ocil.xml
                system: http://scap.nist.gov/schema/ocil/2
        http://www.redhat.com/security/data/oval/Red_Hat_Enterprise_Linux_7.xml
                system: http://oval.mitre.org/XMLSchema/oval-definitions-5
[root@server ~]# 

Lets choose pci-dss and start a scan:

[root@server ~]# oscap xccdf eval --profile pci-dss --results scan.xml --report scan.html /usr/share/xml/scap/ssg/content/ssg-rhel7-xccdf.xml
Title   Ensure Red Hat GPG Key Installed
Rule    ensure_redhat_gpgkey_installed
Ident   CCE-26957-1
Result  pass

Title   Ensure gpgcheck Enabled In Main Yum Configuration
Rule    ensure_gpgcheck_globally_activated
Ident   CCE-26989-4
Result  pass
[Lot of Output immited]

The parameter –results saves the result in a HTML file.

Automated scanning with Redhat Satellite 6

Users of Redhat Satellite 6 can schedule scans of large server farms. The screenshots shows you how compliance tests can be presented to a IT Security Officer.

Compliance Report

Compliance Overview

The Compliance report shows a overview of hosts and a brief look at how many test have been failed.

Compliance Report Detail view

Compliance Report Detail view

The Compliance report detail shows which test have been failed. It also provides a description of each topic.

Host details

Host details

The detail view of a host shows that this host is not compliant. In this case, security errata must be applied and the host must be reconfigured to get compliant to the security policy.

Alternatives to OpenSCAP

There are a few alternatives to OpenSCAP as listed by the NIST’s Security Content Automation Protocol Validated Products.

Further reading

Using Unbound for recursive DNS lookup

Some organizations decide to use its internal authoritative DNS servers as recursive DNS because of easiness and reverse lookup of internal RFC 1918 networks works out of the box. That should be avoided for (at least) two reasons:

  • Cache poisoning can cause security nightmares
  • Authoritative answers are never cached and can cause a high load on the DNS servers.

Cache poisoning is a problem that can lead to severe problems, as more and more information is stored in DNS. Examples:

  • SSHFP entries for SSH fingerprint of servers
  • SRV entries for LDAP and Kerberos server autodetection

If an attacker can manipulate those kind of entries it can potentially be abused for redirecting users to fake authentication services.

There are some protective measures to avoid this kind of problems:

  • The usage of a separate recursive DNS infrastructure
  • Setting up DNSSEC and sign your DNS zones
  • The use of TLS for LDAP queries

This article is about how to set up recursive DNS servers, DNSSEC will be covered in a follow-up article.

Turning off recursion in authoritative DNS servers

In the option section of the bind DNS configuration make sure you have the following line in /etc/named.conf:

allow-recursion { none; };

If you are using a different DNS server software, check the vendor manual. After a restart, check if it is working as expected.

ipa3:~# dig blog.delouw.ch @dns1.example.com

; <<>> DiG 9.9.4-RedHat-9.9.4-38.el7_3.2 <<>> blog.delouw.ch @dns1.example.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 47387
;; flags: qr aa rd; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;blog.delouw.ch.               IN      A

;; AUTHORITY SECTION:
delouw.ch.            3600    IN      SOA     dns1.example.com. hostmaster.delouw.ch. 1488874622 3600 900 1209600 3600

;; Query time: 0 msec
;; SERVER: 192.168.1.1#53(192.168.1.1)
;; WHEN: Thu Mar 09 07:29:22 CET 2017
;; MSG SIZE  rcvd: 105

ipa3:~# 

Using Unbound as recursive DNS

Unbound is very secure, lightweight and high performance DNS server for validating, recursion, and caching of queries. Its astonishing how easy it is to configure Unbound.

Installation on RHEL7, Fedora and probably other Linux and BSD distributions is easy:

recursor1:~# yum -y install unbound

For this example, all configuration is made in /etc/unbound/unbound.conf
First you must define on which IPs Unbound should listen. The default is localhost only.

interface: 0.0.0.0
interface: ::0

The next default that needs to be changed is the access control. Default to refuse all but localhost. In this example you will allow access from two of your RFC 1918 subnets and the RFC 3849 IPv6 range.

access-control: 0.0.0.0/0 refuse
access-control: 127.0.0.0/8 allow
access-control: 192.168.1.0/24 allow
access-control: 192.168.2.0/24 allow
access-control: ::1 allow
access-control: 2001:DB8::/32 allow

Forward PTR queries to your RFC 1918 zones

Unbound has a nice default setting: It ignores any queries to RFC 1918 PTR queries to avoid sending queries to the blackhole servers.

In this example, we need to change the behavior to allow queries for our internal networks 192.168.1.0 and 192.168.2.0.

local-zone: "1.168.192.in-addr.arpa." transparent
local-zone: "2.168.192.in-addr.arpa." transparent

Next up: Forward this queries to our internal DNS server infrastructure (i.e IPA or MS-DNS or simply bind)

forward-zone:
        name: "1.168.192.in-addr.arpa."
        forward-host: ipa1.example.com
        forward-host: ipa2.example.com
        forward-host: ipa3.example.com

forward-zone:
        name: "2.168.192.in-addr.arpa."
        forward-host: ipa1.example.com
        forward-host: ipa2.example.com
        forward-host: ipa3.example.com

This will forward queries at random to DNS servers ipa1,ipa2 and ipa3.example.com. Add more servers as needed.

The final step is to (re)configure your clients to use the newly set up recursive DNS servers.

Have fun 🙂

Secure your system with SELinux

SELinux Logo

SELinux Logo

Introduction to SELinux

SELinux is well known as the most sophisticated Linux Mandatory Access Control (MAC) System. If you install any Fedora or Redhat operating System it is enabled by default and running in enforcing mode. So far so good.

Its available for many years and its not rocket science to use it. This article is supposed to give you some hints how to make your system even more secure and how to solve some troubles SELinux may have on your system.

DAC vs. MAC

Linux and traditional Unix systems are using DAC (Discretionary Access Control). Every user can change access rights to its own files. SELinux is a MAC (Mandatory Access Control) System where access rights are ruled by system wide policies. This can cause confusion when access is denied to a resource. Be aware that DAC will kick in before SELinux policies do. So if access to a resource is denied, please check access rights first. In such a case you will not see any AVC denials in your logs. The return code (EACCES) is the same.

RTFM

There is plenty of information available in the man pages. Some of the configuration file examples also contains additional information.

server:~# man -k selinux

Gives a good overview

Stick to Standards

Sofware installed from a RHEL or Fedora repository is usually not a problem at all, as long as you are using standards for config files, data, ports etc. Stick to the standards wherever possible. I.e. It does not make any sense to store websites in /opt instead of /var/www/html

Standards do not work for you?

If you can not stick to the standards for whatever reason, you can adjust a lot of settings with semanage.

Adding an allowed TCP Port for Apache

If you want to run your Apache httpd on port 8010, Apache will not start and a SELinux AVC denial is filed. To check which ports are allowed for Apache run:

server:~# semanage port -l|grep http_port_t
http_port_t                    tcp      80, 81, 443, 488, 8008, 8009, 8443, 9000
server:~# 

There is nothing like 8010

You can simply add port 8010 to the allowed ports by running

server:~# semanage port -a -t http_port_t 8010 -p tcp

Check again:

server:~# semanage port -l|grep http_port_t
http_port_t                    tcp      8010, 80, 81, 443, 488, 8008, 8009, 8443, 9000

VoilĂ !

Using a non-standard location for HTML files

Lets assume you want to store your HTML files in /opt/srv. To do so, you need to change the file context of that path and restore the file context afterwards.

server:~# semanage fcontext -a -t httpd_sys_rw_content_t '/opt/srv(/.*)?'
server:~# restorecon -R -v /opt/srv

Make use of Boolean variables

There are plenty of bool variables which simple allows to turn on or off a particular protection.

To get a list of defined bools, run

server:~# getsebool -a

You may want to pipe it to less or grep for a search pattern.

As an example, the default behavior is that a web application running in the httpd_t context will not be allowed to send emails. That helps greatly to prevent a vulnerable web application to send out SPAM. Well, if you want to operate a web mail service Apache must be able to send emails. No big deal:

server:~# setsebool -P httpd_can_sendmail on

Troubleshooting

The are some CLI (and GUI) tools available to troubleshoot AVC denials. The most important is sealert. Here is an example of an AVC because of a mislabled file in /var/www/html

sealert -a /var/log/audit/audit.log
SELinux is preventing /usr/sbin/httpd from getattr access on the file /var/www/html/test.html
*****  Plugin restorecon (99.5 confidence) suggests   ************************
If you want to fix the label. 
/var/www/html/test.html default label should be
httpd_sys_content_t.
Then you can run restorecon.
Do
# /sbin/restorecon -v /var/www/html/test.html

As you can see, sealert already provides you a hint how to fix the problem. In more complex cases, audit2why and audit2allow will help you. You simply grep for the misbehaving process:

server:~# grep httpd /var/log/audit/audit.log |audit2allow -m my_local_module

Review the result to check if it makes sense (ensure your grep statement does not catch too much). If you’re confident its okay, run the same command again with a capital M as parameter. It will create you a Local Policy Module which can be inserted:

server:~# grep httpd /var/log/audit/audit.log |audit2allow -M my_local_module
server:~# semodule -i my_local_module.pp

Temporary mitigation of SELinux troubles

If sealert and audit2allow can not immediately solve your problems and you quickly need to get your service up and running again, temporary put your system in permissive mode.

server:~# setenforce permissive

It will stay in pemissive mode until you reboot your system.

Permissive mode does not enforce the SELinux policies, it just logs AVC denials and help you to solve the problems without any service interruption. Be aware: This is a temporary quick fix, not a solution.

Put the affected domain only into permissive mode

If all your investigation did not help, all answers from support did not helped (very unlikely) you can put a particular domain into permissive mode. The rest of the policies are still in enforcing mode, your system still have some protection.

As an example, you can put the Apache module into permissive mode:

server:~# semanage permissive -a http_t

Hardening your System

Most people are not aware of the fact that when a system is in enforcing mode a malicious user with root access can manipulate policies or put SELinux into permissive mode.

There is a method to prevent this: Lock down your system

server:~# setsebool -P secure_mode_policyload on

Be aware: Once active nothing can not be changed during runtime, you need to reboot your system and provide selinux=1 enforcing=0 as grub boot parameter to be able to change any SELinux settings.

Have some fun!

Download “The SELinux Coloring Book” and learn more 🙂

Further reading

Have fun 🙂

Intercepting proxies and spacewalk-repo-sync

More and more companies are using intercepting proxies to scan for malware. Those malware scanners can be problematic due to added latency.

If you using spacewalk-repo-sync to synchronize external yum repositories to your custom software channels and experience the famous message [Errno 256] No more mirrors to try in your log files, then you need to configure spacewalk-repo-sync.

Unfortunately the documentation for that is a bit hidden in the man page. You need to create a directory and create a file.

mkdir /etc/rhn/spacewalk-repo-sync/

Create the configuration item:

echo "[main]" >> /etc/rhn/spacewalk-repo-sync/yum.conf
echo timeout=300 >> /etc/rhn/spacewalk-repo-sync/yum.conf

You need to experiment a bit with the value of the timeout setting, 5min should be good enough for most environments.

/etc/rhn/spacewalk-repo-sync/yum.conf has the same options like yum.conf, have a look for more information in the man page.

Have fun 🙂