Improve your bash shell working experience

This article shows some hints how to improve your bash shell working experience to reach higher productivity. Just simple shortcuts that are not so well known.

Using the History

The bash history is underestimated when it comes to usability. Here some nice stuff to do with the history.

Search the history

Every command is kept in the history. The simplest way to use the history is using the cursor-up/down keys. Most users are aware or [ctrl]-r. Usually you hit [ctrl]-r (r like reverse search) several times and miss the command, roll your eyes, hit [ctrl]-c and do it again. Why not using forward search with [ctrl]-s in such a case? Well, that suspends your terminal. It comes from the ancient times and is not needed anymore.

Turn off terminal suspension

echo "stty -ixon" >> /etc/profile

Now you can search the history back and forward by using [ctrl]-r and [ctrl]-s.

Using another command with the same last argument

When you i.e. do ls a file and decide to edit it, you don’t need to retype the whole file path or using the mouse to copy-paste it. Use the [Alt]-. (dot) combination. It inserts the last argument used. So after ls -la /tmp/file.txt you type vi [Alt].. Review and hit enter to execute.

You can also reuse other than the last arguments, but this is more complex and does not speed up things a lot, copy-paste with your mouse is usually faster in such a case.

Forgot to sudo?

When you want to cat i.e. /etc/sssd/sssd.conf you need root access. As a normal user, access is denied.

[luc@fedora ~]$ cat /etc/sssd/sssd.conf
cat: /etc/sssd/sssd.conf: Permission denied
[luc@fedora ~]$ sudo !!
sudo cat /etc/sssd/sssd.conf
[domain/example.com]

The !! also called bash bang does the trick. It just repeats the same command as used before which all arguments. Be aware that the command is executed immediately.

Bash can copy-paste as well!

Copy-paste is not only available in graphical environments but in the bash shell as well.

If you need to type some different commands all with the same arguments, cut the stuff. Position the curser to the position on the line from where you want to copy and hit [ctrl]-k. When you want to paste, hit [ctrl]-y.

You may also achieve that using othercommand !*. Using !(bash bang) can be dangerous because the command will be executed immediately, the copy-paste method is more safe.

That also works with single words etc. basically everything where you cut or delete some stuff like [alt]-d, [ctrl]-w, [ctrl]-u

Using an editor for copy-paste from websites and word processors

There are a number of reasons why you don’t want to directly copy-paste to a shell. Sometimes the source content has not properly escaped line ends or its just garbage from word processors. You may want to review and edit appropriately before fire the command. There is a super lazy and convenient trick to do so.

The security usecase

Copy-Paste from a Website is a security nightmare. Copy-Paste the following two lines into an editor and you see what I mean.

Sample command
echo “Dont copy-paste”

Second sample

The HTML code used for that is:

Sample command<span style="font-size: 0; position: absolute; left: -100px; top: -100px"><br>:echo "Dont copy-paste"</span>
Second sample

Nice! Use an editor before pasting anything in a terminal, for the sake of security.

The word processor garbage usecase

Lot of documentation is written in word processors such as Libreoffice, MS-Office and others. They replace double hyphens to a single one and nasty stuff such as single quotes to backticks. Just for a thing called usability.

When copy-paste that stuff, you probably want to review and edit it first.

Set the EDITOR environment variable

If you are too lazy to fire up vim, you can set the EDITOR environment variable to an editor of your choice (vim, emacs, nano, whatever), system wide in /etc/profile or /etc/bashrc. A better idea is to put it in ~/.profile or ~/.bashrc.

echo "export EDITOR=vim" >>: ~./bashrc

Afterwards you can just hit [ctrl]-x-e and vim starts up. When save and exit vim, the command will be executed.

What are my Keybindings?

If you wonder what kind of shortcuts are defined in a shell, a lot are. use bind -p to show them.

Have fun 🙂

Manually enroll SLES12 systems to Redhat IdM

RHEL and Ubuntu systems leverage the ipa-client software to easily enrolled them to a Redhat IdM system. Unfortunately SLES12 lacks the required packages. Nevertheless, SLES12 systems can be enrolled manually. This article is about how to achieve this.

Why using IPA for SLES systems?

Most organizations are not pure RHEL or pure SLES shops, the reality shows a heterogeneous mix of Linux distributions in corporate data centers. It makes sense to use the same authentication and authorization system to manage them.

Disclaimer

All the “special” behavior of SLES12 is based on SP2 without any patches, I do not have a SLES subscription for this test. Some of this behavior may have been fixed.

Before touching any system, please have a valid backup ready, just in case.

Preparation work

IPA is picky when it comes to host names, they must be fully qualified. Unfortunately, the default for SLES systems is to use the short host name, this must be changed first, otherwise the functionality will be limited (besides that short host names are a potential security thread).

sles12sp2:~ # hostnamectl set-hostname $(hostname -f)

In case hostname -f does not work, check if the fully qualified host name is set to the primary IP address in /etc/hosts and try again.

Unfortunately this will not survive a reboot, a dirty hack is needed. If someone has a better idea, please let me know.

sles12sp2:~ # echo 'echo $(hostname -f) > /proc/sys/kernel/hostname' >> /etc/init.d/boot.local

Please ensure that system time is correct as Kerberos is picky about having the time in sync with the KDC.

Install the required software

SLES12 comes with the basic IPA libraries and the sssd plugin needed. It just lacks the ipa-client.

sles12sp2:~ # zypper install sssd-ipa sssd-tools sssd-krb5 krb5-client sssd-ad

All dependencies will be installed automatically.

Enable sssd start at boot time, as it is not by default

sles12sp2:~ # systemctl enable sssd

Remove nscd, caching will be done by sssd.

sles12sp2:~ # zypper remove nscd

Log out and in again to get /usr/lib/mit/ in the PATH environment.

Adding the host to IPA

[root@ipa1 ~]# ipa host-add --ip-address=192.168.100.115 sles12sp2.example.com 
----------------------------------
Added host "sles12sp2.example.com"
----------------------------------
  Host name: sles12sp2.example.com
  Principal name: host/sles12sp2.example.com@EXAMPLE.COM
  Principal alias: host/sles12sp2.example.com@EXAMPLE.COM
  Password: False
  Keytab: False
  Managed by: sles12sp2.example.com
[root@ipa1 ~]# 

Generating the Kerberos Keytab and copy it to the destination host

[root@ipa1 ~]# ipa-getkeytab -s ipa1.example.com -p host/sles12sp2.example.com@EXAMPLE.COM -k sles12sp2.example.com.keytab
Keytab successfully retrieved and stored in: sles12sp2.example.com.keytab
[root@ipa1 ~]#  

Copy the Keytab to the system:

[root@ipa1 ~]# scp sles12sp2.example.com.keytab sles12sp2.example.com:/etc/krb5.keytab

Ensure ownership and permissions are set correctly

sles12sp2:~ # chmod 0600 /etc/krb5.keytab
sles12sp2:~ # chown root:root /etc/krb5.keytab

Configuration

Usually Yast is a quite nice tool to configure a SLES system. Unfortunately Yast is very confusing when it comes to SSSD configuration. Lets do it manually.

Get the IPA CA certificate

sles12sp2:~ # mkdir /etc/ipa
sles12sp2:~ # wget http://ipa1.example.com/ipa/config/ca.crt -O /etc/ipa/ca.crt

/etc/krb5.conf

sles12sp2:~ # cat > /etc/krb5.conf << EOF

[plugins]
 localauth = {
  module = sssd:/usr/lib64/sssd/modules/sssd_krb5_localauth_plugin.so
 }


[libdefaults]
  default_realm = EXAMPLE.COM
  dns_lookup_realm = true
  dns_lookup_kdc = true
  rdns = false
  dns_canonicalize_hostname = false
  ticket_lifetime = 24h
  forwardable = true
  udp_preference_limit = 0
  canonicalize = true
  default_ccache_name = KEYRING:persistent:%{uid}

[realms]
  EXAMPLE.COM = {
    pkinit_anchors = FILE:/etc/ipa/ca.crt
    pkinit_pool = FILE:/etc/ipa/ipa.crt

  }

[domain_realm]
  .example.com = EXAMPLE.COM
  example.com = EXAMPLE.COM
  $(hostname) = EXAMPLE.COM
EOF

/etc/sssd/sssd.conf/

sles12sp2:~ # cat > /etc/sssd/sssd.conf << EOF
[domain/example.com]

cache_credentials = True
krb5_store_password_if_offline = True
ipa_domain = example.com
id_provider = ipa
auth_provider = ipa
access_provider = ipa
ipa_hostname = $(hostname)
chpass_provider = ipa
ipa_server = _srv_, ipa1.example.com
ldap_tls_cacert = /etc/ipa/ca.crt
[sssd]
services = nss, sudo, pam, ssh

domains = example.com
[nss]
homedir_substring = /home

[pam]

[sudo]

[autofs]

[ssh]

[ifp]

[secrets]
EOF

Ensure ownership and permissions are correct:

sles12sp2:~ # chown root:root /etc/sssd/sssd.conf
sles12sp2:~ # chmod 600 /etc/sssd/sssd.conf

Restart sssd

sles12sp2:~ # systemctl restart sssd

nsswitch.conf and PAM

Enable sssd

sles12sp2:~ # pam-config --add --sss

Enable automatic homedir creation on first login

sles12sp2:~ # pam-config --add --mkhomedir --mkhomedir-umask=0077

Change nsswitch.conf to use sssd

sles12sp2:~ # sed -i 's/passwd: compat/passwd: compat sss/g' /etc/nsswitch.conf
sles12sp2:~ # sed -i 's/group:  compat/group: compat sss/g' /etc/nsswitch.conf
sles12sp2:~ # echo "sudoers: sss" >> /etc/nsswitch.conf

Configure sshd and ssh to use GSSAPI for authentication

sles12sp2:~ # cat >> /etc/ssh/sshd_config << EOF
GSSAPIAuthentication yes
EOF
sles12sp2:~ # cat >> /etc/ssh/ssh_config << EOF
GSSAPIAuthentication yes
EOF

Reboot to ensure its all working and caches are clean

sles12sp2:~ # reboot

Further readings

Conclusion

Using IPA for authenticating users on SLES systems works, but it is not as comfortable as with RHEL, Fedora and Ubuntu. Suse should include the ipa-client in its distribution.

Enrolling SLES systems is not easy to automate without the ipa-client, probably Ansible could help here.

The functionality is almost the same to that for RHEL7, HBAC (host based access control) is working as expected, the same applies to centralized sudoers. Unfortunately the sssd-tools are quite outdated, sss_cache -E will not delete the sudoers cache. Suse should rebase sssd to the latest upstream version. Suse customers can file a request for enhancement in the SUSE Customer Center 😉

Have fun 🙂

Audit your systems for security compliance with OpenSCAP

OpenSCAP logoIntroduction to (Open)SCAP

SCAP stands for Security Content Automation Protocol. It is an open standard which defines methods for security policy compliance, vulnerability management and measurement etc. This article focuses on the operating system compliance part of SCAP.

It comes originally from the US National Institute of Standards and Technology (NIST) to provide a way for US government agencies to audit its systems for regulatory compliance.

OpenSCAP is a NIST validated open source implementation of SCAP.

Why should I make use of OpenSCAP anyway?

Lot of people will ask this question to them self, in particular System Administrators and Engineers since they are not IT Security Officers.

The simple answer is that you just sit down with the IT Security Officer once and define which systems need to be compliant to what regulatory, With OpenSCAP you can always ensure the systems are configured according the the policy (or policies).

Organizations that need to be compliant according to a official policy will sooner or later facing an external security audit. I experienced that several times, its a nightmare. If you can proof that your systems are scanned regularly with the SCAP standard, you will be very well prepared, an external auditor will not bug you for a long time.

Abbreviations, abbreviations, abbreviations

Its obvious, government agencies love abbreviations 😉 Lets explain the two most important ones.

XCCDF

Extensible Configuration Checklist Description Format. This files, i.e. /usr/share/xml/scap/ssg/content/ssg-rhel7-xccdf.xml contain descriptions used for auditing a system against compliance to a policy.

This files are usually included in your distribution and are updated if needed.

OVAL

Open Vulnerability and Assessment Language. Its used to detect vulnerabilities and patches.

Since vulnerabilities and patches are popping up very quickly they need to be downloaded and distributed to all systems to be audited on a regular base (i.e. daily).

OVAL files can be downloaded as listed below:

Organizations using System Management Tools such as Red Hat Satellite or SUSE Magager will not profit from OVAL patch scans as those products will report which patches have been applied or not by themself. Nevertheless, additional OVAL scans add the benefit of vulnerability scanning regardless of installed patches.

More Abbreviations

More abbreviations and a short description of them can be found here: https://www.open-scap.org/resources/acronyms/

OpenSCAP Scap Security Guide (SSG)

There are a lot of regulations out there. Government of some countries releases policies and sometimes SCAP content for some Operating Systems, mostly RHEL and Windows. The SSG Project works on collecting and implementing content for this policies for the operating systems as well as for some other software such as JBoss. Included in the scap-security-guide are the most important US Government and PCI-DSS for RHEL. Only available for Debian at the moment is the content for the French ANSSI DAT-NT28.

The only Linux distributions I’m aware of that provides packages for scap-security-guide are RHEL and Fedora. However, upstream there is some content for more distributions available. I really hope that all important and fine distributions such as SLES, Debian and Ubuntu will jump on the bandwagon.

Regulations covered by OpenSCAP SSG

Here a list of what is available for the most important Linux distributions.

Red Hat Enterprise Linux 7

  • PCI-DSS (Payment Card Industry – Data Security Standard), Commercial – USA
  • C2S (Commercial Cloud Services), Government – USA
  • USGCB/STIG (United States Government Configuration Baseline/Security Technical Implementation Guide), Government – USA
  • CNSSI 1253 (Committee on National Security Systems), Government – USA
  • CJIS (Criminal Justice Information Services), Government – USA

Debian and Ubuntu

Officially there is nothing available. Its is currently under development, see https://github.com/OpenSCAP/scap-security-guide/tree/master/Ubuntu/16.04 and https://github.com/OpenSCAP/scap-security-guide/tree/master/Debian/8.

As of 2017-03-04 compiling fails.

  • ANSSI DAT-NT28 (Agence nationale de la sécurité des systèmes d’information), Government – France

Suse Linux Entrprise Server

Suse does not provide the scap-security-guide package and there is no XCCDF content for regulatory compliance checks delivered by Suse. However, some basic tests are available. It is not clear if Suse has some plans to join the scap-security-guide community, would be nice to see that. SLES customers can open a support case at https://scc.suse.com/login and ask for enhancement.

Using SCAP content without scap-security-guide

You can make use of SCAP content without the OpenSCAP security guide. Its rather complex and not covered in this article.

Installing the required packages

RHEL 7

[root@server ~]# yum -y install scap-security-guide

All required dependencies will be installed as well

Debian and Ubuntu

root@ubuntu:~# aptitude install python-openscap

All required dependencies will be installed as well

SLES12sp2

sles12sp2:~ # zypper install openscap openscap-content openscap-extra-probes openscap-utils

All required dependencies will be installed as well

Tailoring profiles

For most users it is probably too much to secure its systems according to military standards which includes turning off USB support and the like.

The most important civil regulatory by far is PCI-DSS. Each company handling kind of Credit- or Debitcard data must obey the current standard. As of writing this article this is version 3.2.

PCI-DSS is a de-facto standard in Enterprise Linux environments.

Of course it makes sense for all kind of companies to secure its systems. On systems which are not exposed, security policies can be more relaxed.

Also good to know is that some tests simply do not apply to your system. I.e. if you are using a centralized identity management software such as Redhat IdM with IPA or Microsoft Active Directory then the central instance will take care about the password policies, not the particular system to be audited.

Installation of the SCAP Workbench

The Scap Workbench is available in RHEL to be installed by yum, a binary for Windows and Mac OS is available as well. Needless to say that the source code is available.

Downloads: https://github.com/OpenSCAP/scap-workbench/releases

Usage

In the following examples, we disable the check for AIDE.

SCAP-Workbench Screencast

SCAP-Workbench Screencast

You can save the tailoring file as a single XML file or even better safe it as an RPM for easy distribution to all your systems.

Scanning

The usage is the same on all tested Linux distributions. Be aware, XCCDF scanning makes no sense w/o any SCAP content. If your distribution does not provide you the necessary data, 3rd party providers may.

RHEL 7 comes with the scap-workbench which is GUI that allows you to scan the local or remote systems via SSH. The scap-workbench is a nice tool to scan a handful of servers manually but not to scan a whole zoo of servers.

You also can scan your systems with the CLI on the host itself. Kind of automation can be done with i.e with Ansible.

Manual Scan

The oscap info command gives you an overview which profiles are available.

[root@server ~]# oscap info /usr/share/xml/scap/ssg/content/ssg-rhel7-xccdf.xml
Document type: XCCDF Checklist
Checklist version: 1.1
Imported: 2017-02-14T13:33:08
Status: draft
Generated: 2017-02-14
Resolved: true
Profiles:
        standard
        pci-dss
        C2S
        rht-ccp
        common
        stig-rhel7-workstation-upstream
        stig-rhel7-server-gui-upstream
        stig-rhel7-server-upstream
        ospp-rhel7-server
        nist-cl-il-al
        cjis-rhel7-server
Referenced check files:
        ssg-rhel7-oval.xml
                system: http://oval.mitre.org/XMLSchema/oval-definitions-5
        ssg-rhel7-ocil.xml
                system: http://scap.nist.gov/schema/ocil/2
        http://www.redhat.com/security/data/oval/Red_Hat_Enterprise_Linux_7.xml
                system: http://oval.mitre.org/XMLSchema/oval-definitions-5
[root@server ~]# 

Lets choose pci-dss and start a scan:

[root@server ~]# oscap xccdf eval --profile pci-dss --results scan.xml --report scan.html /usr/share/xml/scap/ssg/content/ssg-rhel7-xccdf.xml
Title   Ensure Red Hat GPG Key Installed
Rule    ensure_redhat_gpgkey_installed
Ident   CCE-26957-1
Result  pass

Title   Ensure gpgcheck Enabled In Main Yum Configuration
Rule    ensure_gpgcheck_globally_activated
Ident   CCE-26989-4
Result  pass
[Lot of Output immited]

The parameter –results saves the result in a HTML file.

Automated scanning with Redhat Satellite 6

Users of Redhat Satellite 6 can schedule scans of large server farms. The screenshots shows you how compliance tests can be presented to a IT Security Officer.

Compliance Report

Compliance Overview

The Compliance report shows a overview of hosts and a brief look at how many test have been failed.

Compliance Report Detail view

Compliance Report Detail view

The Compliance report detail shows which test have been failed. It also provides a description of each topic.

Host details

Host details

The detail view of a host shows that this host is not compliant. In this case, security errata must be applied and the host must be reconfigured to get compliant to the security policy.

Alternatives to OpenSCAP

There are a few alternatives to OpenSCAP as listed by the NIST’s Security Content Automation Protocol Validated Products.

Further reading

Cross distribution system management with Spacewalk

In a perfect world, all systems in a data centre are running the same Linux operating system, a homogeneous system landscape. In real life things are working differently. Windows systems are out of focus in this post, lets concentrate on Linux systems.

Most companies with a large Linux base are either RHEL shops or using SLES. A lot of RHEL users have some SLES systems running and so are SLES users running some RHEL systems. Some companies have additional systems running Debian.

How to handle those heterogeneous system landscapes? Those real world scenarios? Lets assume a company runs 500 RHEL systems, 20 SLES systems and some 10 Debian systems.

At the moment, for the base software management subscription such Linux users are spending a lot of money for RHN Satellite and SUSE Manager. Additionally there are per-system costs for management, provisioning and other modules. The Debian systems are handled manually. A lot of additional costs for a few out-of-strategy systems.

The solution is Spacewalk, the upstream project of the RHN Satellite which is at the same time the upstream for the recently released SUSE Manager. While SUSE offers support for RHEL systems, Red Hat does not (yet) offer support for SLES systems for RHN Satellite.

In Spacewalk Version 1.4 code contributions from SUSE are included and a student at Brno University of Technology contributed Debian support for Spacewalk as part of his master thesis.

While the support for SUSE is already quite stable, the Debian related code still have some rough edges. No wonder, SUSE is using RPM for its packaging wile Debian has its own packaging system. This makes it much easier for SUSE to get Spacewalk ready for its distribution.

At the moment, one can call the Debian support still as experimental, but the goal for the Spacewalk project is to have it fully functional in future releases.

The goal should be that both of the management system from the major enterprise Linux vendors, Red Hat and SUSE should support each others distribution for its Spacewalk based products. Debian is a niche player in the enterprise Linux environment and should also be supported by both products, RHN Satellite and SUSE manager. Nobody does expected to get system support for those distributions by the competing distribution, but having support for the management of it.

Further readings:
Registering Clients
Deb support in Spacewalk

Have fun!