Recent Red Hat Enterprise Linux releases see an expansion in support of the smart card related use cases. However customers usually have a mixed environment and standardize on a specific version of Red Hat Enterprise Linux for period of time. It is important to understand the 

evolution of the smart card related feature to plan your deployment and understand what capabilities are available in what version of the operating system.

Understanding Smart Card Support

When we talk about smart cards support there are several dimensions that need to be considered:

  • Card type
  • Driver
  • PAM module
  • Identity mapping
  • Advanced features
  • Red Hat Enterprise Linux version

Let us first look at those areas in more details.

Card Type

There are different types of cards and they require different logic to process data on the card. There are four types of cards that are supported by Red Hat Enterprise Linux: coolkey cards, CAC, PIV and PKCS#15. The support for different types of cards has been added over the time but coolkey and CAC cards have been supported since Red Hat Enterprise Linux 5. PIV and PKCS#15 was added in later releases. PIV support was added in 6.3 and then carried forward in 7.0 while PKCS#15 was added in 7.2 and was not backported to 6.x stream. Over the course of the releases other capabilities were added, for example the support of the contactless cards. Different types of cards have different capabilities. For example CAC cards have one compartment where the certificates can be stored while PIV card has multiple. For more details about the difference between the types of the cards please read corresponding documentation.

Driver

For many years Red Hat Enterprise Linux provided a coolkey driver to support smart card operations. First it supported coolkey cards, then CAC, then PIV and then PKCS#15 but the number of cards that are formally supported and validated was still limited. Also support of PIV and PKCS#15 was pretty bare bones. Red Hat Enterprise Linux 7.4 brings an alternative driver to coolkey called OpenSC. OpenSC project supports a big variety of cards and has a much better feature coverage than coolkey. However originally the community version of OpenSC lacked support of coolkey and CAC cards. That functionality have been ported from coolkey to OpenSC making OpenSC a full replacement for coolkey which is now deprecated as of 7.4 (see release notes).

PAM module

Pluggable Authentication Module or PAM is a component that sits higher in the stack than a smart card driver and is invoked to perform user authentication using user’s smart card. It leverages driver to interact with the card so to a big extent the type of the card is really abstracted from the PAM module meaning that CAC or PIV cards would look very similar. There might be different sets of certificates on those cards but for a PAM module it is just a bunch of certificates that can be used for different purposes. One (or some) of those certificates can be used for authentication. This is what PAM module uses. To authenticate a user PAM module needs to establish whether user that owns the card knows the PIN. To do that PAM module creates a challenge, encrypts it with a public key fetched from the card and calls into the card for it to use its private key to decrypt the challenge. Access to the private key is controlled by a user PIN. So only if the user types the right PIN and unlocks the private key the card would be able to properly process and respond to the challenge. In addition to the validation of the knowledge of the PIN PAM module needs to make sure that the user is still allowed to login. The certificate might have expired so PAM module checks the expiration and makes sure that this is not the case. The user also might have been laid off so his certificate might have been revoked. PAM module would need to directly or indirectly check for such situation too. In Red Hat Enterprise Linux the check is done by NSS library on behalf of the PAM modules. The check is done over OCSP protocol against an OCSP server or against a certificate revocation list (CRL) published by CA that issued the cert.

Over a long period of time even from early days of Linux there have been two PAM modules pam_pkcs11 and pam_krb5 that were capable of interacting with smart cards and performing user authentication. However there were a lot of challenges with identity mapping that is covered in the next section.

Starting Red Hat Enterprise Linux 7.2 the System Security Service Daemon (SSSD) project started to add smart card related capabilities. There have been a series of releases 7.2, 7.3, 7.4 that implemented different features that will be covered later in this post. Features that were implemented in 7.2 were back ported to 6.8. Features added in later 7.x releases were not ported back and there is no plan to do so.

Identity Mapping

Once the user is authenticated, i.e. the PAM module established that certificate is valid and the user who is in the possession of the card knows the PIN, the user then needs to be mapped to a POSIX user. POSIX users are the ones that Linux system understands. Those users have an id and login name. They run their session on the system and processes within the session are owned by the id they have. This id is called uid. Without mapping to a POSIX user a user session can’t be started.

Challenges with Mapping

Mapping of a user is in fact quite a challenge. In a simple case, when user authenticates with username and password, the PAM module right away knows the username. It can use local or remote lookup to determine whether the user exists and what uid he has. In the case of the smart card authentication it is not that easy. Every certificate has couple different attributes that might identify a user. But historically there is no standard mapping. Moreover in many cases the identity that is baked into a certificate has no relationship to the actual user login. This is true with many government issued CAC cards. The identity on the certificate is a string that identifies the cert but not the user.  So how one can decide which user to map to? Since there is no standard developers from different projects tried their best to solve the problem.

pam_krb5

Pam_krb5 module being a Kerberos module looks at the special field in the certificate that should contain a user Kerberos principal. If it is there - the problem is solved. It can be used automatically by a PAM module to identify user and perform Kerberos certificate based authentication called PKINIT. Unfortunately most of the CAC cards issued over the years do not have this field populated. And this is quite understandable. Government employees get their cards from a central server first and only then get into the office they will work in and register with the IT. So the central server that is issuing the cards has no clue what username will user use in his department. Without this field populated in the certificate Kerberos server does not know how to process the user and thus pam_krb5 can’t be used in such situations. This is quite limiting.

pam_pkcs11

The other module - pam_pkcs11 went the other route. It allows the certificates to be mapped to users in a file. This file is local to a system user logs into. This works OK for a single system but does not scale quite well if you have multiple people accessing same system and a user needs to access multiple systems. Maintaining and copying such file around is very hard so only a brave few went with such kind of deployment creating heavy automation using other tools and means to distribute the files.

The other approach is to store mapping centrally in a central identity solution. Since most of the modern identity solutions are LDAP based it made sense to store the mapping there. Luckily standard LDAP schema allows a user to have an userCertificate attribute where the certificate can be published. So OK let us publish a certificate into an LDAP entry and then just lookup user from a PAM module. If we found a user - great, we will use his login and uid for POSIX user. Problem solved? Not so fast. First it is not clear how you actually lookup a user? Do you get some attribute from the cert and use it as a key in the search? Do you search by the certificate as a blob? Or you create a hash and search by the hash? But then the server would have to store the hashes too somewhere and there is no standard attribute for that approach. Also how to deal with the situations when the certificate is published into more than one user account? Which one should be selected? LDAP lookups do not guarantee an order so if you just pick the first one you might be getting a different account from the previous time you tried to look the user up. Again different solutions did things differently. Pam_pkcs11 implemented enumeration. It would iterate through all the users, get user entry from LDAP and check the cert on the client side. That approach is extremely inefficient and was causing a lot of performance issues. In 7.2 pam_pkcs11 was fixed to do a lookup against LDAP using the whole certificate (a binary blob) as a key for the lookup.

SSSD

When SSSD started adding smart card related features in 7.2 the first thing it did is implemented binary blob lookup against a directory. It could have been any directory that had a userCertificate attribute in the user entry.

This approach worked. However it had couple of drawbacks:

  • Managing certificates in the user entry created a deployment overhead. The IT folks would have in some way extract the certificate from the card and place it into their LDAP. How? If you have a CA that will just publish certificate into LDAP automatically you are all set but this is not the case with many CAC and PIV based deployments as was explained earlier.
  • Using certificate as a blob was losing some semantics. If the user loses a card he will be issued a new one. But this requires his certificate to be updated in the LDAP too. Implementing this workflow is very cumbersome for IT.
  • Finally, there was still no way to disambiguate users if the certificate is mapped to more than one account.

In 7.4 SSSD focused on resolving these issues. It turned out that it is better to have a special attribute that would identify the certificate by a combination of subject and issuer rather than a full certificate blob stored in the LDAP entry. This is what Microsoft Active Directory implemented to support a similar workflow. Such attribute has several advantages:

  • Attribute does not require the certificate to be published into the entry.
  • Creating such attribute is simple, it can be done by IT when they get notification about the user joining organization. IT can create such attribute automatically without waiting for the user to show up and manually register, i.e. upload the certificate that he has on the card into LDAP.

SSSD working in conjunction with IdM team added even more flexible capability so that not only a combination of subject and issuer (which is a de facto standard) but other combination of the attributes can be used.

Advanced Features

Also SSSD and IdM implemented a feature allowing a user to select the account to use if the certificate is mapped to multiple accounts. Why is this important? It is important when a single identity system is used to serve multiple environments and user has different roles in these environments. In this case each of these roles is represented by a dedicated account but certificate information is shared between all of them.

In addition to this SSSD in 7.4 is now capable of performing PKINIT in the same way as pam_krb5 thus surpassing pam_krb5 in pretty much all of its capabilities. As a result pam_krb5 has been deprecated. It will be supported till the end of life of Red Hat Enterprise Linux 7 but most likely will not be included into the next major release.

On top of that SSSD and IdM can now run in FIPS mode.

Last but not least SSSD in 7.4 has logic to recognize the Kerberos principal specified in the certificate and also prompt user for him to explicitly enter the account that should be used rather than inferring it by doing lookups and giving user choice. This forces the user to make up his mind what account he plans to use before he starts authentication making the solution less prone to leaking of information about what accounts the card is mapped to.

As you might have noticed I skipped 7.3 in the description of the SSSD smart card features. Most of the SSSD work in 7.3 related to smart card support was to enable SSSD to work with Active Directory or with IdM when IdM is in trust relationships with Active Directory. SSSD team as usual worked with IdM to allow smart card authentication for Active Directory users with certificates published into AD or with certificates managed in override user entries in IdM. More about this can be read in my post about Red Hat Enterprise Linux 7.3 release.

Summary of the Evolution

Now when we look into the aspects of the smart card support we can create a single table that captures its evolution.

Release 6.0 - 6.3 6.3 - 6.7 6.8+ 7.0 - 7.1 7.2 7.3 7.4
Card Type CAC, coolkey CAC, coolkey, PIV CAC, coolkey, PIV CAC, coolkey, PIV CAC, coolkey, PIV, PKCS#15 CAC, coolkey, PIV, PKCS#15 CAC, coolkey, PIV, PKCS#15
Driver Coolkey Coolkey Coolkey Coolkey Coolkey Coolkey Coolkey/OpenSC
PAM Pam_pkcs11

Pam_krb5

Pam_pkcs11

Pam_krb5

Pam_pkcs11

Pam_krb5

SSSD

Pam_pkcs11

Pam_krb5

Pam_pkcs11

Pam_krb5

SSSD

Pam_pkcs11

Pam_krb5

SSSD

Pam_pkcs11

Pam_krb5

SSSD

Mapping Limited Limited Limited + by certificate blob using SSSD Limited Limited + by certificate blob using SSSD or pam_pkcs11 Limited + by certificate blob using SSSD or pam_pkcs11 Limited + by certificate blob or subject and issuer using SSSD or by blob using pam_pkcs11
Advanced features IdM can be a server AD or IdM in trust with AD can be a server Account selection;

User hints;

PKINIT;

FIPS mode;

So what can we deduce from this? Well, there is a general rule: use the latest server and depending upon what versions your clients have you might get different functionality.

However it is a little bit more complex since depending on  the versions of your clients you might choose different architecture of your deployment.

Stay tuned for more details. In the next post I will focus on specific guidelines that will help you to choose the right deployment steps depending on your situation.