Joining Hands and Singing Merrily Part 3

In my last post, I presented an overview of the major technologies we’re using to build XFIL, including our main choices of programming languages, as well as our tools for development and testing support. Today we’re going to look at how we tackle the problems of authenticating users and agents. This discussion is going to give us an excellent chance to see how we’ve used cryptography to secure user account credentials as well as to provide agents with a secure way to authenticate themselves without user intervention. We’ll also get to have a look at an open source project that Stratum has released on Github, which we use for securing user credentials at rest.


Let's go ahead now and dive into one of the major categories of security-related functionality that is critical to most applications, and certainly to a couple of significant parts of XFIL as well. Authentication is all about establishing identity. In XFIL, there are two external parties, i.e. not XFIL services, that we need to be able to identify: human users, and our agent software.


Our story for identifying users is really not distinct from that of most web applications. XFIL includes a web interface that users can access to configure and download the agent, manage groups and users, and view the results of the tests their agents have run. To authenticate users, we take the traditional approach of requiring users supply their registered email address and corresponding password to log in. We also allow users to add two-factor authentication, which we handle by using Authy. As to be expected, there are a couple more components to this story.

Password Requirements

Weak password requirements are a fairly common problem in web applications that make user accounts easier to compromise. In a modern web application, there is rarely, if ever, any excuse for not

  • allowing users to supply arbitrarily long (maybe up to a really high limit) passwords
  • allowing users to include any characters they can type in their password, including unicode
  • insisting that users include upper and lower-case, numeric, and symbolic characters in their password

to name a few things. We have opted to start out with some fairly strong demands; we insist on ten or more characters, and that at least one upper-case, one lower-case, one number, and one symbol be included in a password.

But of course, things change. There's no guarantee that a few years or even days from now, we won't have reason to strengthen our requirements further, or even to relax them. To cope with the potential need to change, our authentication system has taken a slightly uncommon approach of allowing for configurable password requirements. That is to say that, instead of hardcoding our password requirements, we have written a generic complexity check function that evaluate passwords based on a specification describing our requirements. This way, we can tweak them by simply modifying a configuration file and restarting the service. In Python, the code would look something like this:

import string

def is_password_complex_enough(password, requirements):
    '''Determines if a password meets our configurable complexity requirements'''
    # These are some bad names, I know, but they make things align nicely for this example.
    long_enough = len(password) >= requirements['length']
    has_upper = requirements['upper'] and any(char.isupper() for char in password)
    has_lower = requirements['lower'] and any(char.islower() for char in password)
    has_numer = requirements['numer'] and any(char.isdigit() for char in password)
    has_symbl = requirements['symbl'] and any(char in string.punctuation for char in password)
    return long_enough and has_upper and has_lower and has_numer and has_symbl

# We would load this configuration from a file when the service starts.
requirements = {
    'length': 10,
    'upper': True,
    'lower': True,
    'numer': True,
    'symbl': True
valid = 'H3ll0_W0rlD!'
invalid = 'Password1'

print is_password_complex_enough(valid, requirements)
# >>> True

print is_password_complex_enough(invalid, requirements)
# >>> False

Password Checking

One of the most irresponsible things a developer can do with regards to securing their users' accounts is fail to adequately protect credentials at rest. Some use cases, where more data is highly sensitive, may require using encryption with a modern cipher and a properly managed key, but every application that stores user credentials should absolutely be doing at least one thing: password hashing.

BCrypt has, for some time now, been the darling hash function suggested by many security-conscious folks in the web application world. It is a modern cryptographic hashing function that is designed to allow for users to increase or decrease its computational cost- the amount of processing resources that it requires- by supplying a numeric cost parameter. However, bcrypt is only designed to scale up the amount of processing time that it uses, meaning it is really capable of mitigating attacks exploiting the fact that processors keep getting more and more powerful. It is not really designed to also scale its memory usage as a means of defending against attacks that trade memory for computation or those that exploit parallel computation. In response, some applications have started to use key derivation functions (KDFs), some of which do make it possible to scale complexity in these new ways. One such algorithm is scrypt.

At Stratum, we are using Go for our identity and access management software, largely because of the confidence its simplicity and incredible cryptography libraries afford us. Unfortunately, Go's standard library does not include a password-hashing solution that makes use of scrypt, while it does have one that uses bcrypt. However, it does include an implementation of scrypt, so we aren't going to be breaking rule #1 of cryptography and start rolling our own crypto, but we needed to do a little work to wrap it up for our use case. The effort to provide an interface for using scrypt to do password hashing gave birth to Stratum's own scryptauth library which is completely free and open source. Its interface is meant to mimic Go's own bcrypt API, so it should be easy to pick up.


User authentication is a familiar story to developers. Perhaps more interesting are the challenges posed by authenticating agent software. In order to be able to download a configuration for a test campaign, start probing ingestion services, and to upload result data, agents need to be able to establish their identity the same way a user would. However, there are a couple of new problems to solve:

  1. The agent should authenticate autonomously, without requiring the user input a separate set of credentials every X minutes or hours.
  2. It should not be possible for an attacker to impersonate an agent after having conducted a man-in-the-middle attack at a point, say, where SSL is stripped in a corporate network

To address these problems, the agent authenticates itself using a Challenge-Response Identification scheme.

This approach breaks authentication up into two steps. One where the agent tells us that it intends to authenticate itself and provides us with some information about its identity, and another where we challenge the agent to prove it really is the agent it claims to be by having it sign a message containing data it could not have known beforehand. Provided that the agent is able to create such a signature, we have a reliable guarantee of the agent’s identity. Of course, this scheme depends on users not losing their agent’s private keys, but this is really only one part of a larger system, wherein we might try to detect fraudulent usage, invalidate keys, notify users, and deploy new keys.

Coming Up...

Now that we have completed our overview of the high-level software design process, our choices of development tools and languages, and have begun talking about our approaches to common security features, we're ready to go all-in in our next post and look at some of the much more novel approaches used to secure data in XFIL. We'll be looking at the challenges of authorization, how it is distinct from authentication, and how we make sure sufficient authorization checks have taken place between services. That will be part of an interesting discussion about service-based architectures and how we've borrowed some interesting ideas from some modern distributed systems solutions.