Synopsis: (this presentation is based on a blog post by Farshad Abasi on 2017-03-04; See original post below)
Applications have taken many forms over the years, from single to multi-user, client-server, and distributed architectures. For the most part, these applications followed a monolithic design where various functions lived together inside a walled garden or trust boundary. These functions communicated primarily via memory or the local filesystem, removing the possibility of the network as an attack vector. With the advent of SOA (Service-Oriented Architecture) and microservices, the walls have fallen, and modern applications are being decomposed into discrete and independent units of functionality. Each component usually lives inside a container and is accessible over the network through an exposed API (typically RESTful). This results in flexible and independently deployable components, suitable for DevOps and Agile models.
At the same time, this requires having the right security controls in place to create a similar level of trust between these newly-decoupled units as existed previously when they lived closely together and communicated locally within the same application trust boundary. End-to-end trust needs to be maintained from the time user authentication takes place, all the way through to the end of the user journey across the various units of the application. In addition, the tools and technologies used to facilitate these modern architectures such as container engines and orchestration tools are fairly new and not mature or fully understood, leading to risks from misconfiguration or vulnerabilities that need to be addressed.
This presentation is targeted to application as well as security architects, developers, or anyone else who is dealing with these modern service-based applications and requires practical knowledge on how to best secure these applications. We will further this topic by bringing together the difference security concepts and required controls such as end-to-end trust and policy enforcement points into a single high level architecture pattern that can be applied when building services or microservices based applications.
Date/Time & Location:
Thursday Nov. 22, 2018 – 18:00 PST @ UBC Computer Science Alumni/Industry lecture series (Vancouver, BC)
The presentation above is based on the original blog post below by Farshad Abasi from 2017-03-04:
API and Microservices Security
What are APIs and microservices, and what are some of the security considerations when it comes to these technologies? In this brief article, I am going to provide you with an overview of what these mean, and some basic security principles you should consider while deploying them in your environment.
What are APIs?
Application Programming Interface or APIs are a set of clearly defined communication methods between software components. APIs have been around for quite a while and have made it easy for independent teams to build software components that can work together without knowledge of the inner working of others. Each component only needs to know what functions are exposed as APIs by the other component it wants to interact with. This creates a nice separation layer and promotes modularity, where each component in a system can focus on a certain set of functionality and expose the useful features to the outside world for consumption, while hiding the complexities inside.
What about Microservices?
First off, let’s clarify something: there is no one-to-one relationship between APIs and microservices. Kim Clark does an excellent job in his article, talking about the relationship between SOA, microservices, and APIs. I wanted to mention this, because there are many people and organizations out there who are moving towards microservices and modern/RESTful API based architectures, and the two are often talked about at the same time.
Many organizations have “monolithic” applications which contain several services and functions. A movement began in the 00s towards Service Oriented Architecture (SOA), where organizations would analyze the the applications deployed, and look for services that were duplicated across these applications. Based on this analysis, the applications would be replaced by a set of common services, thereby removing duplication of those services across the organization providing modularity, and allowing development of various applications by combining these common enterprise services. Each service contained many functions that were tightly coupled together to perform that service. In more recent years, another movement has started which further breaks these services into yet smaller microservices by decoupling the tightly coupled functions within each service, and exposing them as APIs. Each microservice focuses on a very specific function which was previously provided by the service along with other related functions. Keep in mind that not all microservices will be exposed via an API, as some may simply provide internal functionality, similar to the concept of private methods of an object that are not exposed in OO programming.
This further breakdown offers a few advantages, leading to flexible and independently deployable systems. In the past, several people would work on different functions making up the service or application, and a problem with one function would prevent successful compilation and roll-out of the entire service or application. By decoupling those functions into microservices, the dependency on other functions is removed, and issues with development and roll-out in one function will no longer impact the others. This also lends itself well to DevOps and Agile models, making a shift from having to design the entire service at once and allowing for continuous deployment.
The API Gateway and the Post-monolithic World
In the world of monolithic applications, all functions reside within the same walled garden, protected by a single point of entry which is typically the login functionality. Once the end-user has passed this authentication point and has been implicitly authorized to access the overall application, the user journey to access the various functions inside will not require further authentication and re-entry of the user credentials. This is due to an architecture where all functions are tightly coupled and cannot be invoked by outsiders to the application. In this scenario, each function may or may not perform further authorization (AuthZ) checks, depending on the requirements and granularity of entitlements scheme.
When we break these monoliths into smaller components such as microservices, we no longer have the single point of entry as we had with the monolithic application. Now each function can be accessed independently by end-users via its microservice API and needs a way to ensure that user is authentic, without requiring the full set of credentials each time. This where the API Gateway comes in, acting as a central point of enforcement for various security policies, including end-user authentication. Once the end-user has gone through a “heavy” authentication process and provided their full set of credentials, an access token is issued to the end-user which can be used for “light” authentication and further interaction with APIs through the API Gateway. The API Gateway guards the APIs and ensures this access token is present and all policies are met before granting access downstream.
Why do I Need End-to-end Trust and AuthZ?
With this microservice based architecture involving an API gateway, we have shifted the responsibility for enforcing common security policies such as authentication, rate-limiting, etc… to the API gateway where those checks take place. Once past this gateway, downstream APIs invoked must be trust that these checks were already performed and avoid performing the checks again. This is done by having the API gateway issue an end-to-end (E2E) trust token once the user has presented a valid access token and all security policies have been enforced and passed successfully. All downstream APIs can simply verify this E2E trust token and assume that the user has been identified and the request has satisfied all applicable policies. In addition, to ensure a compromised E2E trust token (or one generated using a compromised key) cannot be played back via unauthorized components, mutual authentication should be put in place at the transport layer among all components authorized to partake in a specific API ecosystem.
The Need for Fine-grained AuthZ
The E2E trust token discussed so far simply represents that the request has gone through the API gateway, the access token has been validated and necessary policies have been successfully enforced, and the end-user is authorized to access the API protected by that API gateway in general. But what if we have fine-grained authZ requirements? For example, we may want to limit access to an API based on who the access is initiated from combined with the specific resource being accessed (e.g. account ID belonging to a specific user), the HTTP method used (e.g. PUT, GET, POST, DELETE, etc…), at a specific authentication level (e.g. requiring a TOTP token to have been used for authentication by the end-user). What our E2E trust token does not provide is the ability for an API to know whether or not the request meets all the more fine grained authorization requirements for that API. As such, an authZ check can be performed against these criteria to ensure they are fulfilled based on the entitlements information and configuration, prior to fulfilling the API request. Since in some microservices architectures, direct API to API calls do not go through the API gateway, this authZ check needs to take place at each API (which can in turn invoke a central authZ service); however if the architecture requires all API access to go through the API gateway, the authZ check can be performed there instead, and information can be added to the E2E trust token’s payload.
Invocation by External Applications and Services
In cases where an API is being invoked by an external application or service and end-user authentication has already been performed by that application, user context may not be passed down to the API gateway. As such, the API gateway will not receive a user authentication token to be exchanged for an E2E trust token as was the case previously. Here, the originating application has already authenticated the users and the API gateway needs to give implicit trust to that application to act on behalf of all its users. As such, the API gateway needs to ensure the calling application is valid by performing mutual authentication and checking the application’s ID against a list of allowed consumers to whom implicit trust should be given for the APIs. All other security policies should be enforced by the API gateway as before, since we should follow defense-in-depth strategy and cannot always trust that external applications have performed the same security checks.
WHAT OTHER SECURITY ISSUES Should I Care About?
In addition to acting as central a point of authentication enforcement, the API gateway should also enforce other security policies such:
- Rate-limiting: to prevent calling of an API more than the allowed number of times for a given period by the consumer and causing denial of service to others.
- JSON threat protection: to protect against malicious input in the JSON payload resulting in an attack.
- XML threat protection: to protect against malicious input in the XML payload resulting in an attack.
- Other custom policies to centrally address applicable web application security threats
Most required security policies should be provided to developers with the ability to configure and apply them as needed to ease the development process. Custom policies may still need to be coded in case one does not exist to meet a specific need.
Don’t Forget to Log, Monitor, and Detect
As is the case with all security architectures, detective controls such as logging and monitoring play an important role here as well. Each API needs to log all important events including security related ones, and send this data to a central system for further correlation, analysis and monitoring. Events such as success or failure of compliance to a given policy will be important to record for security analysis. In addition, information such as how many times end-points were invoked and response times will assist in preventing denial of service and better profiling of the application and system.
APIs can also take part in fraud detection, by acting as instruments to provide data related to the specific device and location from which the API was accessed. This information can assist in detecting scenarios that match a given fraudulent access pattern.
Apply Group Policy
Grouping objects which have some relationship into a single unit and applying configuration values or policies across the group is not a new concept. This helps in consistent application of those values and policies to the given set of objects in that group. The same principle should be applied to APIs, whereby APIs that address a particular business need should be grouped together and presented to developers along with a service plan and set of policies that apply to that group.
For example, if there are a set of APIs that provide data in XML format and can be used to build an internal application for account servicing by Staff, those APIs will require an authentication policy that integrates with the corporate AD, as well as XML threat protection and a rate-limiting policy that focuses on inside initiated access. On the other hand, another set of APIs that use JSON as their data format and can be used to build an external application for Customers to perform Internet banking will require an authentication policy which uses the customer data repository (e.g. LDAP), enforcement of JSON threat protection, and rate-limiting designed for external use-cases.
Once the policies have been successfully enforced, access should be authorized and limited only to the other APIs within the group. Of course as discussed earlier, fine-grained authZ enforcement may be desired over and above this group level one.
In summary, the following key security concepts should be taken into consideration when designing and implementing API based systems:
- Maintain end-to-end trust across the entire journey
- Ensure authZ is enforced at the right place with the right level of granularity
- Group your APIs to apply configurable security policies consistently
- Don’t forget to log, monitor and detect
- Follow a defense-in-depth strategy and add security at all layers