Spring Cloud Gateway: The Swiss Army Knife of Cloud Development
A microservice has to fulfill many functional and nonfunctional requirements. When implementing one, I mostly start with the happy path to see if I meet these requirements. And with all the other nonfunctional requirements, like protecting my service or scaling various parts independently, I love to work with Spring Cloud Gateway as this tiny and, IMHO, underrated tool is powerful, even with just a few lines of configuration.
What Is Spring Cloud Gateway?
Spring Cloud Gateway is an open source, lightweight and highly customizable API gateway that provides routing, filtering and load-balancing functionality for microservices. It is built on top of Spring Framework and integrates easily with other Spring Cloud components.
If you’re new to Spring Cloud Gateway, this article outlines some common use cases where it can come in handy and requires minimal configuration.
How to Get Started with Spring Cloud Gateway
The easiest way to start experimenting with Spring Cloud Gateway is by using Spring Initializr. So let’s go to start.spring.io and generate a project stub. Pick the project type, language and the versions you want, and be sure to add the Spring Cloud Gateway dependency. Once you are done, hit the Download button.
What you get is a basic project structure like this:
And this is a fully working, almost production-ready Spring Cloud Gateway. The important part is the application.yml, which will hold all further configuration. Now let’s add some magic.
Protect Services through Rate Limiting
Sometimes it’s necessary to protect your service from misbehaving clients to ensure availability for correctly behaving ones. In that case, Spring Cloud Gateway can help with its rate-limiting capabilities. By combining it with a KeyResolver, you can correctly identify all your clients and assign them a quota of requests they are allowed per second.
It also offers a burst mode, where you can get above the assigned quota for a short period of time to cope with sudden bursts of requests.
As the gateway, in that case, requires some sort of memory, you should combine it with an attached Redis cache. This would allow for horizontally scaling your gateways.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
spring: cloud: gateway: default-filters: - name: RequestRateLimiter args: # maximum number of requests per period redis-rate-limiter.replenishRate: 10 # maximum number of requests that can be queued before being rejected redis-rate-limiter.burstCapacity: 20 # the key resolver for rate limiting key-resolver: "#{@apiKeyResolver}" routes: - id: example uri: http://example.com predicates: - Path=/** # key resolver for rate limiting apiKeyResolver: type: com.example.ApiKeyResolver |
The KeyResolver should look like this. It can hold any custom logic required to select the criteria of distinguishing different users. In this example, it’s just the X-API-key header, but another common thing might be an Authorization header or some sort of set cookie.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
package com.example; import org.springframework.cloud.gateway.filter.ratelimit.KeyResolver; import org.springframework.web.server.ServerWebExchange; import reactor.core.publisher.Mono; public class ApiKeyResolver implements KeyResolver { @Override public Mono<String> resolve(ServerWebExchange exchange) { return Mono.justOrEmpty(exchange.getRequest().getHeaders().getFirst("X-API-Key")); } } |
With this, you can easily add a rate-limiting capability to your microservice without having to implement this within the service itself.
Adding a Global Namespace to Various Microservices
As services mature, it might be necessary to do the step from version 1 to version 2. But implementing the new version in the same deployable unit implies the risk — while implementing on version 2, you might accidentally change something related to version 1. So why not leave version 1 as it is and implement the new version as a separate deployment unit?
A simple configuration can help bring these two applications under a common name to seem as though they are one unit of deployment.
|
1 2 3 4 5 6 7 8 9 10 11 12 |
spring: cloud: gateway: routes: - id: api-v1 uri: http://v1api.example.com/ predicates: - Path=/v1/** - id: api-v2 uri: http://v2api.example.com/ predicates: - Path=/v2/** |
This allows us to access our microservice with the URLs https://api.example.com/v1/* and https://api.example.com/v2/* but have them be deployed and maintained separately.
Scale Subcontext of Your Services Independently
In the previous tip, we showed how to bring together components that need to be together. But this can extend to other scenarios as well. So let’s assume our microservice is simply too big to be managed by a single team. We can use the same configuration to unite different logical parts of the service into one common umbrella to look like one service.
|
1 2 3 4 5 6 7 8 9 10 11 12 |
spring: cloud: gateway: routes: - id: locations uri: http://locationapi.example.com/ predicates: - Path=/v1/locations/** - id: weather uri: http://weatherapi.example.com/ predicates: - Path=/v1/weather/** |
This would also allow us to independently scale the different parts of the API as needed.
AB Test Your Service with a Small Customer Group
AB testing is common when developing new applications since it is a good way of testing whether your service meets requirements without having to roll it out completely. AB testing gives only a small group of users access to the new version of a service and asks them whether they like it. One group could be your colleagues within the company network, for example.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
spring: cloud: gateway: routes: - id: example-a uri: http://example.com/service-a predicates: - Header=X-AB-Test, A - RemoteAddr=192.168.1.1/24 - id: example-b uri: http://example.com/service-b predicates: - Header=X-AB-Test, B - RemoteAddr=192.168.10.1/24 - id: example-default uri: http://example.com/service-default predicates: - True |
In this example, the gateway evaluates all the routes and their predicates defined in this order, and if all the predicates match, the corresponding route will be selected. This means:
- Customers coming from 192.168.1.1/24 network having the X-AB-Test header set to A will be presented service variant a.
- Customers coming from 192.168.10.1/24 network having the X-AB-Test header set to B will be presented service variant b.
- All other customers will see that the service-default variant as the corresponding predicate always evaluates to true.
Protect Services by Adding Authentication
This is not really a specific Spring Cloud Gateway feature, but it’s also handy in combination with the gateway. Imagine you have blackbox services and you want to enhance its implementation without having the source code or the permission to change this.
In this case, a gateway as a reverse proxy in front can help add features like authentication.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
spring: security: oauth2: resourceserver: jwt: issuer-uri: https://client.idp.com/oauth2/default audience: api://default cloud: gateway: routes: - id: api uri: http://api.example.com/ predicates: - Path=/** |
As seen here, the gateway parts take care of the traffic forwarding to the target and the Spring Security configuration adds an oauth2 flow for logging in.
Add Audit Logging to Your Services
Another scenario for enhancing existing services might be adding an audit log to the service.
Want to know who’s calling which of your services operations? Add some audit logging. This needs a bit more implementation as I haven’t found a default implementation.
|
1 2 3 4 5 6 7 8 9 10 |
spring: cloud: gateway: routes: - id: backend uri: http://backend.example.com predicates: - Path=/** filters: - name: RequestLogger |
The RequestLogger can be implemented as a Spring bean, just like this:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
@Component public class RequestLogger implements GatewayFilter, Ordered { private static final Logger LOGGER = LoggerFactory.getLogger(RequestLogger.class); @Override public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) { ServerHttpRequest request = exchange.getRequest(); HttpMethod method = request.getMethod(); String uri = request.getURI().toString(); HttpHeaders headers = request.getHeaders(); String authHeader = headers.getFirst(HttpHeaders.AUTHORIZATION); LOGGER.info("Request - method: {}, uri: {}, authHeader: {}", method, uri, authHeader); return chain.filter(exchange); } @Override public int getOrder() { return Ordered.HIGHEST_PRECEDENCE; } } |
In this implementation, the RequestLogger filter logs the HTTP method, Uniform Resource Identifier (URI) and authentication header of each request using the SLF4J logging framework. The filter is implemented as a Spring @Component and is added to the Spring Cloud Gateway filter chain using the filters property in the application.yml file. The Ordered interface is implemented to ensure that this filter has the highest precedence and runs first in the filter chain.
Protect Services by a Circuit Breaker
In some cases, rate limiting, as discussed previously, is not enough to operate a service safely, and if response times for your service get too high, it’s sometimes necessary to cut off traffic for a short period of time to let the service recover itself. This is where all the classical resilience patterns can help. A pattern like the circuit breaker has to be implemented outside of the affected service. The Spring Cloud Gateway can also help here.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 |
spring: cloud: gateway: routes: - id: slow-service uri: http://example.com/slow-service predicates: - Path=/slow/** filters: - name: CircuitBreaker args: name: slow-service fallbackUri: forward:/fallback/slow-service statusCodes: - SERVICE_UNAVAILABLE - name: ResponseTime args: baseName: slow-service timeout: 1000 tripwires: - id: slow-response type: MAX_RESPONSE_TIME threshold: 500 circuitBreaker: enabled: true timeout: 10000 ringBufferSizeInClosedState: 5 ringBufferSizeInHalfOpenState: 2 failureRateThreshold: 50 - id: fast-service uri: http://example.com/fast-service predicates: - Path=/fast/** - id: fallback-slow-service uri: forward:/fallback/slow-service predicates: - Path=/fallback/slow/** |
In this example, calls to the slow service will be monitored and if its response time exceeds 1,000 milliseconds, it will be cut off and given 10 seconds to rest before trying to bring it back in. In the meantime, the fallback is used. This fallback can be a simple error message, sending a “Temporarily Unavailable” status code or maybe some more helpful implementation, depending on the use case.
More Creative Ways of Using Spring Cloud Gateway
Like Legos, there are endless possibilities, combining all these building blocks to build whatever is needed. One of the most creative ways of using this that I have seen is the following:
A team encountered a challenge when using autoscaling in combination with Java applications. They realized that newly started applications were much slower than those that were already running. While this is a normal behavior for Java applications due to the just-in-time (JIT) compiling process, it affects end-user experience. The team added a Spring Cloud Gateway for load balancing to all these services and configured it as load balancing based on the average response time. So backends with fast average response times would get more traffic than backend instances with slower average response times. Overall, this allowed freshly started instances to warm up their JIT without negatively affecting overall performance, and it also helped reduce traffic to overloaded instances, leading to better performance overall for the end user.
As you can see, there are a lot of possibilities where this highly flexible API gateway can be used in a wide range of products. Due to its extensibility, you can easily add new features and capabilities by simply implementing custom filters or predicates and letting the traffic flow just the way you need. That’s why this is one of my favorite tools for shaping microservice landscapes.
