There are more and more scenes that use SDK that calls an external service, such as Azure Cosmos DB, AWS S3, or a service that sends faxes.
In the world where we have lived normally, we only have to worry about the life and death of the database server we manage, and the network distance is very close. Everything was within expectations, but it's not as simple as using these external services.
Network delays are not stable, and sometimes delays in seconds can occur. Many services are provided with endpoints by host name, and IP addresses change without notice.
When such an unstable event occurs, threads that call each service as HTTP and block it may suddenly accumulate, eventually causing the entire service to stop.
In other words, for blocking calls, be careful not to fill the main HTTP thread with a timeout in the function call.
With Ruby, it's relatively easy to write.
require 'timeout'
def callAnotherWorld
sleep 3
return "The result of calling some external service"
end
begin
result = Timeout.timeout(1) do #Time out in 1 second
callAnotherWorld()
end
rescue Timeout::Error
result = ""
end
p result #Blank because it times out
The code is almost the same in Java.
private final ExecutorService threadPool = Executors.newCachedThreadPool();
private String callAnotherWorld() {
Future<String> f = threadPool.submit(() -> innerCallAnotherWorld());
while (true) {
try {
f.get(1, TimeUnit.SECONDS); //Time out in 1 second
} catch (ExecutionException e) {
throw new RuntimeException(e);
} catch (InterruptedException retry) {
}
}
}
private String innerCallAnotherWorld() {
return "The result of calling some external service";
}
In both Ruby and Java, the method is executed in a background thread, and the calling thread waits for its completion.
In the case of Java, the existence of the thread pool and the existence of Future have been seen, which makes me feel a little disappointed.
Resilience4J
So Resilience4J.
Resilience4J is a recent my boom that makes it easy to use the following patterns to improve system stability in distributed architectures:
To realize the timeout for the method call, use TimeLimiter. How to use CircuitBreaker will be a hit soon, but there is not much information about TimeLimiter, so I will write it as a memorandum for myself.
private final ExecutorService threadPool = Executors.newCachedThreadPool();
private final TimeLimiterConfig config = TimeLimiterConfig.custom()
.timeoutDuration(Duration.ofMillis(1000))
.cancelRunningFuture(true)
.build();
private String callAnotherWorld() {
Callable<String> callable = TimeLimiter
.decorateFutureSupplier(
TimeLimiter.of(config),
() -> threadPool.submit(() -> innerCallAnotherWorld())
);
return Try.of(callable::call)
.getOrElse("");
}
It won't be too short, but it will eliminate catch and exception handling, and the code will focus on business logic.
The functions provided by Resilience4J are essential design patterns in today's distributed environment, and are very educational.