Скачать или смотреть How to Efficiently Get Data from a Reactive Microservices System

Learn how to effectively retrieve data from a `reactive microservices system` using asynchronous communication with Kafka and Spring WebFlux.
---
This video is based on the question https://stackoverflow.com/q/63824675/ asked by the user 'Artsiom the Brave' ( https://stackoverflow.com/u/9253292/ ) and on the answer https://stackoverflow.com/a/63855285/ provided by the user 'Levi Ramsey' ( https://stackoverflow.com/u/5641244/ ) at 'Stack Overflow' website. Thanks to these great users and Stackexchange community for their contributions.

Visit these links for original content and any more details, such as alternate solutions, latest updates/developments on topic, comments, revision history etc. For example, the original title of the Question was: How do you get data from a reactive microservices system?

Also, Content (except music) licensed under CC BY-SA https://meta.stackexchange.com/help/l...
The original Question post is licensed under the 'CC BY-SA 4.0' ( https://creativecommons.org/licenses/... ) license, and the original Answer post is licensed under the 'CC BY-SA 4.0' ( https://creativecommons.org/licenses/... ) license.

If anything seems off to you, please feel free to write me at vlogize [AT] gmail [DOT] com.
---
How to Efficiently Get Data from a Reactive Microservices System

In the modern era of software development, designing a reactive microservices system that communicates asynchronously can be quite challenging. This is especially true when dealing with multiple services that need to interact in a complex process. One common use-case involves managing a cluster of virtual machines (VMs) where the system needs to process requests such as creating a new VM.

This guide discusses how to effectively get data from a reactive microservices system, specifically through asynchronous communication using Apache Kafka, while leveraging Spring WebFlux as a framework.

Understanding the Problem

Imagine the following scenario:

A user sends a request to create a new VM.

The API Gateway receives the request, which then goes through a series of steps:

Security Validation

Creation of the VM in the user space

Scheduling resources for the VM

Sending a notification to the user once the operation is successful or failed

While all these steps might seem sequential, they operate asynchronously, thanks to Kafka. This asynchronous method makes it tricky, especially when clients want to issue GET requests to retrieve information about VMs or the cluster state.

Here lies the challenge: how do we retrieve data in such an environment efficiently?

Potential Solutions

In order to effectively manage GET requests, two potential solutions can be considered:

Synchronous GET Requests:

The API Gateway can make synchronous GET calls to other services to aggregate data and return a complete response to the client.

This method can lead to increased latency and defeats the purpose of having an asynchronous system.

Dedicated Data Service:

Instead of retrieving data with each request, the system can aggregate and store user-queryable data in a dedicated service (a database).

The API Gateway then forwards GET requests to this service, allowing for quick responses without needing to engage multiple microservices on each request.

A Better Approach: Using a Saga Management Service

While the above solutions have merit, a more efficient approach would involve the use of a Saga Management Service. Here’s how it works:

Step-By-Step Process:

API Gateway: Upon receiving a request, it forwards the request to the Saga Management Service.

Saga Management Service: This service initiates the workflow by notifying the Security Validation Service.

Security Validation Service: Once validated, it publishes an event.

VM Creation Service: Following the event from the security validation, the Saga Management Service may notify this service to create the VM.

Create VM: This service creates the VM in the userspace and publishes the corresponding event.

Resource Scheduler: Based on the VM creation event, the Saga Management may notify the resource scheduler service.

Schedule Resources: This service schedules the necessary resources and publishes another event.

Final Notification: Ultimately, the Saga Management Service sends a notification regarding the success or failure of the VM creation and resource allocation.

Benefits of Using a Saga Management Service:

Failure Response: The saga manager can handle failures effectively by rolling back actions if necessary or communicating with the appropriate services to check their status.

Event-Rich Responses: This approach allows for publishing detailed events that can help other services, like those responsible for listing VMs, engage with rich contextual information rather than just notifications.

Interactive Requests: The Saga Management Service can respond to interactive requests without clogging the individual microservices with synchronous interactions.

Conc