Serving layer

The core task of the serving layer is to expose the views created by both the batch and speed layer for querying by other systems or users.

Apart from this, there is a good amount of orchestration work done by this layer. The speed layer needs to keep looking at the batch layer to see if the necessary batch operation is done. If so, the batch layer store needs to be updated soon after the batch operation is conducted from the batch view. Soon after this, the existing speed layer view will be discarded and this is also a bookkeeping activity which keeps the speed layer store size in control.

The following figure shows the two layers namely batch and serving in action:

Figure 09: Lambda Architecture - serving layer

When a query comes to the serving layer, it in turn forms and fires two queries to the two different layers namely batch and speed. The queries fired look into the timestamp of records and fetches the results according to the parameters supplied. Once it gets the records from both the layers, it merges and applies various application logic and produces the final output in the desired format by the system or user. Because of the quality of data, the batch layer is more reliable (recomputation aspect when each batch operation is completed), in case of conflicts, the batch view results overrides that of speed layer.