Results

Overview

Diagrams below exemplify executions of each of the compared approaches for the SockShop application. Rectangular boxes at the top of each diagram represent service combinations. Circles represent VM types, which are sorted by their cost. The color of each circle describe the execution state:

1. Green circle: executed experiments where VM type meets the performance target.
2. Red circle: executed experiments where VM type does not meet the performance target.
3. Orange circles: VM types that are determined not to meat the performance target because of Condition 1.
4. White circles: are not executed VM types.

SF :

SF₁ :

SF₂ :

SF₃ :

P :

Kuber :

Detailed results for each of the subject applications, which correspond to Figure 4 in the paper, can be found in the attached spreadsheets.

1. Hotel Reservation Search Cost
2. Media Microsvc Search Cost
3. Social Network Search Cost
4. Sockshop Search Cost
5. Hotel Reservation Execution Time
6. Media Microsvc Execution TIme
7. Social Network Execution Time
8. Sockshop Execution Time

Execution Time Breakdown for Kuber

The table below shows time spent by each of the approaches in each of the phases (setting up VMs for the experiments, executing the experiments, and running the WIP algorithm). While KUBER take 53 hours on average (the last column), the other three approaches execute for hundreds of hours on average. The total execution time of all the experiments is more than four months.

Utilization of VMs by Application Test

To make sure our VM selection and configuration are appropriate for the selected subject applications and their workload, we run each service on each VM type in isolation. Table below shows the average CPU and memory utilization on the smallest and largest VM types, VM1 and VM11, respectively. As none of our VMs are overloaded by a single service, we believe our experiments / VM selection is appropriate.