Blueprint Experiments
This repository contains scripts and instructions for running the experiments from "Blueprint: A Toolchain for Highly-Reconfigurable Microservice Applications".
Important other repositories:
- blueprint-compiler: The main Blueprint repository containing the source code for the Blueprint compiler.
- blueprint-dsb-sn: Blueprint's version (workflow spec + wiring spec) of DeathStarBench-SocialNetwork. This corresponds to BP-DSB-SN in the paper.
- blueprint-dsb-hr: Blueprint's version (workflow spec + wiring spec) of DeathStarBench-HotelReservation. This corresponds to BP-DSB-HR in the paper.
- blueprint-dsb-mm: Blueprint's version (workflow spec + wiring spec) of DeathStarBench-MediaMicroservices.
- blueprint-trainticket: Blueprint's version (workflow spec + wiring spec) of TrainTicket.
- blueprint-sockshop: Blueprint's version (workflow spec + wiring spec) of SockShop.
Abstract
Blueprint is a system for generating variant implementations of microservice systems. Blueprint is open-source and available at blueprint-compiler; our artifact evaluation plan will reference that repository numerous times. This repository contains the scripts needed to run the experiments presented in the paper. We give an overview of how to evaluate Blueprint, describe the workflow of Blueprint experiments, and provide descriptions of how to run each experiment. The primary software artifact is the system itself and is located at blueprint-compiler. The artifact is further supplemented by Blueprint-compatible implementations of various open-source microservice applications available at blueprint-systems. Each application has its own repository that we have already listed above.
Roadmap
Setup
Please follow the instructions in the README of the ae_ktr folder to install dependencies and build blueprint.
If access is possible to MPI machines, then we will pre-configure everything ahead of time on your behalf.
Machines
All experiments used a cluster com- prising eight machines, each with four Intel Xeon E7-8857V2 processors, 48 cores and 1.5 TB RAM. Each service was deployed in a separate container.
However, to make the artifact evaluation task simpler, we provide experimental scripts for a 2-node deployment where the machine producing the workload will be referred to as $MACHINE1 and the machine where the containers are deployed will be referred to as $MACHINE2 in the rest of the documentation.
DISCLAIMER: Depending on how powerful the machines are, you might need to play around with the request rate in a given workload to get the system to produce metastability failures.
Experiments
Experiments can be run using the scripts provided in each experiment's subfolder.
The repository is structured to aid in the reproduction of results from the paper. The following table summarizes different experiments in the paper and the directory containing the respective experiment related information.
| Experiment Name / Section / Paragraph | Related Figures, Tables | Experiment Directory |
|---|---|---|
| AE Kick-the-tires | N/A | ae_ktr |
| 6.1 Reducing Implementation Effort | Tab. 1 | tab1 |
| 6.1 Performance-Driven Design Exploration | Fig. 5 | fig5 |
| 6.2.1 Type 1 Metastability Failure | Fig. 6a | fig6a |
| 6.2.1 Type 2 Metastability Failure | Fig. 6b | fig6b |
| 6.2.1 Type 3 Metastability Failure | Fig. 6c | fig6c |
| 6.2.1 Type 3 Metastability Failure Vulnerability Analysis | Fig. 7 | fig7 |
| 6.2.1 Type 4 Metastability Failure | Fig. 6d | fig6d |
| 6.2.2 Cross-System Inconsistency | Fig. 8 | fig8 |
| 6.3 Reproducible Research | Fig. 9 | fig9 |
| 6.3 Prototyping New Solutions | Fig. 10 | fig10 |
| 6.4 Hotel Reservation | Fig. 11 | fig11 |
| 6.4 Social Network | Fig. 11 | fig11 |
| 6.5 Adding new backends and instantiations | Tab. 2, Tab. 3 | tab2, tab3 |
| 6.5 Adding new plugins | Tab. 4 | tab4 |
Each experiment subdirectory contains the following:
- A README file detailing the exact steps needed to generate the corresponding result.
- List of systems and variants that will be compiled.
- Scripts to generate the pertinent graphs and/or tables for the experiment.
- Recorded results that were used to generate graphs in the accepted version of the paper.