GUESS Introduction

We propose the GUESS system to allow users to visualize the performance and energy consumption of Spark applications when running in serverless or serverful environments. We experimentally evaluate the performance and energy consumption using the TPC-H database benchmark with 22 join query workloads on big data platforms. The GUESS system aims to:

• (1) Visualize the performance and energy consumption of Spark applications when deployed in serverless and serverful environments;
• (2) Compare the differences in the initialization, data processing, and resource reclamation phases of Spark applications when running in serverless versus serverful environments.

Demo Access

To make it more visual, we have posted a brief demo video on YouTube for access.

URL: https://youtu.be/fC8ABz3P3xE

Web Application for Visualization

Visualize the performance

Visualize the energy consumption

Compare the differences in the initialization, data processing, and resource reclamation phases

Article

For more information, please see the article "GUESS: MONITORING JOIN QUERY EXECUTION IN SERVERLESS AND SERVERFUL SPARK".