Input-Dependent Power Usage in GPUs

GPUs are known to be power-hungry, and given the boom in artificial intelligence, are currently the major contributors to high power demands of upcoming datacenters. Most GPU usage in these popular workloads consists of large matrix multiplications (GEMMs). The arithmetic intensity of various GEMM shapes and sizes has been deeply studied in the community. We show that modifying the input data, while maintaining data shapes and sizes, in GPU computation kernels can notably change their power consumption. We experiment with four kinds of input variations: value distribution, bit similarity, placement, and sparsity across different data types. Our findings indicate that these variations can change the GPU power usage during GEMM by almost 40%. We hypothesize that input-dependent power usage variations occur due to changes in the number of bit flips in the GPUs. We propose leveraging this property through compiler and scheduler
optimizations to manage power and reduce energy consumption.