Dynamic Pruning for Distributed Inference via Explainable AI: A healthcare use case

The healthcare sector has undergone a significant transformation with the widespread adoption of Deep Neural Networks (DNN). However, due to privacy constraints and stringent latency requirements, online remote inference is not a viable option in healthcare scenarios. Many efforts have been conducted to enable local computation, such as network compression using pruning or DNN distribution among multiple resource-constrained devices. Yet, it is still challenging to conduct distributed inference due to the latency and energy overheads resulting from intermediate shared data. On the other hand, given that realistic healthcare systems use pre-trained models, local pruning and fine-tuning relying only on the scarce and biased data is not possible. Even pre-pruned DNNs are not efficient as they are not customized to the local load of data and the dynamics of devices. The dynamic and online pruning of DNN without fine-tuning is a promising solution; however, it was not considered in the literature as most well-known techniques do not perform well without adjustment. In this paper, driven by the data restrictions in healthcare sector, we propose a novel pruning strategy based on Explainable AI (XAI), with a target to enhance the pruned DNN performance without fine-tuning. Moreover, to maintain the highest possible accuracy, we propose to combine distribution and pruning techniques to perform online distributed inference assisted by dynamic pruning only when needed. Our experiments show the performance of our pruning criterion compared to other reference techniques, in addition to its ability to assist the distribution by reducing the shared data, while keeping high accuracy.