Energy-Efficient Secure Inference for Neural Networks on Edge Devices
Keywords:
energy-efficient secure inference, edge computing, privacy-preserving machine learning, two-party computation, homomorphic encryption, system architecture, sustainability, governanceAbstract
Secure inference for neural networks on edge devices sits at the intersection of cryptographic protection, constrained computation, and operational sustainability. Edge devices increasingly perform or assist with inference because they reduce latency, preserve local context, and lessen dependence on centralized cloud infrastructure. Yet privacy-preserving mechanisms such as homomorphic encryption, secure multi-party computation, and trusted execution environments impose substantial energy, memory, and communication costs. This paper examines the system-level trade-offs that shape energy-efficient secure inference, with attention to architecture, hardware heterogeneity, deployment governance, robustness, fairness, and policy. It argues that secure inference should not be treated as a standalone cryptographic layer but as a socio-technical infrastructure problem. The analysis considers how model compression, quantization, protocol design, and edge-cloud partitioning interact with security guarantees and energy budgets. It also explores how measurement practices, supply chains, standards, and regulatory expectations influence adoption. The paper concludes that durable progress requires co-design across algorithms, protocols, hardware, and institutions, and that energy efficiency must be evaluated alongside privacy, accuracy, resilience, and equity rather than as a substitute for them.
References
1. Satyanarayanan, M. (2017). The emergence of edge computing. Computer, 50(1), 30-39. https://doi.org/10.1109/MC.2017.9
2. Alrawais, A., Alhothaily, A., Hu, C., & Cheng, X. (2017). Fog computing for the Internet of Things: Security and privacy issues. IEEE Internet Computing, 21(2), 34-42. https://doi.org/10.1109/MIC.2017.37
3. Zhang, C., Patras, P., & Haddadi, H. (2019). Deep learning in mobile and wireless networking: A survey. IEEE Communications Surveys & Tutorials, 21(3), 2224-2287. https://doi.org/10.1109/COMST.2019.2904897
4. Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017). MobileNets: Efficient convolutional neural networks for mobile vision applications. arXiv. https://arxiv.org/abs/1704.04861
5. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L. C. (2018). MobileNetV2: Inverted residuals and linear bottlenecks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 4510-4520. https://doi.org/10.1109/CVPR.2018.00474
6. Han, S., Mao, H., & Dally, W. J. (2016). Deep compression: Compressing deep neural networks with pruning, trained quantization and Huffman coding. International Conference on Learning Representations.
7. Jacob, B., Kligys, S., Chen, B., Zhu, M., Tang, M., Howard, A., Adam, H., & Kalenichenko, D. (2018). Quantization and training of neural networks for efficient integer-arithmetic-only inference. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2704-2713. https://doi.org/10.1109/CVPR.2018.00286
8. Yao, A. C. C. (1986). How to generate and exchange secrets. Proceedings of the 27th Annual Symposium on Foundations of Computer Science, 162-167. https://doi.org/10.1109/SFCS.1986.25
9. Goldreich, O., Micali, S., & Wigderson, A. (1987). How to play any mental game. Proceedings of the 19th Annual ACM Symposium on Theory of Computing, 218-229. https://doi.org/10.1145/28395.28420
10. Gentry, C. (2009). Fully homomorphic encryption using ideal lattices. Proceedings of the 41st Annual ACM Symposium on Theory of Computing, 169-178. https://doi.org/10.1145/1536414.1536440
11. Gilad-Bachrach, R., Dowlin, N., Laine, K., Lauter, K., Naehrig, M., & Wernsing, J. (2016). CryptoNets: Applying neural networks to encrypted data with high throughput and accuracy. International Conference on Machine Learning, 201-210.
12. Mohassel, P., & Zhang, Y. (2017). SecureML: A system for scalable privacy-preserving machine learning. 2017 IEEE Symposium on Security and Privacy, 19-38. https://doi.org/10.1109/SP.2017.12
13. Liu, J., Juuti, M., Lu, Y., & Asokan, N. (2017). Oblivious neural network predictions via MiniONN transformations. Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, 619-631. https://doi.org/10.1145/3133956.3134056
14. Juvekar, C., Vaikuntanathan, V., & Chandrakasan, A. (2018). GAZELLE: A low latency framework for secure neural network inference. Proceedings of the 27th USENIX Security Symposium, 1651-1669.
15. Peng, H., Zhou, S., Luo, Y., Xu, N., Duan, S., Ran, R., ... & Ding, C. (2023). Rrnet: Towards relu-reduced neural network for two-party computation based private inference. arXiv preprint arXiv:2302.02292.
16. Knott, B., Venkataraman, S., Hannun, A., Sengupta, S., & van der Maaten, L. (2021). CrypTen: Secure multi-party computation meets machine learning. Advances in Neural Information Processing Systems, 34.
17. Mohassel, P., & Rindal, P. (2018). ABY3: A mixed protocol framework for machine learning. Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, 35-52. https://doi.org/10.1145/3243734.3243760
18. Rathee, D., Rathee, M., Kumar, N., Chandran, P., Gupta, A., Rastogi, A., & Sharma, R. (2020). CrypTFlow2: Practical 2-party secure inference. Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security, 325-342. https://doi.org/10.1145/3372297.3417274
19. McMahan, B., Moore, E., Ramage, D., Hampson, S., & y Arcas, B. A. (2017). Communication-efficient learning of deep networks from decentralized data. Artificial Intelligence and Statistics, 1273-1282.
20. Strubell, E., Ganesh, A., & McCallum, A. (2019). Energy and policy considerations for deep learning in NLP. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 3645-3650. https://doi.org/10.18653/v1/P19-1355
21. Schwartz, R., Dodge, J., Smith, N. A., & Etzioni, O. (2020). Green AI. Communications of the ACM, 63(12), 54-63. https://doi.org/10.1145/3381831
22. Wu, C. J., Brooks, D., Chen, K., Chen, D., Choudhury, S., Dukhan, M., ... & Zhang, P. (2019). Machine learning at Facebook: Understanding inference at the edge. 2019 IEEE International Symposium on High Performance Computer Architecture, 331-344. https://doi.org/10.1109/HPCA.2019.00048
Downloads
Published
Issue
Section
License
Copyright (c) 2026 International Journal of Artificial Intelligence Engineering and Systems
This work is licensed under a Creative Commons Attribution 4.0 International License.
This article is published under the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
