Enhancing Mobile AI: Feature-Based Knowledge Distillation for Efficient Image Classification

Author First name, Last name, Institution

• Luca Canala, The University of Adelaide
• Muhammad Uzair, The University of Adelaide
• Faheem Ullah, Zayed UniversityFollow
• Babar Shah, Zayed UniversityFollow

Document Type

Conference Proceeding

Source of Publication

Lecture Notes in Networks and Systems

Publication Date

7-2-2026

Abstract

Model compression is essential in machine learning for reducing the size and computational demands of deep neural networks. Techniques such as knowledge distillation and low-rank factorization enable the deployment of efficient models on resource-constrained devices like smartphones. While label-based distillation is well-explored, feature-based methods, which transfer intermediate representations from a larger teacher model to a smaller student model, are less studied. This paper focuses on exploring feature-based knowledge distillation for image classification tasks. From our experiments we were able to produce a significantly more compressed student model with accuracy close to the teacher model, which is able to run up to 13 times faster than the teacher model on resource-limited CPU-only devices.

DOI Link

10.1007/978-3-032-21174-3_31

ISBN

[9783032211736]

ISSN

2367-3370

Publisher

Springer Nature Switzerland

Volume

1889 LNNS

First Page

339

Last Page

348

Disciplines

Computer Sciences

Keywords

Image classification, Knowledge Distillation, Model Compression

Scopus ID

105047672421

Recommended Citation

Canala, Luca; Uzair, Muhammad; Ullah, Faheem; and Shah, Babar, "Enhancing Mobile AI: Feature-Based Knowledge Distillation for Efficient Image Classification" (2026). All Works. 8024.
https://zuscholars.zu.ac.ae/works/8024

Indexed in Scopus

yes

Open Access

no