In recent years, the deployment of deep learning models on resource-constrained edge devices, such as smartphones, wearable devices, IoT devices, edge servers, and embedded systems has increased exponentially, posing challenges due to their limited computational power and memory space.
The study presented here, aimed to employ TinyML (tiny machine learning) techniques for compressing Custom, VGG16, and MobileNet models across datasets taken from the fashion, radiology, and dermatology fields. It prioritizes the following three features: achieving optimal accuracy, reduced inference time, and minimized model size suitable for deployment on resource-constrained edge devices.
The main compression techniques applied here are quantization and pruning. Quantization makes numbers in the model much less precise, while pruning selectively eliminates redundant or unnecessary connections within the neural network architecture without compromising the model too much. This helps in reducing the computational cost and memory usage for optimal and quick deployment and is adjusted to the requirements of TinyML.
For our study, which involved meticulous testing of different permutations and combinations on the datasets, our focus was to tweak the models and parameters to increase accuracy, inference time, and model size. Further, this endeavor leverages quantization techniques from TensorFlow Lite compression to investigate how TinyML might facilitate edge computing and improve user experience. It makes it easier to implement effective and lightweight models, allowing for real-time inference on edge devices with limited resources.
Tiny machine learning or TinyML is a decentralized technique that allows us to deploy machine learning models and algorithms on extremely low-power and small-footprint devices such as microcontrollers and IoT devices. With TinyML, microcontroller-based embedded devices can quickly respond in real time to machine-learning tasks.
Its applications span multiple disciplines of technology such as hardware, algorithms, and software capable of processing on-device sensor data.
The most widely adopted ecosystem for TinyML development is TensorFlow Lite. TensorFlow Lite provides a Python-based environment with an extensive collection of pre-installed libraries and toolkits that offer developers ways to effectively implement machine learning models
The general process followed for deployment on edge devices is shown below:
Data Set Optimization Domains: Fashion, Radiology, and Dermatology
Optimization algorithms were implemented using datasets from a variety of fields such as fashion, radiology, and dermatology.
Utilizing diverse datasets provides important insights into the effectiveness and versatility of optimization methods. Each dataset has different challenges, traits, and subtleties that call for investigating a large range of optimization strategies. Through the integration of diverse datasets, we can acquire a thorough understanding of the performance of the different optimization approaches.
To optimize models, we first executed a script on the data of the respective dataset to gain familiarity with its characteristics. Using TensorFlow Lite, quantization and pruning techniques were employed to meet our primary imperatives of good accuracy, low inference time, and reduced model size.
A. Fashion: FashionMNIST optimization
For the fashion domain, we optimized the models trained on the FashionMNIST dataset.
Fashion datasets contain images of clothing items, accessories, and fashion products.
| FashionMNIST Dataset Specifications | Dataset Information |
|---|---|
| Dataset | Contains 70,000 grayscale images in 10 categories. The images show individual articles of clothing at low resolution (28 by 28 pixels). |
| Train | 60,000 images |
| Test | 10,000 images |
| Classes | 0-9 |
| Class names (labels) | 'T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat', 'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot' |
B. Radiology: Pneumonia dataset optimization
Radiology datasets comprise medical images such as chest X-rays, CT scans, and radiology function test results. For example, machine learning models trained on radiology datasets can analyze chest X-ray images to detect abnormalities such as pneumonia, tuberculosis, or lung cancer, assisting radiologists in accurate diagnosis and treatment planning.
For this dataset, we first organized and prepared a dataset comprising the chest X-ray images of two types of patients: normal and with pneumonia. Then, we resized images to different dimensions (150x150, 1000x1000, and 224x224) to show how well models can adjust. After that, we trained models using both custom and pre-trained approaches, noting how different model setups performed.
| Radio logy Dataset Specifications | Dataset Information |
|---|---|
| Dataset | Chest X-ray images |
| Train | 5216 images (Normal: 1341, Pneumonia: 3875 |
| Test | 624 images (Normal: 234, Pneumonia: 390) |
| Validation | 16 images (Normal: 8, Pneumonia: 8 |
| Classes | 0-1 |
| Class names (labels) | 'Normal', 'Pneumonia' |
C. Dermatology: Skin lesion dataset optimization
Dermatology: Dermatology datasets typically consist of images of skin conditions, lesions, and diseases.
We began by sampling a skin lesion dataset, which included multi-source dermatoscopic images of pigmented lesions. We organized and prepared the dataset through pre-processing to ensure readiness for analysis. Like other datasets, we resized the images to 224x224, demonstrating the adaptability of the models to specific image dimensions. For training, we utilized pre-trained models such as MobileNetV2, and various configurations were assessed to gauge performance metrics.
| Skin Lesion Dataset Specifications | Dataset Information |
|---|---|
| Dataset | Dermatoscopic images |
| Train | 6585 images |
| Test | 840 images |
| Classes | 0-6 |
| Class names (labels) | (AKIEC) Actinic Keratosis’, ‘Basal Cell Carcinoma (BCC)’, ‘Benign Keratosis (BKL)’, ‘Dermatofibroma (DF)’, ‘Melanocytic nevus (NV)’, ‘Melanoma (MEL)’, ‘Vascular lesion (VASC) |
Models Utilized Across Diverse Domains:
| Domain | Dataset | Models Used |
|---|---|---|
| Fashion | FashionMNIST dataset | Custom, VGG16, MobileNET |
| Radiology | Pneumonia dataset | Custom, VGG16, MobileNET |
| Dermatology | Skin lesion dataset | Custom, VGG16, MobileNET |
Srinivas has over 25 years of experience which spans across Consumer Electronics, Biomedical Instrumentation and Medical Imaging. He has led research and development teams, focused on end-to-end 3D/4D quantification applications and released several "concept to research to market" solutions. He also led a cross functional team to drive applied research, product development, human factors team, clinical research, external collaboration and innovation. He has garnered diverse sets of skill sets and problem challenges. and has over 25 Patent filings and 12 Patent Grants across varied domains, mentored over 30+ student projects, been a guide for over 10+ master thesis students, peer reviewer for papers and an IEEE Senior Member (2007).
Malavika is an Electronics and Instrumentation engineer with experience in biomedical Instrumentation. She also brings expertise in business analysis within supply chain management incorporating the tools in data analytics to devise solutions to optimize and track KPIs. She predominantly contributes to shaping impactful solutions for customer-facing endeavors in technology and business domains
https://keras.io/api/applications/vgg/
https://keras.io/api/applications/mobilenet/
For more information on this Whitepaper, please feel free to get in touch with Srinivas Rao Kudavelly at srinivasrao.kudavelly@cyient.com
The advent of 5G technology will revolutionize global farming landscapes and will open up multiple ways to establish and grow precision farming. The figure below shows that every element in modern agriculture once connected to a high speed and high throughput 5G cellular network, works in tandem with the other to optimize resources and maximize yield. The imagery generated from SAR and GPR demand throughput for transferring them to a distant and central location/data cloud. Similarly, to control farming equipment remotely, a low latency communications network in inevitable.
Figure 10. Uses of 5G technology in agriculture
Vishnu Gaddam, an aerospace professional with Cyient since 2011, brings over 15 years of experience in the field. His journey began at GE Aviation, where he assessed both small and large aero engines for static and dynamic behavior. At Cyient, Vishnu works on complete aero engines and evaluates loads under normal and extreme operational conditions. He is a certified Project Management Professional (PMP), Certified Scrum Master (CSM), and Six Sigma Green Belt. As a subject matter expert, Vishnu collaborates across teams, fostering knowledge exchange.
