Speech and hand gestures are among the primary means humans use to communicate with each other. Communication, however, can become challenging for people with speech and hearing impairment. While the latter use sign language to communicate, it is difficult for common people to understand.
This white paper presents a project to simplify communication for the differently abled by translating gestures and signs into text or sounds on an LCD display. This is done by means of a data glove worn by an individual, who proceeds to use the hand to make gestures or signs to communicate. The data glove is fitted with flex sensors along each finger. These sensors detect the bending of the fingers, with each degree of bending resulting in a change in resistance that is converted into voltage. The system is designed to recognize these specific changes in voltage corresponding to different gestures and convert them into voice or text.
A gesture in sign language comprises movement of fingers and tilting of the hand in specific directions with specific angles. Sign language consists of different signs for different words. The proposed idea involves using these flex sensors to translate sign language into speech according to the bending of fingers, facilitating communication between the hearing- and speech-impaired and those who can hear and speak normally.
An ATmega328 controller interfaces this gesture-to-voice/text communication. For people who cannot hear, the converted text is displayed on an LCD screen. For those who cannot see, the system can be designed such that it provides speech output to communicate the message. This dual-mode output ensures that both the hearing-, speech- and visually impaired can benefit from the device. Additionally, the system can be dynamically reconfigured to work as a “smart device” that can adapt to all kinds of sign languages.
Random forest is a powerful machine learning algorithm used for both classification and regression tasks. It is an ensemble learning method that involves combining multiple decision trees to create a more robust and accurate model.
The fundamental idea behind random forest is to generate multiple decision trees, each based on a different subset of training data, and then combine their predictions to make a final decision. Random forest is an extension of the decision tree algorithm, which involves creating a tree-like structure of nodes that split the data based on specific features until a prediction is made at the leaf nodes. While decision trees are effective at capturing complex relationships in the data, they are prone to overfitting-which means they may perform well on the training data but generalize poorly to new, unseen data. Random forest addresses this problem by generating multiple decision trees, each based on a random subset of the features and training data. This randomness helps to reduce overfitting and improve the model's ability to generalize to new data.
The circuit for the data glove is connected using the Tinker Cad Software. The code is programmed into an Arduino, and based on this code, outputs are displayed on an LCD screen.
The data glove is designed with flex sensors attached along each finger. These sensors detect the bending of the fingers by changing resistance when the fingers are bent. When a person moves their middle finger to a certain extent, the flex sensor attached to the finger detects this movement. The bending of the finger alters the resistance in the flex sensor, producing a corresponding voltage. This change in voltage is then interpreted by the system, which translates it into a specific command. This command is displayed on the LCD screen and simultaneously vocalized, enabling both the hearing- and visually impaired to receive the information.
Graphical Representation
The main elements that can be shown in the result section of the software model of the Gesture Vocalizer using the random forest algorithm are:
Random forest models can provide information about the importance of different features in predicting the target variable. A graph showing the relative importance of different features is useful in understanding which features are most important for the model's performance.
A confusion matrix is used to evaluate the performance of the model in predicting the correct gesture based on the vocal input. This matrix shows the number of true positives, true negatives, false positives, and false negatives, and can help to evaluate the accuracy, precision, recall, and F1-score.
.png?width=284&height=155&name=confussion%20matrix%20(1).png)
The ROC (receiver operating characteristic) curve can be used to evaluate the performance of the model in differentiating between various classes. This graph plots the true positive rate against the false positive rate at different threshold levels, and can help to evaluate the overall performance.
.png?width=284&height=248&name=ROC%20cruve%20(1).png)
True Positive Rate (TPR), also known as Sensitivity or Recall, is calculated using the formula
Where:
False Positive Rate (FPR) is calculated using the formula:
Where:
The precision-recall curve is used to evaluate the performance of the model in predicting positive cases. This graph plots precision (the fraction of true positive predictions among all positive predictions) against recall (the fraction of true positive predictions among all actual positive cases) at different threshold levels.
Cross-validation is used to evaluate and report the performance of the model. This could include the mean accuracy, precision, recall, or F1-score across different folds of the data.
Comparison of the random forest and decision tree algorithms results reveals that the accuracy is higher in random forest for the same data set.
This white paper therefore holds that the random forest algorithm is more efficient than previous algorithms such as decision tree, etc.
The gesture vocalizer is an easy-to-use hand glove device designed to help the hearing impaired and non-verbal individuals communicate with the rest of the world using acoustics or sounds. The vocalizer receives a gesture as input and provides text output on an LCD display. Hand gestures are more crucial than other types of gestures (arm, face, head, and body) as they swiftly reflect the user’s perception. People without disabilities rarely learn sign language, making communication with the differently abled community difficult, further isolating them as a result.
Cyient’s gesture vocalizer will help to better connect the differently abled community to the world and make their lives easier with the ability to “talk” easily, effectively, and accurately.
Madhavi Sanapala specializes in electrical schematics and system design. After six months of training as a schematic engineer, she now works as a reliability engineer in the aviation industry at Cyient. With a curious mind and drive for excellence, continuously contributing to advancements in electrical engineering.
Dake Anvesh brings expertise in system design and analysis. After six months of training as a schematic engineer, he now works as a technical author in the aviation industry at Cyient. With an inquisitive mind and a dedication to achieving greatness significantly contributing to industry advancements with his innovative solutions.
Midatana Jahnavi excels in creating detailed electrical diagrams. After six months of training as a schematic engineer, she now works as a technical author in the aviation industry at Cyient. With a curious mind and drive for excellence, continuously contributing to advancements in electrical engineering.
Sanikommu Srinivasulu excels in creating detailed electrical diagrams. After six months of training as a schematic engineer, he now works as a technical author in the Healthcare&Life Sciences (India) industry at Cyient. With a keen mind and a relentless drive for excellence significantly contributing to industry advancements with his innovative solutions.
After review of the De Novo request, the FDA will make a final decision to either grant or decline. FDA will also consider the De Novo request to be withdrawn in certain situations.
Grant De Novo Request (21 CFR 860.260)
If the data and information provided to the FDA demonstrate that general controls or general and special controls are adequate to provide reasonable assurance of safety and effectiveness, and the probable benefits of the device outweigh the probable risks, then the FDA may grant the De Novo request and establish a new classification regulation for the new device type.
Decline De Novo Request (21 CFR 860.260)
De Novo request is declined by the FDA if –
If the De Novo request is declined, the device remains in Class III and the requester may not legally market the device. The FDA will issue a written order to the requester identifying the reasons, which can include lack of performance data that warrant declining the De Novo request. The requester should generally either submit an application for premarket approval under Section 515 of the FD&C Act or collect additional information to address the issues and submit a new De Novo request that includes the additional information.
Withdrawal of a De Novo Request (21 CFR 860.250)
The FDA will consider a De Novo classification request to be withdrawn if—
If the FDA considers a De Novo request to be withdrawn, the FDA notifies the requester with reference to the De Novo request number and the date the FDA considered the De Novo request withdrawn.
Upon receipt of a De Novo request, the FDA will conduct an acceptance review. The acceptance review is an administrative review to assess the completeness of the application and whether it meets the minimum threshold of acceptability. If any of the acceptance elements are not included, a justification has to be provided for the omission.
To aid in the acceptance review, it is recommended to submit an Acceptance Checklist as per the guidance document with the De Novo request that identifies the location of supporting information for each checklist element.
The De Novo request will not be accepted and will receive a Refuse to Accept (RTA) designation if one or more of the elements noted as RTA items in the Acceptance Checklist are not present and no explanation is provided for the omission(s). However, during the RTA review, the FDA staff has the discretion to determine whether the missing checklist elements are needed to ensure the De Novo request is administratively complete to allow the De Novo request to be accepted.
Within 15 calendar days of the Document Control Center receiving the De Novo request, the FDA will notify the requester electronically of the acceptance review result as one of the following:
Once the De Novo request is accepted for substantive review, the FDA conducts a classification review of legally marketed device types and analyzes whether an existing legally marketed device of the same type exists. This information is used to confirm that the device is eligible for De Novo classification.
During the substantive review of a De Novo request, the FDA may identify deficiencies that can be adequately addressed through interactive review and not require a formal request for additional information.
If the issues and deficiencies cannot be addressed through interactive review, an Additional Information letter will be sent to the requester. If an Additional Information letter is sent, then the De Novo request will be placed on hold. The requester has 180 calendar days from the date of the Additional Information letter to submit a complete response to each item in the Additional Information letter.
Note: The response must be sent to the DCC within 180 calendar days of the date of the Additional Information letter. No extensions beyond 180 days are granted. If the FDA does not receive a complete response to all deficiencies in the Additional Information letter within 180 days of the date of the letter, the request will be considered withdrawn and deleted from the FDA's review system. If the De Novo request is deleted, the De Novo requester will need to submit a new request to pursue the FDA's marketing authorization for that device.
The requester must submit their response to an Additional Information letter in electronic format (eCopy), to the DCC of the appropriate center. The response should—
The final step is the De Novo request decision. Under MDUFA IV, the FDA's goal is to decide about a De Novo request in 150 review days. Review days are calculated as the number of calendar days between the date the De Novo request was received by the FDA and the date of the FDA's decision, excluding the days a request was on hold for an Additional Information request.
Cyient offers a one-stop solution, CyARC–Accelerated Regulatory Platform, for helping medical device companies to ensure regulatory compliance. Empowered by Quality Assurance and Regulatory Affairs (QARA) CoE, Cyient has certified professionals across all the functions who have the required skillsets and expertise to support medical device companies throughout the life-cycle of their medical devices.
Abhishek Kumar is an SME in Medical Device Regulatory Affairs, Quality Assurance, and Clinical Affairs with over 13 years of experience. He has led the EU MDR-2017/745 sustenance program, identifying business opportunities for sales teams, and managed the engagement program for a US-based medical device company. He has supported the gap assessment, remediation, and submission of 45+ Technical Documentations as per EU MDR, and created 40+ CERs for Class I, II, and III medical devices according to MEDDEV 2.7.1 Rev-4. Additionally, Abhishek has developed proposals for global markets, including Europe, US, ASEAN, China, Taiwan, and Japan, and prepared and implemented regulatory plans for NPD in 90+ countries by analyzing feasibility, defining requirements, and coordinating cross-functional teams.
