Algorithm and Software Implementation for Hand Gesture-Based Control

Abstract

The use of palm motion identification in controlling digital devices has become popular due to the advancement of synthetic cognition innovation. A palm motion-governed digital pointer framework that utilizes AI algorithms to identify palm gestures and translate them into pointer movements is proposed in this paper. The framework is designed to provide an substitute platform for people who have difficulty using a conventional pointer or keyboard. The proposed framework uses a webcam to capture images of the operator’s palm, which are processed by an AI procedure to identify the gestures being made. The framework is trained using a dataset of palm gestures to identify different gestures. Once the motion is recognized, it is translated into a corresponding pointer motion, which is then executed on the digital display. The framework is designed to be scalable and adaptable to different types of environments and devices. All the entry operations can be virtually governed by using dynamic/static palm gestures along with a speech assistant. In our work we make use of ML and Computer Perception algorithms to identify palm gestures and speech commands, which works without any additional hardware requirements. The model is implemented using CNN and mediapipe structure. This framework has potential applications like enabling palm-free activity of devices in hazardous environments and providing an substitute platform for hardware pointer. Overall, the palm motion-governed digital pointer framework offers a promising approach to enhance operator experience and enhance accessibility through human-computer interaction.