2022Machine Learning

Sign Language Recognition Module

Machine learning system for real-time sign language gesture recognition and translation.

Project

Machine learning system for real-time sign language gesture recognition and translation.

Machine Learning
Computer Vision
Accessibility Technology

Context

The module addresses the limited availability of large, consistent sign-language datasets by providing tools for recording examples, visualizing movements and training new classifiers.

Data pipeline

Body, hand and facial landmarks are captured from MediaPipe and stored as time-series examples. Data augmentation with shift and scale transformations expands the training set.

Model

The recognition network combines a bidirectional LSTM with linear layers, dropout, batch normalization and ReLU activations. It classifies temporal gesture sequences rather than isolated images.

Integration

Predictions and confidence values can be visualized live and transmitted to other GOSAI applications, including the augmented-reality sign-language game.