Modeling and recognition of signs in Signing Exact English (SEE)

Abstract

<P>THE MAIN OBJECTIVE OF THIS WORK WAS TO INVESTIGATE AND DEVELOP EFFECTIVE AND ROBUST ALGORITHMS TO RECOGNIZE TWO COMPONENTS OF SIGN LANGUAGE GESTURES, VIZ; HANDSHAPE AND MOVEMENT TRAJECTORY IN SIGNING EXACT ENGLISH (SEE). THE HANDSHAPE DATA WAS OBTAINED FROM CYBERGLOVE, WHILE TRAJECTORY DATA WAS OBTAINED FROM POLHEMUS MAGNETIC TRACKERS. A LINEAR DECISION TREE WITH FISHER'S LINEAR DISCRIMINANT (FLD) WAS USED TO CLASSIFY HANDSHAPES IN SEE SIGNS. FOR COMPARISON, HANDSHAPE RECOGNITION WITH RADIAL BASIS FUNCTION (RBF) NEURAL NETWORK CLASSIFIER WAS ALSO IMPLEMENTED. FOR THE MOVEMENT TRAJECTORY RECOGNITION, VECTOR QUANTIZATION PRINCIPAL COMPONENT ANALYSIS (VQPCA) WAS EMPLOYED. BOTH PERIODIC AND NON-PERIODIC SEE SIGN HAND GESTURES WERE INVESTIGATED, USING A HIERARCHICAL APPROACH TO RECOGNIZE ISOLATED 3-D HAND GESTURE TRAJECTORIES. PERIODIC AND NON-PERIODIC GESTURES WERE FIRST DIFFERENTIATED, FOLLOWED BY RECOGNITION OF MOVEMENT TRAJECTORY. PERIODICITY DETECTION WAS BASED ON FOURIER ANALYSIS.