Sign language is used as a visual form of communication among the deaf and is considered as an official language in many countries. While there has been many efforts to achieve efficient translation between sign and verbal languages, many of these previous work can be applied in non-mobile context or exploit RGB images which can potentially invade the users' privacy. This work presents our preliminary efforts in designing a mobile device-based sign language translation system using depth-only images. Our system performs image processing on the smartphone-collected depth images to emphasize the subject's hand and upper body gestures and exploits a convolutional neural network for feature extraction. The series of features gathered from word-representing videos are passed through a Long-Short Term Memory (LSTM) model for word-level sign language translation. We train and test our system using a total of 2,200 samples collected from 26 people for 17 words. The classification accuracy of our proposed system using the self-collected data achieves 92% with an efficient image preprocessing phase.
|Title of host publication||2020 International Conference on COMmunication Systems and NETworkS, COMSNETS 2020|
|Publisher||Institute of Electrical and Electronics Engineers Inc.|
|Number of pages||4|
|Publication status||Published - 2020 Jan|
|Event||2020 International Conference on COMmunication Systems and NETworkS, COMSNETS 2020 - Bengaluru, India|
Duration: 2020 Jan 7 → 2020 Jan 11
|Name||2020 International Conference on COMmunication Systems and NETworkS, COMSNETS 2020|
|Conference||2020 International Conference on COMmunication Systems and NETworkS, COMSNETS 2020|
|Period||20/1/7 → 20/1/11|
Bibliographical noteFunding Information:
ACKNOWLEDGEMENTS This work was supported by the the Basic Science Research Program through the National Research Foundation of Korea funded by the Ministry of Science and ICT (2018R1C1B6003869).
© 2020 IEEE.
All Science Journal Classification (ASJC) codes
- Artificial Intelligence
- Computer Networks and Communications
- Computer Science Applications
- Hardware and Architecture
- Signal Processing