[1] H. B. D. Nguyen and H. N. Do, “Deep learning for American sign language fingerspelling recognition system,” 2019 26th Int. Conf. Telecommun. ICT 2019, pp. 314–318, 2019, doi: 10.1109/ICT.2019.8798856.
[2] B. Vivek and N. Dianna Radpour, “Using Deep Convolutional Networks for Gesture Recognition in American Sign Laguage,” 2017.
[3] L. Kumar, “Real-Time Finger Spelling American Sign Language Recognition Using Deep Convolutional Neural Networks,” 2018.
[4] K. Bantupalli, “American Sign Language Recognition Using Machine Learning and Computer Vision,” 2018.
[5] D. Kelly, “Computational Models for the Automatic Learning and Recognition of Irish Sign Language by,” Thesis, p. 293, 2010.
[6] K. Sahithya, P. Road, K. Sahithya, and P. Road, “Sign Language Translator Using Machine Learning,” vol. 13, no. 4, pp. 1–5, 2018.
[7] K. Yamashita, R., Nishio, M., Do, R. K. G., & Togashi, “Convolutional neural networks: an overview and application in radiology. Insights into Imaging,” 2018.
[8] A. Salem and S. Vadera, “A Convolutional Neural Network to Classify American Sign Language Fingerspelling from Depth and Colour Images,” 2017.
[9] R. Daroya, D. Peralta, and P. Naval, “Alphabet Sign Language Image Classification Using Deep Learning,” IEEE Reg. 10 Annu. Int. Conf. Proceedings/TENCON, vol. 2018-Octob, no. October, pp. 646–650, 2018, doi: 10.1109/TENCON.2018.8650241.
[10] L. Rioux-Maldague and P. Giguere, “Sign language fingerspelling classification from depth and color images using a deep belief network,” Proc. - Conf. Comput. Robot Vision, CRV 2014, pp. 92–97, 2014, doi: 10.1109/CRV.2014.20.
[11] C. M. Jin, Z. Omar, and M. H. Jaward, “A mobile application of American sign language translation via image processing algorithms,” Proc. - 2016 IEEE Reg. 10 Symp. TENSYMP 2016, pp. 104–109, 2016, doi: 10.1109/TENCONSpring.2016.7519386.
[12] A. Bhattacharya, V. Zope, K. Kumbhar, P. Borwankar, and A. Mendes, “Classification of Sign Language Gestures using Machine Learning,” Int. J. Adv. Res. Comput. Commun. Eng., vol. 8, no. 12, pp. 97–103, 2019, doi: 10.17148/IJARCCE.2019.81219.
[13] R. Kurdyumov, P. Ho, and J. Ng, “Sign Language Classification Using Webcam Images,” pp. 1–4, 2011, [Online]. Available: http://cs229.stanford.edu/proj2011/KurdyumovHoNg-SignLanguageClassificationUsingWebcamImages.pdf.
[14] Z. Parcheta and C. D. Martinez Hinarejos, “Sign Language Gesture Classification using Neural Networks,” no. November, pp. 127–131, 2018, doi: 10.21437/iberspeech.2018-27.
[15] R. Gupta and S. Rajan, “Comparative Analysis of Convolution Neural Network Models for Continuous Indian Sign Language Classification,” Procedia Comput. Sci., vol. 171, no. 2019, pp. 1542–1550, 2020, doi: 10.1016/j.procs.2020.04.165.
[16] Q. Ji, J. Huang, W. He, and Y. Sun, “Optimized deep convolutional neural networks for identification of macular diseases from optical coherence tomography images,” Algorithms, vol. 12, no. 3, pp. 1–12, 2019, doi: 10.3390/a12030051.
[17] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely connected convolutional networks,” Proc. - 30th IEEE Conf. Comput. Vis. Pattern Recognition, CVPR 2017, vol. 2017-Janua, pp. 2261–2269, 2017, doi: 10.1109/CVPR.2017.243.
[2] B. Vivek and N. Dianna Radpour, “Using Deep Convolutional Networks for Gesture Recognition in American Sign Laguage,” 2017.
[3] L. Kumar, “Real-Time Finger Spelling American Sign Language Recognition Using Deep Convolutional Neural Networks,” 2018.
[4] K. Bantupalli, “American Sign Language Recognition Using Machine Learning and Computer Vision,” 2018.
[5] D. Kelly, “Computational Models for the Automatic Learning and Recognition of Irish Sign Language by,” Thesis, p. 293, 2010.
[6] K. Sahithya, P. Road, K. Sahithya, and P. Road, “Sign Language Translator Using Machine Learning,” vol. 13, no. 4, pp. 1–5, 2018.
[7] K. Yamashita, R., Nishio, M., Do, R. K. G., & Togashi, “Convolutional neural networks: an overview and application in radiology. Insights into Imaging,” 2018.
[8] A. Salem and S. Vadera, “A Convolutional Neural Network to Classify American Sign Language Fingerspelling from Depth and Colour Images,” 2017.
[9] R. Daroya, D. Peralta, and P. Naval, “Alphabet Sign Language Image Classification Using Deep Learning,” IEEE Reg. 10 Annu. Int. Conf. Proceedings/TENCON, vol. 2018-Octob, no. October, pp. 646–650, 2018, doi: 10.1109/TENCON.2018.8650241.
[10] L. Rioux-Maldague and P. Giguere, “Sign language fingerspelling classification from depth and color images using a deep belief network,” Proc. - Conf. Comput. Robot Vision, CRV 2014, pp. 92–97, 2014, doi: 10.1109/CRV.2014.20.
[11] C. M. Jin, Z. Omar, and M. H. Jaward, “A mobile application of American sign language translation via image processing algorithms,” Proc. - 2016 IEEE Reg. 10 Symp. TENSYMP 2016, pp. 104–109, 2016, doi: 10.1109/TENCONSpring.2016.7519386.
[12] A. Bhattacharya, V. Zope, K. Kumbhar, P. Borwankar, and A. Mendes, “Classification of Sign Language Gestures using Machine Learning,” Int. J. Adv. Res. Comput. Commun. Eng., vol. 8, no. 12, pp. 97–103, 2019, doi: 10.17148/IJARCCE.2019.81219.
[13] R. Kurdyumov, P. Ho, and J. Ng, “Sign Language Classification Using Webcam Images,” pp. 1–4, 2011, [Online]. Available: http://cs229.stanford.edu/proj2011/KurdyumovHoNg-SignLanguageClassificationUsingWebcamImages.pdf.
[14] Z. Parcheta and C. D. Martinez Hinarejos, “Sign Language Gesture Classification using Neural Networks,” no. November, pp. 127–131, 2018, doi: 10.21437/iberspeech.2018-27.
[15] R. Gupta and S. Rajan, “Comparative Analysis of Convolution Neural Network Models for Continuous Indian Sign Language Classification,” Procedia Comput. Sci., vol. 171, no. 2019, pp. 1542–1550, 2020, doi: 10.1016/j.procs.2020.04.165.
[16] Q. Ji, J. Huang, W. He, and Y. Sun, “Optimized deep convolutional neural networks for identification of macular diseases from optical coherence tomography images,” Algorithms, vol. 12, no. 3, pp. 1–12, 2019, doi: 10.3390/a12030051.
[17] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely connected convolutional networks,” Proc. - 30th IEEE Conf. Comput. Vis. Pattern Recognition, CVPR 2017, vol. 2017-Janua, pp. 2261–2269, 2017, doi: 10.1109/CVPR.2017.243.
- Abstract viewed - 786 times
- PDF downloaded - 688 times
Affiliations
Anna Dwi Marjusalinah
Graduate Program in Computer Science, Faculty of Computer Science, Universitas Sriwijaya
Samsuryadi Samsuryadi
Affiliation not stated
Muhammad Ali Buchari
Department of Computer System, Faculty of Computer Science, Universitas Sriwijaya
Classification of Finger Spelling American Sign Language Using Convolutional Neural Network
Abstract
Sign language is a combination of complex hand movements, body postures, and facial expressions. However, only a limited number of people can understand and use it. A computer aid sign language recognition with finger spelling style utilizing a convolutional neural network (CNN) is proposed to reduce the burden. We compared two CNN architectures such as Resnet 50, and DenseNet 121 to classify the American sign language dataset. Several data splitting proportions were also tested. From the experimental result, it is shown that the Resnet 50 architecture with 80:20 data splitting for training and testing indicates the best performance with an accuracy of 0.999913, sensitivity 0.998966, precision 0.998958, specificity 0.999955, F1-score 0.999913, and error 0.0000898.