abstract

The human eye movements hold a lot of valuable information since they reveal the attention of individuals, hence the eye movement recognition (EMR) can be used in many vital fields. EMR can play an important role in Human-Computer Interaction (HCI), neurological disease early detection, assistive technologies, and interactive applications, especially for disabled people, neurological diseases such as Amyotrophic Lateral Sclerosis (ALS), Parkinson’s disease, and people who have severe injuries in healthcare rooms, this spectrum of people are very in need for this technology to enable them to use their eye’s moves to interact with their environments such as controlling the wheelchair. The eye muscles are less affected by ALS and Parkinson’s disease. Although its importance, EMR is widely neglected in the literature, especially the computer vision-based and deep learning-based approaches and that’s what this work focuses on it. EMR is known for its challenges such as the verity of eye region features, lighting issues, complex calibration, computing demand, and high cost. And from  the academic side, there is a lack of standardization in the literature which impacts the industrial side.
This thesis presents a comprehensive study on the recognition of eye movements using advanced deep learning models and addressing the gaps in the field. The work focuses on the development of an interactive system that translates eye movements and blinks into commands to assist people with disabilities, including those with Parkinson’s, ALS, and other conditions. To help them control their wheelchairs, regulating healthcare room amenities through user-directed technology.
The core of this research involved the collection, preprocessing, and annotation of a dataset involve 10,125 eye images. These images represent a diverse set of subjects varying in age, environmental conditions such as
different lighting, presence of shadows, and reflections from eyeglasses, and from multiple distances. This diverse dataset allows for a robust analysis of eye movement patterns, which is critical for the system’s accuracy and reliability.
Various types of deep learning models were evaluated, including InceptionV3, ResNet-18, EfficientNet-B0, SqueezeNetV1.1, AlexNet, VGG16, proposed custom model, and utilizing Vision Transformer (ViT) model for the first time in EMR field which reveal promising capabilities. These models were implemented using both fine-tuning and training-from-scratch strategies to determine the most effective approach for EMR. The models underwent rigorous testing including real-time testing. The findings indicated that models are highly effective in classifying eye movements into five directional states. It shows impressive validation accuracies, with ResNet-18 leading at 99.82% accuracy. The real-time application is also highlighted by the remarkable interference times for IceptionV3 and VGG16.
In conclusion, this thesis not only presents the technical understanding of EMR but also contributes to the society by enhancing the independency of individuals with disabilities. The successful implementation of this work can lead to profound improvements in quality of life and the widening of horizons for assistive technologies.