Abstract

Based on data from Polda Metro Jaya Traffic Sector, the number of traffic accidents throughout 2020 was 7,565. With a factor of 1,018 drowsy drivers or 13% of the total incidents. A solution is needed for that. The aim of this research is to calculate the average value of the duration of detection time and the number of flashes per minute, by knowing when the driver is drowsy or not drowsy. The method used is Eye Aspect Ratio (EAR), where the driver will be detected whether he is sleepy or not, by analyzing the parameters of the number of blinks per minute and the duration of the blinks. If the eyes are open (EAR value) less than 0.45 and more than or equal to 3 seconds in one blink, it is categorized as sleepy. Tests were carried out on lighting, namely: morning, afternoon, evening and night light. Test results in daylight and evening light conditions with a light intensity value of 78 lux mean the duration of drowsiness detection is 3.26 seconds and the difference in time duration with the reference theory is 0.26 seconds. Meanwhile, for night light with a light intensity value of 15 lux, the average duration of drowsiness detection is 4.04 seconds and the difference in time duration with the reference theory is 1.04 seconds. Meanwhile, the number of blinks per minute when you are sleepy is 5-8 blinks/minute and when you are not sleepy it is 13-17 blinks/minute, for morning, afternoon, evening and night light conditions. It can be concluded that overall, this system can work well for day and evening and night light conditions.

Implementation of the EAR Method in Detecting Drowsiness in Vehicle Drivers

Implementasi Metode EAR Dalam Mendeteksi Kondisi Mengantuk Pengemudi Kendaraan Bermotor

Andy Suryowinoto ^1*) , Ainur Rohman ²⁾ , Wahyu Setyo Pambudi ³⁾

^{123 )} Department of Electrical Engineering, Institut Teknologi Adhi Tama Surabaya, Indonesia

^{( 1 )} andysuryo@itats.ac.id

^{( 2 )} ainur.rohman101@hotmail.com

^{( 3 )} wahyusp@itats.ac.id

Abstract _ Based on data from Polda Metro Jaya Traffic Sector, the number of traffic accidents throughout 2020 was 7,565. With a factor of 1,018 drowsy drivers or 13% of the total incidents. A solution is needed for that. The aim of this research is to calculate the average value of the duration of detection time and the number of flashes per minute, by knowing when the driver is drowsy or not drowsy. The method used is Eye Aspect Ratio (EAR), where the driver will be detected whether he is sleepy or not, by analyzing the parameters of the number of blinks per minute and the duration of the blinks. If the eyes are open (EAR value) less than 0.45 and more than or equal to 3 seconds in one blink, it is categorized as sleepy. Tests were carried out on lighting, namely: morning, afternoon, evening and night light. Test results in daylight and evening light conditions with a light intensity value of 78 lux mean the duration of drowsiness detection is 3.26 seconds and the difference in time duration with the reference theory is 0.26 seconds. Meanwhile, for night light with a light intensity value of 15 lux, the average duration of drowsiness detection is 4.04 seconds and the difference in time duration with the reference theory is 1.04 seconds. Meanwhile, the number of blinks per minute when you are sleepy is 5-8 blinks/minute and when you are not sleepy it is 13-17 blinks/minute, for morning, afternoon, evening and night light conditions. It can be concluded that overall, this system can work well for day and evening and night light conditions .

Keywords: Sleepy Eye Detection, Face Landmark, Eye Aspect Ratio, Raspberry Pi, Computer Vision

Abstrak_ Berdasarkan data dari Polda Metro Jaya Bidang Lalu Lintas, jumlah kecelakaan lalu lintas, sepanjang tahun 2020 ialah 7.565. Dengan faktor pengendara mengantuk sebanyak 1.018 atau 13% dari total kejadian. Diperlukan sebuah solusi untuk itu. Tujuan penelitan ini untuk menghitung nilai rata-rata durasi waktu deteksi dan banyak kedipan/menit, dengan mengetahui saat kondisi pengemudi mengantuk atau kondisi tidak mengantuk. Metode yang digunakan adalah Eye Aspect Ratio (EAR), dimana pengemudi akan dideteksi apakah mengantuk atau tidak, dengan analisa parameter jumlah kedipan per menitnya dan lama kedipan. Untuk mata terbuka (nilai EAR) kurang dari 0,45 dan waktu lebih atau sama dengan 3 detik dalam satu kedipan, maka dikategorikan mengantuk. Pengujian dilakukan pada pencahayaan yaitu: cahaya pagi, siang, sore dan malam hari. Hasil pengujian pada kondisi cahaya siang, sore hari dengan nilai intensitas cahaya 78 lux rata-rata durasi deteksi mengantuk dalam waktu 3,26 detik dan selisih durasi waktu dengan teori acuan sebesar 0,26 detik. Sedangkan untuk cahaya malam hari dengan nilai intensitas cahaya 15 lux rata-rata durasi deteksi mengantuk dalam waktu 4,04 detik dan selisih durasi waktu dengan teori acuan sebesar 1,04 detik. Sedangkan jumlah kedipan per menit saat kondisi mengantuk sebanyak 5-8 kedipan/menit dan saat kondisi tidak mengantuk sebanyak 13-17 kedipan/menit, untuk kondisi cahaya pagi, siang, sore, dan malam hari. Dapat disimpulkan, bahwa secara keseluruhan sistem ini dapat bekerja dengan baik untuk kondisi cahaya siang, dan sore, malam hari.

Kata Kunci: Deteksi Mata Mengantuk, Face Landmark, Eye Aspect Ratio, Raspberry Pi, Computer Vision

Traffic accidents are one of the accidents that often occur on roads in large numbers. Based on data from Polda Metro Jaya Traffic Sector, the number of traffic accidents throughout 2020 was 7,565 cases . The main reason for traffic accidents is human error, which is responsible for 61% of cases. Faulty components in vehicles, like faulty brakes or worn-out tires, lead to 9% of accidents, while issues with the environment and infrastructure, such as poorly maintained roads or uneven surfaces, are responsible for 30% of accidents. Nevertheless, tired motorists are a major factor in car crashes. Accidents due to drowsy drivers were 1,018 cases, 449 cases were due to vehicle suitability, and 387 roadworthiness cases, so it can be ascertained that drowsiness is a problem often faced by motorists According to the National Sleep Foundation, 54% of adult drivers experience drowsiness while driving, with 28% admitting to falling asleep behind the wheel. Furthermore, over 40% confessed to dozing off behind the wheel at least once during driving. Being sleepy is seen as moving from being awake to falling asleep. The initial indication of feeling tired. is the inability to open the eyes . Frequently closing your eyes makes it impossible to carry out their tasks effectively. The early detection system for drivers when driving a car can be seen from several parameters, namely the detection of sleepy eyes, frequency of eye blinks, yawning, and head movements . Several studies on drowsiness detection using digital image processing have been carried out , The drowsy driver detection system is essential for avoiding accidents on the road, as it alerts the driver when signs of drowsiness are detected. Earlier studies conducted by Kuswara in his research title "Digital Image Based Drowsiness Eye Detection Application Using the Haar Classifier Method in Real Time" explained: The closed eyes detection application using the Haar Cascade Classifier method can detect closed eyes with results that are in accordance with those expected, namely in the form of an eye blink with a distance of 40 cm from the face to the webcam with a time of 400 ms and an accuracy of 98%. In the next research by Yusri, entitled "Detection of Driver Drowsiness by Calculating the Speed of Eye Blinking" , explains a simple real-time detection system for drowsiness based only on the speed of eye blinking obtained from the eye aspect ratio. For eye detection, the author uses the linear HOG (Histogram Oriented Gradient) and SVM (Support Vector Machine) methods. If the eye blink rate falls below an empirically determined threshold, the system will trigger an alarm, thereby preventing the driver from entering the microsleep phase. In further research by Islam entitled "A Study on Tiredness Assessment by Using Eye Blink Detection", explains: Driver distraction can be analyzed by tracking eye blinks using facial landmark detection to identify when a driver loses focus or concentration. Due to the issues mentioned earlier and existing studies, scientists created a computer vision system to identify if a person is sleepy by analyzing the eye aspect ratio. This is done by comparing the duration of eye closure, based on the eyes' position determined using facial landmark detection. This method helps in detecting driver fatigue. Counting the total amount of blinks in a minute is a way to identify drowsiness. Next, the results of the calculation are examined against the specified eye aspect ratio threshold. If the results fall below the threshold, the driver will be classified as either drowsy or unconscious.

2.1 Research Stages

Method for system testing , The overall system design illustrated in Figure 1 is displayed. The system design involves the hardware, software, and interface mechanical design. Hardware is made up of components such as input, processing, and output.

[Figure 1 about here.]

Utilizing a Logitech C170 webcam as a digital image input tool in video mode that is used for capturing images [13]. Following the capturing of the image, it is processed on a Raspberry Pi 4B that has a 32 GB microSD card for storage and boot, a 5V-3A power supply for powering, and has been customized for the image processing algorithm on the Raspberry Pi OS [14]. The end result can activate the LED, set off the buzzer, and show cautionary messages on the 16x2 alphanumeric LCD. In order for the Raspberry Pi to share and view operations from the Raspberry Pi OS with a Windows 10 computer, a CAT5e Ethernet cable must be used to connect the devices. The VNC Viewer software should be configured on the Windows computer, which should be on the same network as a Wi-Fi or cellphone hotspot. using SSH (Secure Shell).

2.2 Software

In software design, you can see the process flow console in the system in Figure 2 regarding the design and program steps created to detect sleepiness with the following flow: face detection, facial landmark detection, determining facial landmarks using the following equation, which will later be processed in the program algorithm, as follows.

(1)

(2)(2)

(3)(3)

(4)

(5)

(6)

Where, represents the distance between the upper lip and lower lip, while and , represent the distance of eye opening, and represent the angle of the mouth, and and represent the slope of the linear polynomial fitting. These facial characteristics can be constructed by combining the five characteristics ( as follows.

Then determine the distance value between the camera and the face, find or determine facial landmark points as follows.

[Figure 2 about here.]

Then, determine the EAR value, compare the EAR value according to the threshold for making a decision whether the driver or someone is drowsy .

[Figure 3 about here.]

The way the drowsy driver detection system works starts from accessing the webcam. By using a haar cascade classifier model, the system will detect faces. The function of face detection is to find Regions of Interest (ROI) on faces. This functions for the next process, namely finding facial landmark points (determining the area of the eyebrows, nose, jaw line, mouth, especially the eyes) . When the points on the face have been detected, the next step is to determine the distance from the face to the camera and determine the landmark points in the eyes. By using eye parameters, the distance and 4 points in one eye (top, bottom, left, right) with a total of 8 landmark points in both eyes can be determined. Distance is an additional parameter to test the distance between the face and the camera of a sleepy person, while the 4 eye landmark points are used to determine the eye aspect ratio (EAR) value Using the following equation.

Where, is the average distance between points p₂ and p₆; while is the average distance between points p₃ and p₅; And is the average distance between points p₁ and p₄. The following is an illustration of the ratio points of the EAR in Figure 3 below.

[Figure 4 about here.]

[Figure 5 about here.]

Figure 4 shows the condition of the eye covering the average distance between the set of points p₁ to p₆; Meanwhile, Figure 5 shows the open eye condition with a set of points p₁ to p₆. This comparison of eye condition is called the eye aspect ratio.

3.1 Hardware

This system includes a sequence of buzzers, LEDs, and LCDs, along with a webcam or camera connected to the Raspberry Pi in hardware design. To establish an alert mechanism for detecting drowsiness, an alphanumeric LCD connected via an I2C interface on the Raspberry Pi displays a warning message, triggers a buzzer, and activates the LED. Using a 5V-3A power supply as the power source on a raspberry pi.

[Figure 6 about here.]

[Figure 7 about here.]

In figure 6 there is a placement of the supporting components for the processing unit and LCD display system, and the power supply regulator. And in figure 7 there is a box packing all these components, with access to the I/O interface to make it easier to install the system on a motorized vehicle.

3.2 Implementation and testing of the system

In the process of implementing the placement of the hardware box and webcam in a motorized vehicle, this layout is intended so that it can obtain the optimal position when detecting the driver. in front of the driver's face in the car.

[Figure 8 about here.]

In figure 8. The position of the webcam is placed facing in front of the driver, so that it can be easy to read the driver's condition, and every effort is made to get bright and good lighting in the vehicle. The testing process was carried out using a device implemented in morning, afternoon, evening and night light conditions

[Figure 9 about here.]

The first test shown in Figure 9 shows the results of testing the number of blinks per minute when the driver is in a sleepy-eyed condition which was tested using 3 test subjects. The diagram shows that the number of participants or the three test subjects blinked per minute.

[Table 1 about here.]

Based on the test results in Table 1 of distance testing in bright lighting conditions, data was obtained that, from 10 test distances, at a distance of 10 and 20 cm the system could not detect faces because the distance was too close, but at a distance of 30 cm to 100 cm, the system able to identify faces with eye parameters marked with green dots, as shown in Figure 9.

3.3 Discussion

This test was carried out using a digital lux meter with a light intensity value of 78 lux. This test uses a distance from the face to the camera that varies from 30 – 100 cm. Testing and data collection was done 10 times.

[Figure 10 about here.]

Figure 10 shows a comparison between the theoretical duration of detection, namely when a driver is said to be sleepy when there is no blinking with his eyes closed or partially closed (eye aspect ratio < 0.45) with a time of approximately 3 seconds. For prototype testing, 10 trials were carried out (with a timer using a stopwatch) showing the length or duration of drowsiness detection during the day (value 78 lux). The length of time the system detects no blinking (drowsiness) produces an average value of 3.29 seconds. Then, to see how far the gap is, look for the difference with the reference time duration, namely 3 seconds in each trial (trials 1 to 10). If you look at the average, the time difference is 0.29 seconds. From the graph, it can also be seen that the lines are very close together, which indicates a very slight difference between the detection duration of the prototype test and that of the reference theory (3 seconds).

[Figure 11 about here.]

Figure 11 above shows a comparison between the theoretical duration of detection, namely when the driver is said to be sleepy when there is no blinking with the eyes closed or partially closed (eye aspect ratio of the eye < 0.45) with a time of approximately 3 seconds. For prototype testing, 10 trials were carried out (with a timer using a stopwatch) showing the length or duration of drowsiness detection at night (value 15 lux). The length of time the system detects no blinking (drowsiness) produces an average value of 3.86 seconds. Then, to see how far the gap is, look for the difference with the reference time duration, namely 3 seconds in each trial (trials 1 to 10). If you look at the average, the time difference is 0.86 seconds. From the graph it can also be seen that the lines do not coincide, which indicates that the system is a little slow in detecting in low light conditions

Based on the design and system testing that has been carried out, it can be concluded that the test results using the Eye Aspect Ratio (EAR) obtained an average duration of drowsiness detection within 3.26 seconds at a light intensity level of 78 lux, in the afternoon and 4.04 seconds , at a light intensity level of 15 lux at night. Meanwhile, the number of blinks per minute, when sleepy, is 5-8 blinks/minute. For the duration of blinks per minute when you are not sleepy, it is 13-17 blinks/minute, both in morning, afternoon and evening light conditions, as well as at night. The system can work well, using a Raspberry Pi 4, and successfully turns on the LED, sounds the buzzer, detects drowsiness and displays the condition in character text on the 16x2 alphanumeric LCD.

REFERENCES

*Corespondent e-mail address

Peer reviewed under reponsibility of Muhammadiyah University Ponorogo, Indonesia

© 2024 Muhammadiyah University Sidoarjo, All right reserved, This is an open access article under the CC BY license()

Received: 2024-04-05

Accepted: 2024-04-15

Published: 2024-04-30

DAFTAR TABEL

Table 1 . Testing of the device's distance measurement detection system towards the driver

DAFTAR GAMBAR

Figure 1 . Block Diagram of How the System Works

Figure 2 . Coordinates of points on Facial landmarks

Figure 3 . Display software flow on Drowsiness detection process system

Figure 4 . Eye ratio in closed eyes position

Figure 5 . Eye ratio in open eyes position

Figure 6 . Component layout

Figure 7 . Hardware box with I/O interface

Figure 8 . Hardware implementation in motorized vehicles

Figure 9 . Testing EAR readings on raspberry Pi 4

Figure 10 . Graph of test results comparing the duration of detection compared between the reference theory and system testing in daylight conditions

Figure 11 . Graph of test results comparing the duration of detection compared between the reference theory and system testing in light conditions at night

References

1. “Angka Kecelakaan Masih Tinggi, Kelalaian Pengemudi Jadi Faktor Utama.” Accessed: Apr. 05, 2024. [Online]. Available: https://otomotif.kompas.com/read/2021/03/31/193100715/angka-kecelakaan-masih-tinggi-kelalaian-pengemudi-jadi-faktor-utama
2. V. Irawan, A. Rizal, and I. Purnamasari, “Penerapan Algoritma K-Mean Clustering Pada Kecelakaan Lalu Lintas Berdasarkan Kabupaten/Kota Di Provinsi Jawa Tengah Tahun 2020,” Jurnal Ilmiah Wahana Pendidikan, vol. 8, no. 10, pp. 293–300, 2020, doi: 10.5281/zenodo.6820090.
3. I. Imanuddin, R. Maulana, and M. Munawir, “Deteksi Mata Mengantuk Pada Pengemudi Mobil Menggunakan Metode Viola Jones,” JOINTECS (Journal of Information Technology and Computer Science), vol. 4, no. 2, p. 45, Jul. 2019, doi: 10.31328/JOINTECS.V4I2.1005.
4. A. Hidayati and L. Y. Hendrati, “Traffic Accident Risk Analysis by Knowledge, the Use of Traffic Lane, and Speed,” Jurnal Berkala Epidemiologi, vol. 4, no. 2, p. 275, Feb. 2017, doi: 10.20473/jbe.v4i2.2016.275-287.
5. A. H. K. T. Mustofa, “Sistem Peringatan Dini Menggunakan Deteksi Kemiringan Kepala pada Pengemudi Kendaraan Bermotor yang Mengantuk,” JURNAL TEKNIK ITS , vol. 7, no. 2, pp. 281–286, 2018.
6. T. Herlambang et al., “Developing Design Of Automatic Egg Quality Detector Using Roi And Rgb Template Methods,” BAREKENG: Jurnal Ilmu Matematika dan Terapan, vol. 16, no. 2, pp. 569–582, Jun. 2022, doi: 10.30598/BAREKENGVOL16ISS2PP569-582.
7. R. Sulistyowati, A. Suryowinoto, H. A. A. Sujono, and I. Iswahyudi, “Monitoring of road damage detection systems using image processing methods and Google Map,” IOP Conf Ser Mater Sci Eng, vol. 1010, no. 1, 2021, doi: 10.1088/1757-899X/1010/1/012017.
8. B. Hartiansyah, “Deteksi Dan Identifikasi Kondisi Kantuk Pengendara Kendaraan Bermotor Menggunakan Eye Detection Analysis,” JATI (Jurnal Mahasiswa Teknik Informatika), vol. 3, no. 1, pp. 59–64, Feb. 2019, doi: 10.36040/JATI.V3I1.541.
9. R. Kuswara, “Aplikasi pendeteksi mata mengantuk berbasis citra digital menggunakan metode haar classifier secara real rime,” Nov. 2013, Accessed: Apr. 05, 2024. [Online]. Available: http://elib.unikom.ac.id/gdl.php?mod=browse&op=read&id=jbptunikompp-gdl-randykuswa-32320
10. M. F. Yusri, P. Mangat, and O. Wasenmüller, “Detection of Driver Drowsiness by Calculating the Speed of Eye Blinking,” Oct. 2021, [Online]. Available: http://arxiv.org/abs/2110.11223
11. A. Islam, N. Rahaman, and M. A. Rahman Ahad, “A Study on Tiredness Assessment by Using Eye Blink Detection,” Jurnal Kejuruteraan, vol. 31, no. 2, pp. 209–214, Oct. 2019, doi: 10.17576/JKUKM-2019-31(2)-04.
12. “A new vision for center-based engineering research,” A New Vision for Center-Based Engineering Research, pp. 1–88, Aug. 2017, doi: 10.17226/24767.
13. A. Suryowinoto, T. Herlambang, R. Tsusanto, and F. A. Susanto, “Prototype of an Automatic Entrance Gate Security System Using a Facial Recognition Camera Based on The Haarcascade Method,” J Phys Conf Ser, vol. 2117, no. 1, p. 012015, 2021, doi: 10.1088/1742-6596/2117/1/012015.
14. C. T. Hsieh, “Development of a Low Cost and Raspberry-based Thermal Imaging System for Monitoring Human Body Temperature,” Proceedings of Technical Papers - International Microsystems, Packaging, Assembly, and Circuits Technology Conference, IMPACT, vol. 2021-December, pp. 248–251, 2021, doi: 10.1109/IMPACT53160.2021.9696451.
15. K. Vijay, W. R. Sabarish Abishek, V. U. Sabarish, and R. Sanjeev Krishnan, “Private Cloud Storage using Raspberry PI via Virtual Network Computing - An Analysis,” 2023 International Conference on Computer Communication and Informatics, ICCCI 2023, 2023, doi: 10.1109/ICCCI56745.2023.10128489.
16. V. Uma Maheswari, R. Aluvalu, M. V. V. Prasad Kantipudi, K. K. Chennam, K. Kotecha, and J. R. Saini, “Driver Drowsiness Prediction Based on Multiple Aspects Using Image Processing Techniques,” IEEE Access, vol. 10, pp. 54980–54990, 2022, doi: 10.1109/ACCESS.2022.3176451.
17. M. Ma and J. Wang, “Multi-View Face Detection and Landmark Localization Based on MTCNN,” Proceedings 2018 Chinese Automation Congress, CAC 2018, pp. 4200–4205, Jul. 2018, doi: 10.1109/CAC.2018.8623535.
18. J. Chandiwala and S. Agarwal, “Driver’s real-time Drowsiness Detection using Adaptable Eye Aspect Ratio and Smart Alarm System,” 2021 7th International Conference on Advanced Computing and Communication Systems, ICACCS 2021, pp. 1350–1355, Mar. 2021, doi: 10.1109/ICACCS51430.2021.9441756.
19. S. K. Parai, T. Singhadeo, S. Gain, A. Dutta, A. Palui, and D. Chatterjee, “Face Recognition through Landmarks to Find Accurate Distance Between Two Landmarks Using Pythagorean Algorithm,” 2023 14th International Conference on Computing Communication and Networking Technologies, ICCCNT 2023, 2023, doi: 10.1109/ICCCNT56998.2023.10307052.
20. P. Singh, S. Mahim, and S. Prakash, “Real-Time EAR Based Drowsiness Detection Model for Driver Assistant System,” 2022 International Conference on Signal and Information Processing, IConSIP 2022, 2022, doi: 10.1109/ICONSIP49665.2022.10007514.