An Experimental System Development for Head Posture Estimation Based on 3-D Images Measurement

Although automobile is an indispensable vehicle to modern life, it also serves as a social problem with a big traffic accident. Among the reasons of traffic accidents, careless driving accounts for the largest part. So in order to avoid the careless driving, a system which can measure the posture of a driver and warns driver to drive carefully in the case of looking aside is necessary. Although the image measurement method is used broadly, there is a problem on which measurement accuracy is influenced by environment light, makeup of the driver, etc. in the general method based on the two-dimensional image. Therefore, in this study, we propose an image measurement method to obtain the head posture of driver. First we use three-dimensional measurement method which based on the infrared pattern projection to get 3-D information of head, and then we calculate the angle for faces. In this paper, we explain the composition method of an experiment system, and the results of head posture measurement experiment.


Introduction
Although automobile is an indispensable vehicle to modern life, it also serves as a social problem with a big traffic accident.
Figure 1 shows the statistics of the causes of traffic accidents in 2012 from the Japan Metropolitan Police Department.It shows that in violation of the rules of safe driving is 58.4%.Among them, 30.5% are haphazard driving, and safety careless driving accounted for 16.7% [1].
In order to prevent such problems, it is necessary that the driver bear in mind the safe driving consciously.But it can't keep safe driving according to psychological factors and physical factors.Therefore, a system which can warn the driver and determine the operating conditions in an objective way is necessary.
In recent years, techniques for measuring and quantitative evaluation whether the driver is looking aside from the outside have been proposed.There are two methods about the careless driving judgment, one is mainly according to the viewing direction of the driver, another is according to the face direction of the driver.
The first method is taking a 2-D color image of the driver's face [2].Through the position of the pupil and the iris of eye, the sight line can be detected.However, this method can't detect the face direction which is necessary in the judgment of careless driving.The second method is extracting the main parts of the face such as the eyes or mouth [3].And then use the position relationship to detect the face direction.However, it only can detect the face direction between -15 degrees and 15 degrees, it is too narrow.Besides the two methods, there are many other problems when using C color image, such as the change of environment light, the detection of the face or eyes, the speed problem of calculation, etc.
In order to solve those problems and to build a usable measurement system of head posture of driver, in this study, we propose a 3-D measurement technique based on the infrared pattern projection, and we develop an experimental system to verify the validity of the proposal technique [4,5].

Theoretical Method
Before detecting the rotation angle, we should think about the rapidity and reliability of the method, the influence of the driver and so on.So in this study, to improve the 3-D measurement speed, slit pattern is used.Slit pattern also has the advantage of simple structure and high precision.In the study, I use the pattern which is projected on the nose, so to make sure that there must have one projected on the nose, more than one patterns are needed.But too many patterns will cost more time.So, 3 patterns are just enough to achieve the rapidity and reliability.And to avoid the influence on the driver, infrared pattern projection is used.
In this section, we will introduce the 3-D measurement and the head posture estimation method.

3-D Feature Measurement Based Infrared Slit Pattern Projection
This part describes a 3-D feature measurement method using infrared slit pattern projection method.
Figure 2 shows a schematic of the 3-D feature measurement which is used in the study.
It is composed of an infrared camera and infrared projector.Firstly, we use the infrared projector projected the infrared pattern onto the target object and captured by an infrared camera.Then, we extract the patterns from the observed images which are obtained.At last, we can calculate the 3-D coordinates by using the principle of triangulation from the position information of patterns which is extracted and the positional relationship of infrared projector and infrared camera.
Figure 3 shows the steps of 3-D measurement.
1) The infrared pattern projection In the experiment, we use the information of the pattern which is projected on people's nose.So we should make sure that the pattern is must projected on the nose.When we use only one pattern, it is possible that no one is projected on the nose because of the movement of people's face.This problem can be solved by increasing  the number of the projected slit patterns, but it can be extracted fast and stable when the number of the patterns is small.So in this study, we use 3 slit patterns projected on the driver's face.Besides it, when we use the visible light, it will affect the driver's driving, and the color characteristics of the image can't respond to changes of ambient light rapidly.For these problems, we use infrared slit patterns projection.
Figure 4 shows the observed image, (a) and (b) is the image by a normal camera and an infrared camera respectively.
2) Take the image After projected the infrared slit patterns on people's face, we use the infrared camera to take photos.Figure 5 shows the observed pattern images of each face direction.
3) Extract the projection pattern After obtain the observed images, we should extract the projection pattern.In this study, we use binarization method in appropriate threshold values [6]. Figure 6 shows the extracted pattern images of each face direction.
4) Calculate the 3-Dcoordinates After we extract the projection pattern, we should calculate the 3-D coordinates of the pattern.
In the vertical stereo, using the actual angle of the center coordinates in the vertical direction of the extracted pattern which is (x 1 , y 1 ) and the actual projection angle of the infrared projectors which is (x 2 , y 2 ) to calculate the center coordinates of 3-D coordinates which is  (X,Y,Z) according to the principle of triangulation.The formula is as follows: where, d is the distance between the projector and the camera, and f is the focal length of the camera

Determination of the Head Posture
In the 2.1 part, we have said that there must have one pattern projected on people's nose, then how to make sure of it will be introduced in this part.And in this part, we will also introduce a way of the determination of the head posture.Firstly, we should mark each pattern; secondly, we should extract one which is projected on the nose; thirdly, we should calculate the 3-D form to fit a straight line and calculate the slope; at last, we should simulate a diagram about the slope and the rotation angle.
Figure 7 shows the steps of the face direction estimation.
1) Mark the pattern Firstly, scan the pattern from the start to the end, and then we can obtain the number of the pattern.The number is three.From the first place where the 3 points occurred in the same column, we scan the image again, and then mark the pattern from the top to bottom.The first pattern is red, the second is green, and the last one is blue.
2) Extract the pattern which is projected on the nose After mark the pattern, we should extract one which is projected on the nose.The pattern includes nose which is convex and cheek which is concave.So detecting a concave portion and a convex portion of 3-D coordinates of each pattern, we can extract the pattern through the number of convex portions and concave portions.The detection of the uneven portion is by using first derivative and second derivative.
Figure 8 shows detection principle of the uneven part.Pattern A is projected on the forehead which has three convex-concave portions.Pattern B is projected on the nose which has five convex-concave portions.Pattern C is projected on the chin which has one convex-concave portions.Pattern B which has the largest number of the convex portions and concave portions is the one which projected on the nose.
Figure 9 shows judgment images of each face direction.
3) Calculate the 3-D form to fit a straight line and calculate the slope After we got the pattern which is projected on the nose, we should use principle of triangulation to calculate the 3-D coordinate of the pattern, and use the 3-D coordinate to fit a straight line.
When the degree is more than 20 degrees, we can only detect one side of the pattern, and then we use all of the information to fit a straight line without any judgment which can reduce the computation time.When the degree is between 0 degree and 20 degrees, the pattern has both sides of the pattern, and then we choose one side of the   pattern which has more information to fit a straight line.
When it is 0 degree, we also can get both sides which are the same, so we can get any side of the pattern to fit a straight line.
Figure 10 shows fitting straight line images of each face direction.The blue line is the pattern which is projected on the nose, and the red line is the straight line.
After we got the fitting straight lines, we can obtain the slope of the straight line easily.

4) Simulate a diagram about the slope and the rotation angle
After we got the slope of each straight line, we will find that there is a certain relationship between the slope and the rotation angle.As the angle increases, the slope becomes smaller.Using the least squares method, we can find that their relationship is a straight line.Figure 11 shows the relationship schema about the slope and the rotation angle.

Experiment System Development
In order to verify the effectiveness of the proposed method, we construct an experimental system.The system is constituted by camera, mannequin, infrared projector and software system.To avoid the influence of ambient light, we use an infrared camera.To avoid the influence on driver, we use an infrared projector.And we use c# to build a software interface.
By the system, firstly, we should input the user's login information; secondly, take image of the mannequin by the infrared camera in real time; at last, use the experimental system to calculate the rotation degree and determine whether the driver is careless driving.Figure 12 shows the steps of experiment.

Input the User's Login Information
Everyone's face is different.Some people's faces are relatively small, some large, some long, some round.Therefore, as described in section 2.2, the linear relationship between the curves and angles is different to everyone, so we should input the user's information before we use it.

Take Image in Real Time
In the experiment, we use a camera which has the focal length of 6mm.The image resolution is 1024 * 768 pixels.The infrared projector is Industrial Luminar Ace LA-100 IR.The mannequin is made of reinforced plastic and the height is 36cm, the head circumference is 54cm.

Calculate the Rotation Degree and Judgment
After input the login information, we should use the interface to calculate the rotation degree and determine whether the driver is careless driving.Figure 13 shows an envisioned system interface design which has following functions: initial data entry, start and judgment.When you press the button "Start", the system begin running.If the driver is normal driving, it will show "Normal driving"; if the driver is looking something for a long time, it will show "Please attention and look ahead"; if the driver is rotate a large angle, it will show "Warning!You are careless driving now" and remind the driver that he may cause an accident.But now we still cannot make sure the critical angle of careless driving, so we only can set an adjuster.Figure 14 shows the running result fig-

ures.
If the experimental system is proved that it is feasible to determine whether the driver is careless driving, we can make the actual software to use in the real life.

Experimental Results of the Angle Detection
In the experiment, we use a mannequin and a rotation stage, the mannequin is put on the rotation stage.The true value of rotation angle is determined by the rotation of the rotating stage.We use the theoretical value to compare with the experiment value and we can examine the error.The angle of rotation was measured between −90˚ and 90˚.Table 1 is a summary of the face direction estimation experimental results.The theoretical value and experiment value of the rotation angle can be seen in the Table.1.In the experiment, the model is controlled to turn left or turn right from 0 degree to 90 degrees at the interval of 10 degrees.θ is theoretical value, θ 1 is experiment value, θ ∆ is error value and t is the calculate time from read the image to observe the rotation degree.It can be seen that the average error of the rotation angle is 2.2 degrees.The maximum error is 9 degrees.When the face turn to 90 degrees, the error is large, that is because the information of the 90 degrees pattern is less.
The time of whole experiment can be divided into two parts, which is taking photos and calculating the degree.The time of taking photos is based on the exposure time and the transmission time of image, and in my experiment, the time of taking photos is 116 milliseconds.And the calculating time is shown in the Table.1.The average of calculating time is 29.8 milliseconds by using ordinary computer.The CPU of computer we used is RTM i5-3570, and the main frequent is 3.4 GHz.

Conclusion
From the causes of traffic accidents, we can see that it is important to avoid the careless driving.For it, we proposed the method to calculate whether the driver is careless driving.And we build an experimental system which can prove the correctness of our methods.
In this study, first to improve the 3-Dmeasurement speed, slit pattern is used.And to prevent the influence of projection light on the driver and the influence of the ambient light, 3 infrared pattern projections are used.In the experiment, the model is controlled to turn left or turn right from 0 degree to 90 degrees at the interval of 10 degrees.The average error is 2.2 degrees.It is possible that measurement and processing speed of the proposed technique are usable.But in this study, we still use the mannequin to detect the face direction, so in the future, we will use the real people to do the experiment and make a reliable system used in real life.

Figure 1 .
Figure 1.The traffic accidents reasons scale drawing.

Figure 2 .
Figure 2. A block diagram of the proposed system.

Figure 7 .
Figure 7.The steps of the face direction estimation.

Figure 8 .
Figure 8.Detection principle of the uneven parts.

Figure 11 .
Figure 11.The relationship about slope and rotation angle.

Figure 12 .
Figure 12.The steps of experiment.