Occlusion Robust Low-Contrast Sperm Tracking Using Switchable Weight Particle Filtering

Sperm motility analysis has a particular place in male fertility diagnosis. Computerized sperm tracking has an important role in extracting sperm trajectory and measuring sperm’s dynamic features. Due to free movements of sperms in three dimensions, occlusion has remained a challenging problem in this area. This paper aims to present a robust single sperm tracking method being able to handle misdetections in sperm occlusion scenes. In this paper, a robust method of segmentation was utilized to provide the required measurements for a switchable weight particle filtering which was designed for single sperm tracking. In each frame, the target sperm was categorized in one of these three stages: before occlusion, occlusion, and after occlusion where the occlusion had been detected based on sperm’s physical characteristics. Depending on the target sperm stage, particles were weighted differently. In order to evaluate the algorithm, two groups of samples were studied where an expert had selected a single sperm of each sample to track manually and automatically. In the first group, the sperms with no occlusion along their trajectories were tracked to depict the general compatibility of the algorithm with sperm tracking. In the second group, the algorithm was applied on the sperms which had at least one occlusion during their path. The algorithm showed an accuracy of 95% on the first group and 86.66% on the second group which illustrate the robustness of the algorithm against occlusion.


Introduction
Human semen analysis is an important experiment in male fertility diagnosis in which morphology and dynamic characteristics of sperms are examined.The semen analysis can be performed manually or automatically.The manual semen analysis depends on the operator's experiences and skills, and the results can be affected by human errors as well.These imperfections and the brilliant abilities of computerized methods in fast and accurate distinguishing of sperm movement have motivated researchers to develop the computerized tracking algorithms.Usually sperms are divided into four groups in terms of motility [1]: 1) Rapid progressive motility; 2) Slow progressive motility; 3) Non-progressive motility; 4) Immobility.
Because of free movements of sperm in semen, mathematical models cannot explain all the above categories.Moreover this diversity in sperm movements compounds the tracking problems.
From the decade of the 1980 designing sperm imaging systems, tracking algorithms, and computerized analysis became center of attention broadly and valuable works were published in this area as well [2]- [4].Some of them dealt with supplementary tools like acoustic device [3], piezo-electric device [5] and lens-free on-chip imaging technique [6] to provide 3D trajectory of sperm.Some others utilized optical tweezers to measure both sperm motility and energy [7] [8].In [9], a lens-less charge-coupled device (CCD) and a microfluidic system were used to improve field of view (FOV) of microscope and provide automatic recording of sperm.
In another aspect, it has been tried to develop algorithms in order to achieve more accurate and robust sperm trackers.Using visual evaluation of microscopic field [10], template matching [11] [12], particle and Kalman filters [13] [14], nearest neighbor technique [15], time differential method [16], optimal matching [17] co-registration process based on block matching [18], high-speed visual feedback [19], and optical flow [20] are some tracking algorithms for sperm cells employed so far.
In computer assisted sperm analysis (CASA) system, multi-object tracking algorithm with specific standards is utilized [21].These systems have board applications for human and animal sperm analysis and in-vitro fertilization (IVF) as well [22] [23].CASA systems are dealt with from a variety of angles like their capability [4] [24], comparison of existed methods [25] [26], accuracy and precision [27] [28], and quantitative analysis [29] [30].This paper attempts to improve the algorithmic approach by providing occlusion robust single sperm tracking algorithm.

Object Restrictions
In this paper, the following object restrictions were assumed: • Imaging system is source of some noises and disturbances such as the errors caused by slides, mirrors, microscope lenses, camera lenses, and ambient.It is assumed that these factors remain constant in all frames and are not affected by the sample's or the sperm's movements.• The sperm can swim out of the plane of focus [31].Consequently, average intensity for each sperm head changes over time.• The sperm may occlude along its path especially, when the semen has a high concentration.In this case, the algorithm should be able to make a distinction between target sperm and the other ones.• Directions of sperm head and movement are not necessarily the same, specially, when the sperm belongs to the first or second movement category.• There are some local changes in background, hence it couldn't be considered static.

Preprocessing
In order to reduce the undesirable effects caused by imaging system, freely movement of sperm, the following preprocessing algorithms were used.

• Smoothing
Based on report of WHO [1], a progressive sperm can move more than 100 μm per second.In [32] it is discussed that based on selected sampling rate sperm's motility characteristics are partly different.In this paper, image acquisition was performed with 30 fps.Thus, rapid progressive sperm's movements were unconnected to some extent.To obviate the problem, a sampled spatiotemporal Gaussian filter was utilized [33]: where Σ is a 3 × 3 covariance matrix and p = (x, y, t) denotes a pixel position (x, y) at time t.In practice, a separable kernel was used in which the standard deviation for every dimension was 1.5 and its length was 5 pixels.
• Morphological filtering Proper use of morphological filters, gives this opportunity to employ the appearance-based information for the objects and also partly expunges undesirable cells.In addition to sperm cells, human semen includes blood cells and cytoplasmic parts.According to WHO, size of sperm head is 3 -5 μm which is a useful constraint to seclude sperms from other types of cells [1].Nonetheless, in some samples, non-sperm cells move slowly and locally which misleads motion-based segmentation methods into misdetection.In this paper, a sperm-shaped top hat filter was employed to address this problem.Size of the structure element was designed considering sperm head size.

Background Removing
Although the morphological filtering reduces the background to some extent, some debris still appear in the images.Furthermore, the images are labeled by the camera.Therefore, a proper background removing algorithm was utilized.Because of some unwanted local slow movements, the statistical methods did not have enough proficiency.Therefore, an adaptive temporal median filter was employed to detect the background.In this method, original image was defined as sum of background and foreground as follows: where α denotes the update constant changing in each step.I and B depict foreground (sperm objects) and background intensity respectively [34].The median changes in accordance to standard deviation and length of time series in each point.Moreover, if a local ramp arises in the median, it will be modified.Finally, median measures are limited by Chebyshev constrain whose magnitude is 0.777 time of standard deviation of the data.

Particle Filter
Generally, particle filtering is a method for estimating probability distribution.In the other words, particle filtering is a sequential Monte Carlo estimation using importance sampling method.In this method, particles are utilized for point representation of probability distribution and are allotted weights to determine the probability in each point.This is the important sampling technique which determines particle weights.To use particle filtering, it is only needed to express the problem in state space.If t x is a state variable at time t, then the particles and their weights are shown as { } x W and posterior probability is approximated by: ( ) ( ) where t Y denotes measurements.Based on the Bayesian network principal, posterior probability can be approximated by computing ( ) . Nonetheless, computing ( ) is in need of solving complex integrals [35].Therefore, reaching optimum solution based on approximate integrations is impracticable.Instead, importance sampling is a competent candidate to solve this problem in which a specific distribution q is used to generate random samples.Assuming chain rule and Markovian condition on the state variable ( ) x t and condi- tional independency of ( ) y t , the weights can be obtained using importance sampling as follows [35]: where . By rewriting the equation as recursive form, we have: The likelihood or the transition probability is mostly chosen as the proposal distribution.Using the prior dis-tribution instead of the proposal distribution makes the sampling easier and accelerates the weighting process.
Replacing q with ( ) 4), final normalized weights will be obtained as follows: This process is repeated in each step to approximate the state variable [35].

Object Representation
The first step of each tracking algorithm, is selecting a proper type of representation [36].Based on a specific application and object characteristics, different types of representations are utilized.Of all object representation types, point representation is the simplest and most popular one causing decrease in computational costs.This is often employed where objects are small [36].Sperm cells occupy small areas on every frame.In addition, the subsequent analysis such as computing percent motility and curvilinear velocity are defined based on point localization.Therefore, in this paper the point representation was used to locate the sperms.Due to free movement of sperm, the sperms do not have invariant appearances and their average intensity may change a long time.Therefore, object representations which are based on shape and boundaries of an object were not considered in this paper.Figure 1 depicts five consecutive frames of a singular sperm.Each frame includes sperm elliptic boundary and a line which illustrates its head direction.It shows that the head direction doesn't change necessarily smoothly.In conclusion, in this study a head-independent sperm tracking algorithm was addressed.

State Space Model
In this paper, in order to model sperm movement the expansion of Taylor series around sperm head position was used in which center of the target sperm is shown by variable x.This expansion around x leads to the sperm motion equation: where ( ) , x x y = and T denotes the time interval between two consecutive frames.Although, using higher order equations bring complex models, a large number of particles are required to reach accurate estimation.So, because of calculating higher order derivatives the computational cost extremely goes up.Hence, truncated Taylor series including only first order derivative was used and the rest of series were modeled by a Gaussian random process denoted by t v [37].For simplification, a vector representation was utilized in which two-dimensional (2D) position and velocity explain the state of motion.The state vector variables is where ( ) x y denotes the target position and ( ) x x y =    shows its velocity.Equation ( 9) can be written as: where A is the transition matrix and C shows the gain matrix [17].

Likelihood
Euclidian distance is common criterion to measure the likelihood.In this paper also the distance between the positions and the velocities were computed for the particles. where shows the most probable measurement for target sperm position.For combining normalized distances, a simple definition was used.
After acquiring total distance, the likelihood probability can be calculated as: ( ) where σ denotes noise standard deviation of measurements [18].

Segmentation and Detection
Watershed segmentation algorithm is based on topography of intensity.This method comprises principle concepts of other segmentation methods such as thresholding, edge detection and growing region.Furthermore, Watershed includes contiguous boundaries [38].In this method based on topography of intensity, each area in an image is divided into three categories.First category, called minimum regions, belongs to local minimum intensity areas.Second category, called catchment basin, depicts a set of points if a droplet of water drips there, then, it slips down towards a specific minimum region.Third category, called watershed lines, refers to the areas where droplets of water have the same chance to slip down towards adjacent minimum regions.
The main goal in this method is finding watershed lines.At the beginning, assume that minimum regions are punched and water gradually goes up from the minimum intensity to the maximum.If level of water is determined by l ; magnitude of lwill change from minimum intensity up to maximum.If C − and l T as follows: 1) 2) and includes only one flooded catchment basin under level and includes more than one flooded catchment basin under level 1 l − .Third state happens just when a new catchment basin is flooded with water, so a new dam is constructed to prevent.
The watershed algorithm suffers from over-segmentation [38] especially in occlusion senses, moreover the smoothing filter decreases the possibility of discrimination between occluded sperms.Thus, in this study, a local region around the single target was used to apply Otsu thresholding [39], then a binary watershed algorithm based on specific distance transform was employed to reach final sperm segments.The experience showed that this approach reduces the over-segmentations.Eventually, the segments whose sizes were in the range of a sperm were labeled and their centers were determined.
In binary watershed algorithm, the distance between each 0 pixel and the nearest 1 pixel is defined as basis of the transform.The distance has different definitions, however based on [40], the best definition for segmenting correlated objects in binary images is the chessboard distance which is defined as: Therefore, the chessboard distance was applied for binary watershed segmentation.
In order to avoid superfluous measurements, only a specific neighborhood around the estimated position of the target sperm atwas considered.All detection process was performed in this area called "search window".On one hand, if search window is too small (i.e. as same as a sperm head), the algorithm will not be able to distinguish whether an occlusion has been occurred or not.On the other hand, if it is too large, it may accommodate several sperms which bewilder the algorithm.Hence, there is a tradeoff between large window and algorithm complexity.The image system provided 2 μm pixel resolution under which the progressive sperms were able to move more than two pixels in each frame.Moreover, watershed algorithm needed to compute fast fourier transform (FFT) for which a window of size 2 m brings faster performance.Consequently, a 2 2 m m × search window was assigned for the detection.

Tracking and Occlusion Detection
The tracking began by selecting a singular sperm in a frame.Center of each detected segment which had fulfilled the sperm size constraint was considered as k y (see Figure 2).Since the search window had a specific size, only limited number of sperms was detected.Afterward, the distance between k y and last position of the target sperm at 1 t − was computed.Finally, position and displacement of every k y were saved as the mea- surements.Figure 2 shows search window at t where detection algorithm labels only two of four segments.The two segments, smaller than sperm head, are rejected.Also y 1 and y 2 are taken part in target selection where the two arrows in Figure 2 represent their displacements.
In this paper, in order to improve target detection and avoid mistracking after occlusions, three stages were defined for target sperm called "before occlusion", "occlusion" and "after occlusion".
• Before occlusion shows that the target sperm never had an occlusion along its trajectory or it has passed occlusion stage already.
• Occlusion shows the stage when the detection algorithm is not able to make distinction between the target and occluded sperms.The basic idea is that detecting a mass, remarkably larger than sperm size, is the sign of occlusion.• After occlusion shows the stage when an occlusion has been obviated and detection algorithm is able to separate the target again.If the target sperm is in before occlusion or occlusion stages, the closest k y to approximate state x MMSE will be the most likely position for the target sperm shown by K y .The main problem appears when the target sperm is in after occlusion stage, since the nearest distance cannot be trusted any more.In this case, history of the target sperm movements is utilized to discern it.Hence, the average direction and velocity of the target along three prior frames were calculated: ( ) where o t denotes the moment at which the sperms has just separated.Thus, K y after occlusion is determined by the estimated position of the target sperm using * , a a x θ  .Following the determination of K y , those particles which are closer to K y in terms of position and velocity, are allocated more weights.The distance between measurements obtained from K y and particles were com- puted by Equations ( 12), ( 13) and ( 14) and the likelihood was measured by Equations ( 15) and ( 16).

Implementation
200 particles were used for tracking.On the assumption that t v is uncorrelated, particles were propagated throughout the search window where their initial weights had been assigned 1/N.Then, the weighting was performed using likelihood function.In addition, the systematic resampling was utilized to avoid generating ineffective or too dominant particles [41].Finally, using minimum mean square error estimation, approximate state was computed as follows: Using MMSE x , next position of search window was determined:

Results and Discussions
In this section, different steps of the sperm tracking algorithm used for the microscopic sperm images are discussed.
Figure 4 shows the result of morphological filtering and background removing.As shown in Figure 4, not only the sperms were highlighted, but also average intensity of the background was decreased intensely so that the sperms were observed as bright objects.
As the second step in preprocessing, the median adaptive background removing technique was responsible for eliminating imaging artifacts and very slow moving non-sperm cells.Figure 4(c) displays the preprocessed image after adaptive median background removing and open filtering.In the obtained image both the marker and noisy pixels have been disappeared drastically.
After preprocessing, in tracking, this is the sperm occlusion determines approach of the algorithm in selecting proper criteria for diagnosing the target sperm.Figure 5 schematically exhibits the algorithm manner toward sperm occlusion via the three consecutive frames.The solid ellipse shows the target and the dashed one denotes a pesky sperm. 1  y and 2 y are the detected segment centers, t is time, and the dashed square refers to the search window.The small solid circles show the previous target sperm positions selected by the algorithmwhile the small hollow circle shows the predicted position for the target sperm in each frame.In the frame Figure 5(a), the target is in the before-occlusion status and the arrows reveal the distance between x MMSE and the segments  y .According to the nearest distance criterion 1 y is selected as the sperm position.In the frame Figure 5(b), the occlusion occurs so that the watershed algorithm detects a unique segment.So, merely 1  y is observed and considered as the sperm position.Eventually in frame Figure 5(c), the pesky sperm which is assumingly unwilling to travel long distance, causes misdetection, if the nearest distance criterion is applied for tracking.In such scenarios, in this paper the average velocity was used to predict the new target position as shown by the hollow square.Afterward, proximity to the new point was the criterion for 1 y and 2 y .Figure 6 represents operation of the algorithm for real data in which 4 consecutive frames discover a real sperm occlusion scene.In order to assess the method, two groups of patients were established.The proficiency and compatibility of the method with the categories of sperm motility was evaluated using the first group including 20 videos where a target sperm from different movement categories was selected from each video so that it had no occlusion along its trajectory.In contrast, to evaluate the robustness of the algorithm against the occlusion, 30 videos were considered so that the chosen sperms had one or more occlusion during their paths.
In the first hand, an expert was asked to label the target in each frame.Then, every sperm was tracked by the algorithm.The comparison between the manual track and the algorithm result was fulfilled by object tracking error (OTE) [21] which measures errors in every frame as: where M x denotes the manual sperm position and n is number of frames until the sperm exits the scene.In each group, the average tracking errors was determined.In the case that a sperm hides or the tracker misses the target, the correspondent error extremely increases.Thus, in order to avoid the misconception caused by this phenomenon, average error was computed for only sperms which were tracked completely.Table 1 illustrates the results of low contrast sperm tracking for both groups.Of all 20 sperms with no occlusion in their path, only one sperm was missed.This can be interpreted as the ability and compatibility of the algorithm in tracking all sperm categories.Figure 7 shows the trajectories detected by the algorithm for the sperms of different types of motility.
Also, the algorithm achieved satisfactory results in the case of occlusion scenes.In this respect, Table 2 shows the accuracy and the average tracking error in each group.When a sperm is crossing another, it is hard to locate the target sperm even for an expert, therefore the second group shows higher average tracking error.However, through the eyes of specialists, this range of error has trivial influence on subsequent examinations.

Conclusions
This paper described different phases of single sperm tracking including preprocessing and localization with special attention to be devoted to properly adapting them to moving sperm objects and the occlusion problem.
Although it has been trying to cope with sperm occlusion through advanced imaging device such as laser and acoustic waves, this paper attempted to find a solution by virtue of a sheer algorithmic approach which helps small laboratories to be involved with this research field without having access to modern electronic equipments.
In the case of preprocessing, the sperm movements were smoothed to reduce the effect of the low sampling rate, using a spatio-temporal Gaussian filter.In addition, an adaptive background removing technique was applied to expunge the impact of non-sperm moving objects and undesirable artifacts in the slides.Although appearance-based methods, such as template matching, are able to segment low-contrast sperms, they are hardly able to separate the target sperm from occluded one because low-contrast sperms do not adhere strictly to shape-based regularities.Nonetheless, it is still possible to take advantage of some constraints on sperms' shape and size.In this paper, object-oriented morphological filters and the size constraints were used to feature the sperm objects and eliminate the waste debris.
On one side, complex segmentation methods increase computations.On the other side, simple methods are    not effectively able to cope with the sperm occlusion problem.Gray Watershed algorithm counts as a powerful segmentation method, however internal changes of intensity in a small object causes over-segmentation in Gray Watershed method.Therefore, in this paper, the binary watershed algorithm based on the chessboard distance transform was utilized.In addition, combination of the thresholding and the binary watershed algorithm provided high-efficiency sperm segmentation.The proposed low-contrast sperm tracking method is a plain and efficient solution to the occlusion problem.In contrast to statistical tracking methods, selecting complex weighting techniques to encompass all sperm movement categories, the proposed method defines three stages for each target sperm to reduce the composi-tional cost and improve the performance of tracking.In this regard, extra computations are carried out only when an occlusion occurs which saves the operation time.In addition, all computations were limited to a search window.It was designed so that it had the minimum size for searching space and was large enough to monitor sperm occlusions.
The result showed that the switchable weight particle filtering has the great ability to track low-contrast sperms.It provided the opportunity to track the sperms based on their motion models instead of the head directions.In this paper, the weights were given so that the particles were congregating around the target position.Therefore, it is expected that the performance of the algorithm can be improved, using more complex proposal distributions and sampling methods.

Figure 1 .
Figure 1.Variation in size, shape and direction of a low contrast sperm during image acquisition.

1 lC
− denotes the regions relied under the surface of water level 1 l − , l T will comprise the regions under level l .Therefore, there are three possible states between 1 l

Figure 2 .
Figure 2. Labeling sperms in the search window.Of all detected objects only the two meeting the sperm constraints are labeled as y 1 and y 2 .

Figure 3
Figure 3 depicts flowchart of the algorithm which explains how this algorithm uses the measurements to change the weights of the particles regarding the stage of the target sperm.

Figure 3 .Figure 4 .
Figure 3. Flowchart of the proposed low contrast sperm tracking algorithm.

Figure 5 .
Figure 5.The scheme of sperm tracking in a occlusion scene.(a) Before occlusion; (b) Occlusion; (c) After occlusion.

Figures 6 (
a1)-(a4) display distance transform of the search area around the target sperm after thresholding.As you can see in this raw (a) although the chessboard distance has usually acceptable performance for sperm discrimination however, some cases like Figure6(a2) challenge it.Therefore, based on size of the detected object an occlusion is flagged.Figures 6(b1)-(b4) denote object boundaries corresponding to Figures 6(a1)-(a4) including single points referring to the target sperm positions selected by the algorithm.The marker points show the algorithm success in coping sperm occlusion problem.

Figure 6 .
Figure 6.Four consecutive frames showing a target sperm from before collision until distancing from the other sperm.(a) Denotes chessboard distance transform; (b) Watershed segmentation and labeling the target sperm.

Figure 7 .
Figure 7. Representation of the performance of the algorithm in tree types of sperm movement.(a) Rapid progressive motility; (b) Slow progressive motility; (c) Non-progressive motility.

Table 1 .
The results of sperm tracking for two groups.

Table 2 .
The accuracy and error of the algorithm for two groups.