Activation Function: Cell Recognition Based on YoLov5s/m

Activation functions play a critical role in neural networks. The paper mainly studies activation functions with four activation functions that were the selec-tion for reference and comparison. The Mish activation function was ex-pending as the Mish_PLUS activation function, the Sigmoid activation function, and the Tanh were combined to obtain a new Sigmoid_Tanh activation function. We used the recently popular YoLov5s and YoLov5m as the basic structure of the neural network. The function realized in this article was the recognition function of red blood cells, white blood cells, and platelets. Through the role and comparison of different activation functions in the neural network structure, the test results show that, in this paper, the training precision curve under the Sigmoid_Tanh activation function was better than that under the action of other activation functions. That means that the accuracy of cell recognition under the activation function was higher.


Introduction
The blood of an organism contains many components. It was one of the tasks of biomedical research to find out the cells in the blood, for instance, using some methods rapidly and accurately screening of red blood cells, white blood cells, platelets, etc. [1] [2] [3]. The application of artificial intelligence technology in the biomedical field provided a critical research theory for the development of biomedical. Using artificial intelligence recognition technology is one of the methods to screening out different types of cells in biological blood for example, adopting machine learning and deep learning methods to screen diverse cells types [4] [5]. The previous means have an application value for screening and identifying different cells types, but it was not perfect. How to screen diverse cells quickly, efficiently, and accurately is one of the research fields of biomedicine.
Activation functions were a critical part of the design of a neural network. It helps the neural network to learn the complex patterns in the data, just like the neuron-based model in the human brain. The activation function obtains information from the front neuron and then transmits it to the next neuron [6]. As shown in Figure 1, in the neuron, Input is also applied to a function after weighting and summing. This function was the activation function. The activation function was an introduction to increase the nonlinearity of the neural network model. Each layer without the activation function was equivalent to matrix multiplication. However, the neuron information of the previous layer cannot transfer to the neuron information of the next layer. The function can realize the transmission of neuron information, so the activation function was an indispensable part of an artificial neural network. To improve the computational performance of neural networks, people have studied the activation functions in neural networks. Common activation functions include Sigmoid [7], Tanh [8], SiLU [9], Hardswish [10], Mish [11], MemoryEfficientMish, etc.
The SiLU activation function was also called the Swish activation function. The Swish was an activation function proposed by the Google team in recent years and was composed of the previous activation function. The expression of the is the Sigmoid. Because the saturation of the Sigmoid tends to cause the gradient to disappear, learn from the effect of Re-LU, when x → +∞ , then ( ) f x x → , but when x → −∞ , then ( ) 0 f x → , the general trend of the function is similar to ReLU but more complicated than Re-LU [12]. It is a reflection in the function by adding a few hyper-parameters and then showing more characteristics. Adding a hyperparameter to Swish makes the function expression ( ) ) ( f x x x σ β = * , and we say that β can be a constant or a trainable parameter. Although this Swish nonlinearity improves accuracy, its cost is non-zero in an embedded environment. It is much more expensive to calculate the Sigmoid function on a mobile device. The author of MobileNetV3 used Hardswish & Hardsigmoid to replace the Sigmoid layer in ReLU6 & SE-block. But only in the latter  , and the value range is ( ) 0, +∞ . Softplus can regard as the smoothness of ReLU. The Mish activation function image is shown in Figure  2.
The MemoryEfficientMish activation function was the first derivative of the Mish activation function, and its formula was x x  In this paper, the activation function was the research object mainly. Two kinds of neural network morphology in YoLov5 are used as an experimental basis to conduct experiments on different activation functions and explore the accuracy influence of different kinds of activation functions on cell recognition.

Mish_PLUS Activation Function
Given the excellent performance of the Mish activation function in YoLo, this article expands the Mish activation function to obtain the Mish_PLUS activation function and use the Mish_PLUS activation function to the YoLov5s and Yo-Lov5m neural network structures. The formula of the Mish_PLUS activation function is like (6). In the Mish_PLUS activation function ( ) It can be seen from the comparison of Figure 2 and Figure 3 that when x approaches zero to the Mish activation function approaches zero from a negative value. The Mish_PLUS activation function approaches zero from a positive value.

Sigmoid_Tanh Activation Function
At present, there are dozens of activation functions widely used in neural network structures, and different activation functions have different roles in neural networks. And their advantages and disadvantages are also so. Commonly used activation functions include: Sigmoid, Tanh, ReLU [14], Leaky ReLU [15], ELU [16], SELUs [17], GELUs [18], PreLU [19], MaxOut [20], RReLU [21], etc., and some activation functions are expanded based on the original activation function to obtain a variant of the activation function. In this article, the classic Sigmoid and Tanh are mainly studied. The Sigmoid formula and derivative are: The Tanh formula and derivative are: 1 tanh e e The source of formula (7) (7). Since the visualized curve is S-shaped, called S-curve, as shown in Figure 4. Used in the neural network for the output of hidden layer neurons, the value range is (0, 1). It can map a real number to the interval of (0, 1), so it was two classifications in training. The difference in image features is more complicated or the difference is not special large, the effect was better. The advantage of Sigmoid as an activation function is that the curve presented is relatively smooth, and it was easy to derive during the calculation process, as shown in formula (8). The disadvantage is that the activation function has a large amount of calculation. When backpropagating to find the error gradient, the derivation involves division. When backpropagating, the gradient disappears easily, and the training of the deep network cannot complete. The hyperbolic tangent function is a kind of it, called Tanh. In the neural network, Tanh is used as the activation function of the neuron to transmit information. It is a non-linear function. The formula is shown in (9), and its derivative is shown in (10). The Tanh function converts the final result of the fitted curve to the interval of (−1, 1). The maximum negative number is infinitely close to −1, and the maximum positive number is infinitely close to 1. Tanh solves whether the output of Sigmoid is zero centers, but there is still a saturation problem ( Figure 5).   activation function are combined for experimentation. That is to say, the Sigmoid activation function and the Tanh activation function are multiplication to obtain a new activation function, the formula as shown in (11), its value range is (−1, 1), and the obtained visualization curve in Figure 6. We can see from Figure 6 x x

Results
In this paper, on the one hand, inspired by the Mish activation function, the Mish activation function is extended based on the Mish activation function; on the other hand, the Sigmoid activation function and the Tanh activation function choose from the widely used activation functions. Function as the research object, multiply the Sigmoid activation function and Tanh activation function to get a new activation function for the experiment. The codes of Mish_PLUS activation function and Sigmoid_Tanh activation function are in Table 1.
In this paper, YoLov5s and YoLov5m as the neural network structure, and the function realized was the recognition function of white blood cells, red blood cells, and platelets. There is a dataset of blood cells photos, originally open-sourced by https://github.com/cosmicad/dataset. In this paper, the dataset was exported via roboflow.ai on February 23, 2021. There are 874 images across three classes: WBC (white blood cells), RBC (red blood cells), and Platelets. The following sections were the parameter results obtained under the conditions of different activation functions. Table 2 shows the results of different activation functions under the neural Figure 6. Image of Sigmoid_Tanh activation function. We use different activation functions to identify cell types, and the recall and precision are representation, by curves with different colors. Each activation function is training 200 times. Knowledge about recall and precision can obtain from [22]. Figure 7 and Figure 8 are the curves of recall and precision under different activation functions. It can see from Figure 7 that the green curve under the Hardswish activation function shows the worst effect, and the curve change is unstable. When the training reaches 128 times, it displays a turning point in the recall, and the recall increases before the 128 times training. The recall dropped sharply after 128 training sessions and did not change much. The blue curve represents the Sigmoid_Tanh activation function, which presents the best effect. The curve maintains a steady upward trend. When the training reaches

Cell Recognition Results
Under different activation functions, we conduct experiments on the identification of cell types. Figure 10 shows the cell recognition results under activation functions based on the YoLov5s neural network structure. It can see from the sparse degree of cell-type recognition in Figure 10 that the cell type recognition effect under the Hardswish activation function was the worst, and the cell type recognition effect under the Sigmoid_Tanh activation function was the best. The cell type recognition under the Mish_PLUS activation function can also achieve better results, but it was not as good as the cell type recognition under the Me-moryEfficientMish activation function. Table 3 shows the parameter results of different activation functions under the neural network structure YoLov5m. It can be seen from Table 3  For the diverse cell types recognition, each activation function had has been trained 200 times to obtain the recall and precision of the types. Figure 11 and Figure 12 are the curves of recall and precision under different activation functions. It can see from Figure 11 that the red curve represents the result under the      presents a better effect, and the mAP obtains a better score. The red curve under the Mish activation function gets a poor result and the mAP of different types of cell recognition to a poor score. Figure 14 shows the cell recognition results under different activation functions based on the YoLov5m neural network structure. It can see from the sparse degree of cell-type recognition in Figure 14 that the cell type recognition effect under the Mish activation function was the worst, and the cell type recognition effect under the Sigmoid_Tanh activation function was the best. The cell type recognition under the Mish_PLUS activation function can also achieve better results, but it was not as good as the cell type recognition under the Sigmo-id_Tanh activation function.

Discussion
As a necessary condition in the neural network structure, the activation function directly affects results on certain functions. In recent years, the research on the activation function has been intensification. Common activation functions include: Sigmoid, Tanh, ReLU, LReLU, PReLU, Swish, etc. The activation function was also widely used. For example, the activation function used to mobile robots as a robot recognition and the algorithm for understanding the scene; in drone technology, the drone needs to face a complex environment, and the activation function algorithm in machine learning plays an important role in the understanding of the drone scene. This paper mainly studies the activation function, but there were limitations in cell recognition. Mainly as follows: 1) This paper takes cells as recognition objects and tests the performance of recognition cells through different activation functions. If you replace other recognition objects, maybe not be able to achieve the desired effect. 2) There are many common activation functions. There were only compares and tests of the extended activation function with the four activation functions tapes, and does not compare with other common activation functions.
3) Due to the limitation of experimental equipment, there was only a simple training, which has certain restraints for the training results.
4) The research in this paper focuses more on recognition, which needs to experiment under a high-power microscope in practical application.
Finding an efficient and suitable activation function is the subject of future research. In the future, the activation function needs further research.

Conclusion
This paper focuses on finding a suitable activation function for research. Using YoLov5s and YoLov5m as the basic structure of the neural network, the activation functions SiLU, Hardswish, MemoryEfficientMish, Mish, Mish_PLUS, Sigmoid_Tanh were a test. The Mish_PLUS activation function was an improved form of Mish, and Sigmoid_Tanh combines the Sigmoid activation function and Tanh activation function to a new Sigmoid_Tanh activation function. The data set used in this article was the BCCD.v4-416x416 data set, which realizes the function of identifying red blood cells, white blood cells, and platelets. The test results show that the Sigmoid_Tanh activation function obtained in this paper can play a positive role in cell recognition.

Conflicts of Interest
The author declares that there is no conflict of interest regarding the publication of this paper.