Many systems of handwritten digit recognition built using the complete set of features in order to enhance the accuracy. However, these systems lagged in terms of time and memory. These two issues are very critical issues especially for real time applications. Therefore, using Feature Selection (FS) with suitable machine learning technique for digit recognition contributes to facilitate solving the issues of time and memory by minimizing the number of features used to train the model. This paper examines various FS methods with several classification techniques using MNIST dataset. In addition, models of different algorithms (i.e. linear, non-linear, ensemble, and deep learning) are implemented and compared in order to study their suitability for digit recognition. The objective of this study is to identify a subset of relevant features that provides at least the same accuracy as the complete set of features in addition to reducing the required time, computational complexity, and required storage for digit recognition. The experimental results proved that 60% of the complete set of features reduces the training time up to third of the required time using the complete set of features. Moreover, the classifiers trained using the proposed subset achieve the same accuracy as the classifiers trained using the complete set of features.