1. Introduction
The part of the electromagnetic spectrum, which is sensed by our eyes, is called optical, thereby the visible objects are called optical images. Human visual analyzer was the first “instrument” that enabled to discover the world of microorganisms. As is well known, it was Antonie van Leeuwenhoek who observed single microbial cells at the end of the XVII century by improving vision by simple but ingenious device with a lens, which is called “the Leeuwenhoek microscope”. In 1877, Julius Richard Petri, working in Robert Koch’s laboratory, invented a unique technique to study microorganisms by naked eye. It was microbial cell cultivation in cylindrical plates (now known as Petri dishes) on the surface or within of a gel-like nutrient medium up to visible microbial colonies. Application of photography significantly facilitated the work with microbiological optical images and many mysteries of the microbial world were deciphered by visual observations. Up to now, characterization of microorganisms as optical objects play substantial role in both basic and applied microbiological research. However, all the methods based on the visual examinations are inevitably subjective, in most cases, they are qualitative, and some quantitative approaches are relatively high time and labor consuming. Moreover, it is impossible to “extract” completely and quantitatively all the diverse information, which an optical image contains, by only vision. This information comprises such features as color (or, physically correctly, the spectral properties) and its spatial distribution; size and shape of individual pieces, their mutual position and number; in some cases, it is glowing (e.g. fluorescence), its intensity and spectral characteristics; dynamics of the features. In the second half of the XX century, fundamentally new approach for dealing with these types of information has been developed. It was computer digital image analysis (CDIA).
CDIA is one of the operations of more general procedure called computer digital image processing (CDIP) [1] [2] [3] [4]. CDIP deals with the images (e.g. photographs) obtained in or converted into the digital form. A digitized image is a set of small elements in 2D space called pixels (in 3D space they are called voxels). Each pixel (voxel) contains the digitally coded information on its X − Y − (Z) location in a Cartesian coordinate system and optical features at this point of space. The optical information depends on the imaging system employed. For a monochrome digital cameras, the information obtained consists of the data in gray scale units (often 8 bit 256 gray levels). The digital images from conventional three-channel color cameras and other filter-based imaging systems (color imaging) can be a set of pixels with the optical information encoded in one of the four color spaces RGB, HSV, CIE-Lab, and YCrCb (RGB is the most popular as 24 bit combinations of red, green and blue values with 8-bits for each color). Multispectral imaging systems capture images with information on tens spectral regions (bands) in each pixel. Images from hyperspectral imaging systems provide information on space distribution of many contiguous spectral band close to the continuous spectrum. Multispectral and hyperspectral imaging can visualize the visible light as well as near-infrared to infrared. Finally, the raw digital data of the images are treated by a computer as variables of a huge mathematical model, constructed of various algorithms. The outcome of this treatment can be manipulation of the images for improving their quality and “more expressive” visual appearance of some components and/or extraction of some quantitative digitally coded information and its analysis, i.e. CDIA.
For decades of the development in this field of the Computer Science, a large number of CDIP and CDIA software along with various approaches of their application emerged. Biomedical research was one of the first “users” of these “tools” (e.g. [5]) and, up to now, it remains among their most interested customers. Currently, there are numerous biomedical research oriented CDIP and CDIA programs available both commercially (e.g. MCID™ Core and MCID™ Analysis, http://www.mcid.co.uk/) and free via Internet [6]. There is software tailored for specific applications (e.g. Image Analyst MKII, Image Analyst Software, Novato, CA, https://www.imageanalyst.net/; CMEIAS, http://cme.msu.edu/cmeias/intro.shtml, [7]). Also, many research instruments such as densitometers for scanning electrophoresis gels (e.g. a GS800 Calibrated Densitometer, BioRad, UK, http://www.bio-rad.com/ru-ru/product/gs-800-calibrated-densitometer/), optical microscopes (e.g. Nikon, http://www.nikon.com/; Olympus, http://www.olympus-ims.com/; Leica, http://www.leica-microsystems.com/), and computer colony counters (see below) are equipped with the instrument-adapted CDIP and CDIA.
Optical images of the microbiological objects recorded in the form of the digital photographs can be quantitatively processed by using CDIP and CDIA. The objective of this paper is to review recent applications of this approach in the microbiological research and practice. The reviews of the studies in this field before 1998 can be found in the comprehensive book [8] and the paper [9]. In the following text, for convenience, the optical images of the microbiological objects are divided into two categories, macroscopic, which can be observed with the naked eye, and microscopic, which can be seen only with a microscope.
2. Macroscopic Objects
2.1. Colony Counting
Quantitative assessment of the live microbial cell concentration/content in various specimens by their growth in Petri dishes up to the visible colonies is one of the most popular assays in microbiology. However, visual/manual colony counting is very tedious and subjective. Hence, interest in automating the colony counting procedure arose long ago. The first apparatus for automated colony counting was described in 1957. It worked by scanning the Petri dish image on a CRT screen by a photomultiplier and processing of the digitized image by a computer [10]. This device has not been widely distributed and used. In the 1970s, colony counters based on the CDIA were designed. The principal parts of this type of colony counters are trans- and epi-illumination system, a CCD camera to capture digital images of the Petri dishes and a computer with an appropriate software (schematically presented in [11]). Several CDIA based computer colony counters (CCC) are presently available on the market (reviewed in [12]) including CCC specially designed for counting mammalian cell colonies (GelCount™, Oxford Optronix Ltd., UK, http://www.oxford-optronix.com/). Also, a number of “home- made” CDIA based systems for automated colony counting have been reported from several laboratories [13]-[18].
CCC can count colonies in the digital images both automatically and manually (visually). In the automatic mode of operation, they can count colonies independently of their number with the rate of less than 1 s per plate. However, strictly speaking, they enumerate the number of elements with a definite set of properties, rather than the number of colonies. Therefore, the choice of criteria to distinguish the elements for counting (segmentation algorithms) is vital to obtain adequate results. Most frequently, the optical density and/or the size of elements (colonies) are used as such criteria for Petri dish examination in transmitted light. These criteria may also include the color and/or shape of colonies or their various combinations. In many CCC, some typical criteria, for example, colony optical density and size, are introduced by default. However, it is quite evident that, without “tuning” of the equipment in each particular case, there is a risk of inadequate data acquisition.
The major problem of using the automatic mode is the presence of confluent colonies. Although many commercial [12] and “home-made” [19] [20] CCC provide an option to discriminate between individual colonies within a conglomerate, this is actually realized only in case of rather small conglomerates. Anyhow, automatic colony counting of the samples with the confluent colonies is prone to an error. Manual (visual) colony counting in digital images by marking the colonies with virtual (computer) markers largely resolves the confluence problem and is more accurate than automatic counting. The principles, advantages and problems of computer colony counting in the automatic and visual detection modes are described in detail in [12] [21] [22].
One more point should be stressed. When interpreting the results of colony count (colony forming units, CFU) in Petri dishes, it should be kept in mind that the CFU do not necessarily directly correspond to the number of seeded viable cells and their concentration/content in the specimens. This may be true only for microbial species which grow as individual cells and do not form any aggregates in natural conditions. However, even in such cultures, partial aggregation may be induced by certain laboratory manipulations, for instance, during the sample preparation. In any case, to reveal the aggregates, the samples are to be analyzed by microscopy before inoculation into Petri dishes. Principal problems of viable cell content determination by colony forming ability arise with the specimens containing microorganisms growing as inseparable conglomerates, cell consortia of different species, and populations of a single species containing damaged or so called viable but nonculturable forms [23].
2.2. Identification
External appearance of the colonies is one of the essential phenotypic characteristics in microbial identification. The major visual parameters of a colony used for identification are coloring, texture (mucous, dry, pastelike, loose, dense, or other), shape (convex, flat, conical, or other), surface characteristics (smooth, wrinkled, rugulose, or other), and edge type (smooth, villous, or other) [24]. The differences in colony appearance may be distinctive features for differentiating microorganisms at the level of various taxonomic groups, including strains of a single species. Importantly, all these characteristics depend upon cultivation conditions, primarily, on medium composition, temperature, and growth time [25]. Although colony appearance cannot be the sole parameter used for identification, under standardized culturing conditions, it may be used as a presumptive indicator in a number of practical applications, when culture express diagnostics is required. Screening for a culture with certain characteristics, checking purity of isolated cultures, disease diagnostics, sanitary and epidemiological expertise are the situations of this kind. Computer image analysis may be helpful in such cases. Examples of application of this approach are presented below.
One obvious case to start with is a description of a colony color by words used in many microbiological papers on identification. It is always subjective and often rather vague. For instance, among the distinctive features of the eight isolated strains of the fungus Metarhizium presented in [26] was their colony color described as “Lumiere Green”, “Pale Olivine”, “Dark Yellowish Green” etc. Using CDIA, the color description can be made quantitative and thereby objective. One simple way to do it, for instance, is by “RGB profiling” procedure of the ImageJ software (available free from https://imagej.nih.gov/ij/). RGB profile across the colony centre provides an objective quantitative characterization of color, thickness and diameter of the microbial colony [27].
A general platform for quantitative colony morphology presentation and analysis by CDIA have been developed using 16 yeast strains as model organisms [28]. 8 features space has been used for visualizing changes in colony morphology and for supervised classification of colony phenotypes. The developed software was suggested to enable the automated analysis of colony types even at scales not possible using manual scoring (i.e., extremely large numbers of images). Also, a web application has been built for easy and rapid sharing of results. This integrative environment for data exploration can be extended to other large-scale image analysis projects and to other colony forming microorganisms.
Numerous studies have been conducted aiming at automated microbial identification and quantification using CDIA of colony images obtained by different imaging techniques. Two CDIA systems using color imaging and the Luv color space were proposed to support diagnostics in urinary tract infections (UTI). The first one has been designed as a modular processing chain specialized to the detection and identification of the colonies of the main UTI pathogens growing on a diagnostic chromogenic medium CHROMagarTM [29]. The main modules were denoising, segmentation and colony identification via segments filtering, isolated colony detection and colony classification. Support Vector Machines (SVMs) with Radial Basis Function kernels [30] were adopted for classifying colonies. The system has been tested on the most clinically relevant bacterial species: Enterococcus faecalis, Escherichia coli, Klebsiella, Proteus mirabilis, Staphylococcus aureus, Streptococcus agalactiae. In conclusion, it was stated that the system offers satisfactory classification performances although some overfitting was observed which could be addressed in prospect by collecting more samples. The second system called Automatic Infection Detector (AID) was developed and tested on the same bacteria plus several species of Enterobacter, Serratia, Pseudomonas, Proteus, and fungus Candida grown on a chromogenic medium UriSelect 4 [31]. The pipeline of the AID comprised of a suitable pre-processing phase, involving spatial clustering, for isolating colonies from the culture medium, and classification of the detected colonies based on both artificial neural networks and SVMs. Finally, besides the infection identification and classification, the AID system also performed the bacterial count, giving an estimate of the number of microorganisms per milliliter of urine.
An automated method has been developed for direct identification of the fungal species of the genus Penicillium by means of the CDIA of their colonies [32]. The method is based on the extraction of the colony texture information from the digital color images. To this end, 27 features quantitatively assessed in the RGB color space were used. The texture measurements of colonies of the nine species were analyzed by supervised and unsupervised data classifiers. A clustering of the data into the correct species was confirmed. The obtained CDIA species classifier was tested on 151 colonies incubated on yeast extract sucrose agar. This resulted in a correct classification rate of 100% when used on the training set and 96% using cross-validation. The same methods applied to 194 colonies incubated on Czapek yeast extract agar resulted in a correct classification rate of 98% on the training set and 71% using cross-validation.
The same approach was applied for clone identification of Penicillium commune isolates [33]. A total of 77 P. commune isolates were classified into groups containing the same genotype determined by DNA fingerprinting and different colony colors. The CDIA data were used for cluster analysis. The Jeffreys-Matusitas distance between the feature distributions was adopted to express the similarity between regions in two colonies, and to evaluate the overall similarity. The nearest neighbor classification rule was used. On a dataset from 137 isolates, a “leave-one-out” cross-validation identification rate of approximately 93-98% compared with the result of DNA fingerprinting was obtained.
Multispectral imaging combined with CDIA as a means of objective identification of the species of the genus Penicillium was demonstrated [34]. This technique was used to address the problem of choosing the subset of growth media for identification. Multispectral analysis was expected to provide additional information about the chemistry of the fungal colonies. In this study 18 spectral bands were used: 10 in the visual region and 8 in the near infrared region. Cultures were grown on three media [Czapek yeast extract agar (CYA), oatmeal agar (OAT), and yeast extract sucrose agar (YES)]. Statistical tests indicated that YES combined with CYA is the best choice of media in this case. However, for the objective identification one medium was shown to be sufficient to discriminate between the species. Statistical tests proved that there were significant differences between the species on all individual media, and that these differences were the largest on YES. The species have been classified using only 3-4 spectral bands with a 100% correct classification rate using both “leave-one-out” cross-validation and test set validation.
Hyperspectral imaging, a comparatively new tool in biomedical research [35] [36], was used as a primary source of information for the identification of several microbial species by the CDIA of their colonies. Although this imaging requires rather sophisticated and expensive equipment, it provides better spectral discrimination of the colored colonies in comparison with the color and multispectral imaging.
Some general principles of using hyperspectral imaging of colonies for identifying bacteria were considered by performing experiments on several strains of lactic acid bacteria, enterobacteria and Staphilococcus aureus grown on various media [37]. Two alternative data processing pipelines, so called radiometrically corrected and physics- based vision, were examined. The data on hyperspectral reflectance were treated by minima/maxima/average analysis and Principal Component Analysis (PCA). The results obtained have shown that satisfactory growth media specific bacterial classification could be done using PCA of the hyperspectral reflectance data by both pipelines. It was also shown that the colonies can be counted making use of classical segmentation and classification algorithms.
A new screening technique using hyperspectral imaging of colonies was developed to detect a foodborne pathogenic bacteria Campylobacter in the presence of non-Cam- pylobacter bacterial species [38] [39]. A reflectance spectral library of the colonies of 11 Campylobacter and 6 non-Campylobacter species in the region from 400 to 900 nm was constructed. Colony classification algorithms, including single-band thresholding, band-ratio thresholding and spectral feature fitting were developed. With a band ratio algorithm using two bands at 426 and 458 nm chosen from continuum-removed spectra of the colonies, bacterial identification accuracy achieved 97% - 99%. It was concluded that the developed hyperspectral reflectance imaging protocol is applicable for early detection of Campylobacter and could be used in other pathogen detection studies.
Visible and near-infrared hyperspectral imaging and chemometrics were used to detect and classify non-O157 serogroups (O26, O45, O103, O111, O121 and O145) of shiga toxin-producing Escherichia coli (STEC) grown on chromogenic selective medium [40] [41]. To this end, spectral libraries for each pure culture serogroup colony were built. The prediction model was based on supervised linear classification of factor scores obtained by PCA. Classification was carried out by Linear Discriminant Analysis (LDA) and SVMs. Chemometric preprocessing methods and other operating parameters, such as scatter correction, first derivative, moving average, sample size and number of principal components (PCA), were compared with a classification and regression tree (CART) method, configured as a classification tree and followed by brute-force searching from candidates selected by the CART. Cross-validation (CV), such as hold out and k-fold CV, was used to validate the prediction performance of candidate models. Serogroups O111 and O121 showed consistently over 99% classification accuracy regardless of the classification algorithms. However, the classification accuracies of serogroups O26, O45, O103 and O145 showed varying results from 84% up to 100%, depending on which preprocessing treatment and prediction model were adopted. In the later study [42], the 428 nm band was determined as the optimal wavelength for non- O157 STEC colony segmentation for automated target colony counting. The accuracy of the developed colony segmentation and counting algorithm was over 99 %. The average of the colony classification algorithm using automated colony segments was 92.5 %. The authors also noted that further studies are needed for automating quantitative assessment of the target bacteria in the presence of background microflora.
Development of a visible and near-infrared hyperspectral imaging technique for automated screening of the two foodborne pathogens Salmonella enteritidis (SE) and Salmonella typhimurium (ST) was described [43]. SE and ST were grown on brilliant green sulfa (BGS) and/or xylose lysine tergitol 4 (XLT4) agar plates in the presence of Acinetobacter baumannii, Enterobacter cloacae, Pseudomonas putida, Citrobacter koseri, Staphylococcus aureus, Aeromonas salmonicida, Klebsiella oxytoca, and Escherichia coli. Five different machine-learning algorithms, including Mahalanobis distance, k-nearest neighbor, LDA, Quadratic Discriminant Analysis (QDA), and SVMs in addition to PCA, were applied for Salmonella detection and classification. When trained on the data from pure cultures of Salmonella and known background microflora, the classification accuracy of each classification algorithm in detecting Salmonella on BGS agar was about 98% on average, although it was difficult to differentiate between SE and ST. The classification accuracy in detecting Salmonella colonies on XLT4 agar was about 88% on average while the detection accuracy for ST colonies were over 99%. The validation of the classification algorithms with independent test samples of chicken carcass rinses spiked with SE and ST showed that the best performance was achieved by QDA with the prediction accuracy of about 99% (Kappa coefficient = 0.97).
The potential of hyperspectral imaging in the region of 400 - 900 nm of colonies for identifying microbial cultures of 19 bacterial strains of 10 genera was evaluated [44]. Microbial colonies were subcategorized in 6 chromogenic classes after growth on a chromogenic culture medium (chromID® CPS Elite, bioMérieux, France). Hyperspectral imaging was used in a linescan configuration. Interclass classification accuracies of 100% were achieved by applying algorithms relying on Linear Spectral Unmixing, and using Diffuse Reflectance Spectra as input data. In order to simplify the technique, the performance of using only the most discriminant 14 spectral channels (a model for a multispectral approach) or 3 channels (a model of an RGB image) was evaluated. The overall classification performance remained unchanged only for multispectral model. With the same intention to simplify and make the screening (classifying) of non-O157 STEC serogroups more cost-effective, a spectral reconstruction technique for predicting hyperspectral images from RGB color images was developed [45]. In this work, the already developed hyperspectral image classification algorithm(s) [38] [39] [40] were used. Reconstruction of the hyperspectral images from the RGB color images was made by polynomial multivariate least-squares regression analysis. The accuracy of the hyperspectral image classification algorithm based on k-nearest neighbors algorithm of PCA was validated to be 99% with the original hyperspectral images and 92% with the test set. It was suggested that color-based hyperspectral imaging would be feasible with prediction accuracy compared to true hyperspectral imaging.
In this context, by the way, it is worth mentioning an application of near infrared hyperspectral imaging for indirect assessment of spoilage bacteria loads in meat [46]. It was established that bacterial spoilage of meat leads to the chemical changes which can be detected by infrared spectroscopy. Relevant changes in the spectra can be used for correlating with actual bacterial loads using chemometric methods. Hyperspectral reflectance images in the region of 910 - 1700 nm of raw chicken breast fillets were acquired and were transformed into hypercubes in absorbance and Kubelka?Munck (K?M) units. Full wavelength partial least regression models were established to correlate the three spectral profiles with measured bacterial counts, and the best calibration model was based on absorbance spectra, where the correlation coefficients (R) were 0.97 and 0.93, and the root mean squared errors (RMSEs) were 0.37 and 0.57 log10 colony forming units (CFU) per gram for calibration and cross validation, respectively. To simplify the models, several wavelengths were selected by stepwise regression. More robustness was found in the resulting simplified models and the model based on K?M spectra was found to be excellent with an indicative high ratio of performance to deviation value of 3.02. The correlation coefficients and RMSEs for this model were 0.96 and 0.40 log10 CFU per gram as well as 0.94 and 0.50 log10 CFU per gram for calibration and cross validation, respectively. Visualization maps produced by applying the developed models to the images could be an alternative to test the adaptability of a calibration model. Moreover, multi-spectral imaging systems were suggested to be developed for online applications.
Colony diffraction pattern imaging with subsequent CDIA was studied as a means of bacterial identification [47] [48]. For this purpose, an optical system with converging spherical wave illumination of colonies for Fresnel patterns recording as digital images was devised. Colonies of Salmonella enteritidis, Salmonella typhimurium, Staphylococcus aureus, Staphylococcus intermedius, Escherichia coli, Proteus mirabilis, Pseudomonas aeruginosa and Citrobacter freundii were analyzed. CDIA comprised image processing by ImageJ software and subsequent data treatment by three classification algorithms, LDA, QDA and SVMs, with classifier performance assessment by cross-validation. Single factor one way ANOVA and Fisher divergence were used to find the most discriminative one of 20 chosen features. The study have shown that the proposed method had very high identification accuracy of over 98% [47], and after optimization [48] it was demonstrated to have high sensitivity and specificity with very small identification error of 1.34%.
An automated CDIA-based system BARDOT (the abbreviation from Bacterial Rapid Detection using Optical scattering Technology) for bacteria identification was designed [49] [50]. It captures 635-nm laser beam forward-scatter unique digital images (scattergrams) of individual bacterial colonies. For further analysis, a number of features are extracted from the digital scatter patterns. Rotation-invariant features are characterized using magnitudes of Zernike moments. Texture features are calculated using Haralick gray-level co-occurrence matrices. Bacteria identification is performed using classification by SVMs algorithm and a scatter classification signature library. The technique has been demonstrated to differentiate Escherichia, Listeria, Salmonella, Staphylococcus, and Vibrio at the genus level with 90 to 99% accuracy [49]. It was also successful in differentiating Listeria species [51]. An improved pattern-analysis and image-processing algorithm was used to extract features from the scatter images to identify and differentiate various Vibrio species, especially, V. cholerae, V. parahaemolyticus and V. vulnificus from other vibrios [52]. It was shown that BARDOT can detect most of the Salmonella serovars tested in the set of the 20 most prevalent serovars [positive predictive value (PPV) of classification precision level reaching 86%] and all serovars in the top 8 group (PPVs ranging from 68% to 93%). Furthermore, it can detect Salmonella in food samples in the presence of background microflora [50].
Fractal and multifractal geometries are useful tools for quantitative morphology description in the 1D, 2D or 3D images. They found numerous applications in biomedical research dealing with pattern recognition, texture analysis and segmentation [53]. CDIA was used for quantitative description of colony morphology images of the fungus Metarrhizium anisopliae by multifractal analysis [54]. It was done for steroid biotransformation activity strain screening by colony recognition and classification. Three features of 14 were selected on the multifractal spectrum of each morphological image in two-dimensional feature vector spaces. A feature augmented vector was generated and used for a classifier design. A statistical least mean square error algorithm was applied to design a piecewise-linear classifier. Representative colony samples were used as training and test sample sets, which were previously classified into a high biotransformation-activity class and another class by a trained person. After passing the training phase, the piecewise-linear classifier could be used for automated classification of unknown colony samples. The developed method provided fast classification and identification of the colonies of individual strains having different steroid biotransformation activity. A correct recognition rate of 96% was achieved.
2.3. Physiology, Biochemistry and Molecular Biology
Parameters of the colony growth under different conditions on/within the solid nutrient media hold valuable information on physiology, biochemistry and molecular biology of microbial cells. However, manual extraction and analysis of these parameters is practically impossible mostly because of their huge amount. CDIA is the tool of choice in this area of research.
An example of one of the first applications of this approach was the study with the goal to understand the role of mammalian hormones as growth promoting signals in the opportunistic yeast Candida albicans development [55]. In this work, the size of colonies growing on agar media supplemented with beta estradiol was compared to those growing without the estrogen supplement. Colony area measurements were conducted by the original “Image tool” CDIA software (University of Texas Health Science Center at San Antonio) using series of digital video images. It was found that growth of the yeast strains under study were not uniformly stimulated by estradiol. Growth of one estrogen-responsive strain was evaluated in a chemically defined medium in the presence of 17-alpha and 17-beta isomers of estradiol. The beta isomer promoted more rapid growth of the test organism and resulted in greater biomass production than the alpha isomer. It was suggested that sterols could be involved in the regulation of fungal metabolism, at least in some strains, and played a role in the virulence development.
The lag phase, characterizing the physiological transition of bacteria before the exponential growth, is an essential parameter of the bacterial population development. Its assessment is of importance, for example, in case of investigating bacterial contaminations occurring in foods. The lag distributions of Listeria monocytogenes cells subjected to situations reproducing conditions encountered during the contamination of cheese, were studied by automated CDIA of the macroscopically visible bacterial colony growth [56]. The kinetics of the bacterial colony growth on agar was measured with an automatic image acquisition system that included a WASP2 spiral plater, a circular motorized platform and a digital camera allowing the simultaneous study of four plates. Image processing and analysis were conducted using Matlab. The results obtained on colonies were compared with lag distributions assessed by the conventional turbidity measurements in broth. An original method to retrieve lag in broth and agar without any knowledge of the growth rate was also proposed. Means and standard deviations of lag distributions for the two different stresses were found to be similar in broth and on agar.
A high throughput Phenotypic Array Analysis (PAA) system was developed for global, quantitative analysis of gene interactions using large microbial mutant collections [57] [58]. The system allowed the colony early phase kinetic growth rates measurements which could further be used for quantifying cell proliferation phenotypes. The whole approach is based on time-lapse imaging of agar spotted cell arrays and their CDIA by specially designed YeastXtract software. The PAA system was experimentally tested on yeast cells of Saccharomyces cerevisiae. The accuracy and precision for image analysis of agar culture arrays was comparable to optical density measurements of liquid cultures. It was shown that cell proliferation could be measured over about seven generations, including four to five generations of relatively constant exponential phase growth. A growth model, based on the logistic function, increased precision and accuracy of maximum specific rate measurements, compared to empirical methods. The logistic function model was also more robust against data sparseness, meaning that less data was required to obtain accurate, precise, quantitative growth phenotypes.
Some of yeast functional genomics approaches are based on growth differences under different conditions. A computerized image analysis system called Growth Detector (GD) have been developed to automatically acquire quantitative and comparative information for yeast colony growth to study the biology of a cell on a model organism, Saccharomyces cerevisiae [59]. The set of non-essential gene deletion strains has been used. The array of the strains was replicated by pinning method using a 384-floating pin replicator. Images were captured with a Hewlett-Packard PhotoSmart 735 camera. The GD software was written in Matlab. It provided image processing and some post-processing calculations. Using GD, a genetic linkage between the molecular activity of the plant-derived antifungal compound berberine and gene expression components was detected. A novel association for the yeast mek1 gene with DNA damage repair was also identified by GD and confirmed by a plasmid repair assay.
A collection of image analysis algorithms, called Colonyzer, for automatic quantification of the size, granularity, color and location of microbial colonies grown on solid agar was developed using the open source packages: Python, RPy and the Python Imaging Library [60]. It can quantitatively analyze images of the colonies of the cultures growing in any rectangular array format, seeded by either pinning or spotting. Colonyzer's particular strength is its sensitivity in detecting cultures with low density. It is suitable for high-throughput screening by comparing growth rates of distinct microbial cultures on solid agar and may be useful for rapid quantifying genetic interactions. Colonyzer is an open-source free software accessible via Internt (http://research.ncl.ac.uk/colonyzer/), allowing users to assess it, adapt it to particular research requirements and to contribute to its development. Application of this package was demonstrated in Quantitative Fitness Analysis (QFA) [61]. This experimental and computational workflow produces growth rate estimates of microbial cultures spotted and grown on solid agar plates. It can be used for comparing fitnesses of microbial cultures derived from the growth rates for genome-wide genetic interaction or drug screens investigating up to thousands of independent cultures. The main QFA procedures include the inoculation of independent dilute liquid microbial cultures onto solid agar plates which are incubated and regularly photographed. Photographs from each time-point are analyzed by the Colonyzer, producing quantitative cell density estimates, which are used to construct growth curves. Fitnesses of 384 strains of the yeast Saccharomyces cerevisiae were analyzed using QFA with robotic inoculation techniques and it was shown that fitness estimates were comparatively precise and well reproducible.
3. Microscopic Objects
3.1. Microcolonies and Biofilms
Light forward-scattering imaging of microcolonies (diameters ranging from 30 to 300 μm) was used to study the possibility of discriminating different bacterial species at a very early stage of growth (6 h of incubation at 37˚C), on thin layers of agar media using a microscope based instrument [62]. Scatterograms treatment and comparison were done using a projection of each scattering pattern along the 120 first Zernike polynomials. The 120-dimension vectors corresponding each to one given scatterogram were then classified using 3 different learning algorithms (Bayes Network, Continuous Naive Bayes, Sequential Minimal Optimization), so as to compare the confusion matrices and select the best performing approach. A recognition (identification) rate of nearly 80% was achieved of 7 gram-negative bacteria at species level with a database of more than 1000 scatterograms. Also, it was shown that four strains of Escherichia coli and two species of coagulase-negative staphylococci (S. haemolyticus and S. cohnii) could be discriminated with a recognition rate of 82%.
Facing the needs of directed evolution programs and of exploring metagenomes, a new high throughput enzyme screening system was developed [63] [64]. It was based on the optical detection by CCD camera of the microcolony array system with a fluorescent chemo-sensor and automated CDIA providing simultaneous time-resolved monitoring of enzyme activity of up to 7000 single microcolonies. Microcolony chips in the size of a micro-titer plate were made by spotting robots to get microcolonies grown from single cells. Contact of the chip with the fluorescent chemo-sensor produced fluorescent response corresponding to the enzymatic activity of a microcolony which was recorded by the CCD camera. The obtained images were treated by a CDIA software. The method was verified by a model screening using esterase and choline oxidase activities of Escherichia coli with fluoresceine and pO2-sensitive particles as indicators. It was proved to provide reliable enzyme activity measurements within single micro-colonies allowing the discrimination of activity differences in the range of 10% - 20%.
For better understanding the cheese ripening process, a study was undertaken to assess spatial distribution as well as the distance between bacterial microcolonies within the cheese matrix [65]. To this end, Lactococcus lactis strain producing green fluorescent protein (GFP) was inoculated into a model cheese at various inoculation levels. Spatial distribution and microcolony diameters were evaluated using CDIA of the confocal microscopy photographs of fluorescent colonies by R software [66]. It was shown that bacterial colonies were randomly distributed, fitting Poisson’s model. Also, it was found that the lower the inoculation level, the larger the colonies were and the further away from each other. It has been suggested that the distribution and the interfacial area of colonies can have a significant influence on the cheese-ripening process on a microscopic scale (see also a review on this topic [67]).
A CDIA software called “daime” (the abbreviation from digital image analysis in microbial ecology) was developed specifically for microbial ecology studies [68]. This computer program integrated digital image processing, image analysis and 3D visualization features. It was tailored to work with digital images acquired by a confocal laser scanning microscope (CLSM) to applications of fluorescence in situ hybridization (FISH) with rRNA-targeted probes. Together with the program a new method was presented for quantifying spatial localization patterns of microorganisms in complex samples. Its utility was demonstrated for identifying environmental biofilm microorganisms involved in mutualistic interactions. This method consists of an image analysis algorithm implemented in daime and a protocol to preserve the spatial structures of microbial communities during FISH with rRNA-targeted oligonucleotide probes (‘3D-FISH’). In addition, the 3D visualization capabilities of daime were illustrated by reconstructions of a nitrifying biofilm. Using this stereological technique on activated sludge, quantitative evidence was obtained that functionally linked ammonia and nitrite oxidizing bacteria cluster together in their habitat.
In further studies, using daime software (version 2.0) [69], new methods have been designed and applied for sequential-FISH analysis of directionally dependent (anisotropic) multispecies biofilms [70]. Upon multiple populations detecting, an automated tool for vertical-distribution analysis in anisotropic biofilms was applied. Distinct stratification patterns of the ammonia oxidizers Nitrosomonas oligotropha subclusters I and II and the nitrite oxidizer Nitrospira sublineage I in three different types of wastewater biofilms was found. Based on these findings, niche differentiation between the N. oligotropha subclusters was suggested, which could explain their coexistence in the same biofilms. Coaggregation analysis showed that N. oligotropha subcluster II aggregated closer to Nitrospira than did N. oligotropha subcluster I in a pilot plant nitrifying trickling filter (NTF) and a movingbed biofilm reactor, but not in a full-scale NTF, indicating important ecophysiological differences between these phylogenetically closely related subclusters.
3.2. Single Cells
CDIA of single bacterial cell hyperspectral images was studied as a potential method for rapid identification of foodborne pathogens [71]. A microscope equipped with an acousto-optic tunable filter was used to acquire dark field light scattering hyperspectral images in the region between 450 and 800 nm of five serotypes of Salmonella (Kentucky, Enteritidis, Typhimurium, Infantis, and Heidelberg) and five species of Staphylococcus (aureus, haemolyticus, hyicus, simulans, and sciuri) bacterial cells. The acquired images were converted to hyperspectral image format with HSiAnalysis software (Gooch & Housego, Orlando, Fl.). It was found that there were distinct scattering intensity peaks at nine wavelengths for both Salmonella and Staphylococcus cells. For the cells of both genera, the scattering intensity of the cell wall was brighter than that of the cytoplasmic membrane. Using scattering intensity data from five serotypes of Salmonella and five species of Staphylococcus bacterial cells, a classification has been done by the SVMs classification algorithm, and identification accuracy obtained was of 99.9% with a kappa coefficient of 0.9998. It was concluded, however, that the classification models need to be validated with bacterial cultures from more food matrices. Further research is also needed to validate the method with positively identified colonies using confirmatory testing, such as latex agglutination or polymerase chain reaction tests.
To characterize morphological diversity in growing microbial communities revealed by phase-contrast microscopy, the system, called CMEIAS (the abbreviation from Center for Microbial Ecology Image Analysis System) have been developed [7]. It is a free downloadable CDIA interactive system which consists of several custom plug-ins for UTHSCSA ImageTool [72]. By measuring various morphological features of individual cells, CMEIAS classifies them into one of 11 predominant bacterial morphotypes, including cocci, spirals, curved rods, U-shaped rods, regular straight rods, unbranched filaments, ellipsoids, clubs, rods with extended prostheca, rudimentary branched rods, and branched filaments. 1937 phase-contrast grayscale digital images of various diverse microbial communities were used for training and testing the shape classifier. It was shown to have an accuracy of 96.0% on a training set of 1471 cells and 97.0% on a test set of 4270 cells representing all 11 bacterial morphotype classes. A nutrient shift-up perturbation in two continuously fed anaerobic bioreactors with morphologically distinct start communities was presented as an example of CMEIAS application for characterizing microbial population dynamics in ecological systems. To date, three upgraded versions of CMEIAS were published with options of color segmentation [73], quadrate maker [74] and fractal analysis [75].
Morphological segmentation was used in a method developed for determining the time to first division of individual bacterial cells growing on agar media [76]. Digital images of bacterial cells of Escherichia coli, Listeria monocytogenes and Pseudomonas aeruginosa were acquired by phase-contrast microscopy at intervals. CDIA was conducted by Image Pro v.4.5 (Media Cybernetics, Maryland, USA) and data were then analyzed by in-house written Visual Basic programs. The time to first division was estimated by calculating the area of the smallest rectangle that can be drawn around an object, divided by the area of the object itself. This parameter was found to increase suddenly during growth at a time that correlated with cell division as estimated by visual inspection of the digital images. This method was used successfully to generate lag time distributions for populations of the gram-negative rods, but did not work with the coccoid organism Staphylococcus aureus. This method provides an objective measure of the time to first cell division, whilst automation of the data processing allows a large number of cells to be examined per experiment.
Two single cell morphometry CDIA algorithms were designed for budding yeast studies by light microscopy imaging. The first was used in a robotic method to automatically obtain quantitative morphology features of Saccharomyces cerevisiae yeast cells of four deletant strains: YLR371w, YDR349c, YLR192c, and YDR414c [77]. It was demonstrated that the method provided an efficient means of getting statistically significant morphological differences between strains, and that these differences varied with growth stage. The second algorithm was designed for identification of cell cycle phases based on cell bud size [78]. It automatically extracted from the light microscopy images the cell geometrical features including compactness, axis ratio, and bud size. The features were then used for classification by the linear SVMs, distance-based classification, and k-nearest-neighbor algorithm. The algorithm was shown to be effective in automatic classification of the cells at different stages of the cell cycle after it was provided with labeled training data.
Multicolor staining, in particular, combined with fluorescence microscopy observation is a very popular approach in microbiology. A number of studies were conducted for developing CDIA based methods to make this procedure more objective and less labor intensive. Color segmentation was the main principle in all these methods.
Using programming language C, a CDIA system named BACS capable of the automatic enumeration of actively respiring bacteria in river water was developed [79]. The epifluorescence microscopy images were acquired after double-staining of bacteria by 4'-6'-diamidino-2-phenylindole (DAPI) (all cells fluoresce blue) and 5-cyano-2,3-ditolyl tetrazolium chloride (CTC) (respiring cells fluoresce orange). The automatic enumeration was conducted after color segmentation. The regression coefficients of DAPI and CTC-stained bacteria in river water between microscopic visual counts and digital image analysis exceeded 0.96 and 0.93, respectively. An improved version of BACS was used to simultaneously identify Escherichia coli O157:H7 in milk samples and assess their respiratory activity [80]. In this case, bacterial cells were triple stained by CTC, DAPI and FITC-labelled fluorescent antibody (green-yellow fluorescence). A good correlation was found between the counts of actively respiring (r = 0.93) and total (r = 0.94) Escherichia coli O157:H7 measured by CDIA and visual observation.
A vital double-staining of the yeast Saccharomyces cerevisiae cells by ethidium bromide (EB) and DAPI was used to assess a fraction of damaged cells in the population. EB-stainable damaged cells fluorescing red and DAPI-stainable all cells fluorescing blue could be revealed simultaneously in the same sample by fluorescence microscopy. A CDIA procedure based on the color segmentation was developed for the automatic determination of the relative number of damaged cells using ImageJ software [81]. A good correlation has been found between the viability rates determined by the plate count method and the relative numbers of intact cells assessed by the developed procedure in the dry preparation of a commercial ethanol-producing yeast strain after rehydration under various conditions [82]. Similar approach with ImageJ color segmentation was applied in a viability assay for unicellular cyanobacteria, in which red chlorophyll fluorescence and an unspecific green autofluorescence was used for the differentiation of viable and non-viable cells [83]. Both autofluorescence signals could be observed simultaneously allowing a direct classification of viable and non-viable cells. The assay was validated for the model organism Synechocystis sp. PCC 6803 by plating/colony count, absorption spectra and chlorophyll measurements.
Color segmentation option of the CMEIAS (see above) was used to quantify the in situ spatial scale of N-acylhomoserine lactone (AHL)-mediated cell-to-cell communication of Pseudomonas putida colonized on tomato and wheat root surfaces [84] (more on the microbial cell-to-cell communication see the review [85]). In this study, spatial distribution of two strains of Pseudomonas putida was investigated by confocal microscopy. The first strain served as an “AHL-source” strain. It constitutively expressed red fluorescent protein. The second strain was an “AHL-sensor” containing AHL-in- ducible reporter plasmid with a green fluorescent protein-encoding sensor cassette. The results of the work indicated that the effective “calling distance” on root surfaces was very long-range in proportion to the size of individual bacteria. It was concluded that AHL-mediated cell-to-cell communication occurs not only within dense populations (so called “quorum sensing”), but also in very small groups and over long ranges between individual bacteria. The authors proposed that this cell-to-cell communication is governed more by the in situ spatial proximity of cells within AHL-gradients than the requirement for a quorum group of high population density.
CDIA of fluorescent microscopic images of triple-stained Saccharomyces cerevisiae yeast cells was shown to make it possible getting and treating single cell morphological features as quantitative data [86]. For this purpose, mannoprotein (as a cell wall component marker), the actin cytoskeleton, and nuclear DNA were specifically stained simultaneously by rhodamine?phalloidin, fluorescein isothiocyanate?conjugated concanavalin and DAPI, respectively. The developed CDIA was employed for high-dimen- sional and quantitative phenotyping of yeast mutants [87] presented in the Saccharomyces cerevisiae Morphological Database (SCMD) [88]. This data-mining approach revealed that deletion of nearly half of the yeast genes not essential for growth affects some morphological traits. Similar morphological phenotypes may be caused by deletions of functionally related genes, enabling a functional assignment of a locus to a specific cellular pathway. The approach was upgraded for investigating fission yeast [89] and, in addition to the originally developed 501 parameters for cell wall morphology, nuclear DNA, and actin, 610 parameters for the morphology of some other subcellular components, were proposed [90]. Also, it was demonstrated to be useful for studies of antifungal drugs [91] [92].
Computer image processing algorithms of the ImageJ software have been tailored for subcellular microfluorimetry of the single live (not fixed) Saccharomyces cerevisiae yeast cells. The algorithms were used for quantitative processing of the color digital fluorescence microscopy images aiming to evaluate intravacuolar viscosity and to investigate the intracellular distribution of anthracycline anticancer drug doxorubicin [93].
Movement of insoluble polyphosphate complexes (IPCs) stained by DAPI in the vacuoles of the cells was studied [94]. As the first step, this movement was demonstrated to be Brownian motion. By fluorescence measurements, on fluorescein isothiocyanate-labelled latex microspheres, a methodology was developed for measuring a fluorescing particle’s two-dimensional (2D) displacements and its size. Using this methodology, in four yeast cells, the 2D displacements and sizes of the IPCs were evaluated. Apparent viscosity values in the vacuoles of the cells, computed by the Einstein- Smoluchowski equation using the obtained data, were found to be 2.16 ± 0.60, 2.52 ± 0.63, 3.32 ± 0.9 and 11.3 ± 1.7 cP. (More on intracellular viscosity see the review [95]).
Since the method presented above was too time and labor-consuming for intravacuolar viscosity studies on population level, one more method for intravacuolar viscosity assessment in yeast cells was developed [96]. By visual observations, fluorescent dye quinacrine was shown to be specifically accumulated within the vacuoles of the cells. There was no detectable binding of the dye within the vacuoles as it freely and completely flowed out of the vacuoles upon ATP depletion of the cells. The fluorescence anisotropy of quinacrine was measured by CDIA microfluorimetry in the vacuoles of 39 cells using images acquired in a fluorescence microscope equipped with polarizers. From cell to cell, this parameter changed in the range 0.032 - 0.086. Using the Perrin plot as a calibration curve, apparent viscosity values of the vacuolar milieu were calculated for each cell. The population of the cells studied was heterogeneous with regard to vacuolar viscosity, which was in the range 3.5 ± 0.4 - 14.06 ± 0.64 cP. There was a characteristic distribution of the frequencies of cells with apparent viscosities within certain limits, and cells with viscosity values in the range 5 - 6 cP were the most frequent.
A study was undertaken to test if intact yeast cells of S. cerevisiae can be used as a model for locating intracellular sites/targets of the DNA-directed drugs. With this goal in mind, intracellular distribution of anthracycline anticancer drug doxorubicin (DR), as an example, was investigated along with fluorescent DNA markers DAPI and ethidium (E) [97] [98]. Application of DAPI in combination with DR or E visualized intracellular location of the nuclei and the mitochondria. Red, green and blue components of the fluorescence intensity were quantitatively assessed by the ImageJ “Analyze” plugin in the selected subcellular regions of interest. The obtained data were called “pseudospectra” as they roughly corresponded to the real spectra of the dyes. Using analysis of these data, called “pseudospectral analysis”, it was established that all three dyes were located in the nuclei and in the mitochondria. In contrast to DAPI, which interacts only with DNA, an appreciable fraction of DR and E probably associated with the mitochondrial membranes. When added combined, DAPI competed with DR and E for binding sites on DNA. It was concluded that this approach may be applied to designing new DNA-targeted drugs at the stage of preliminary assessment of their interaction with eukaryotic cells with the yeast Saccharomyces cerevisiae as a model.
Green fluorescent protein (GFP) tagging technology combined with fluorescence microscopy imaging is a method to study intracellular location/function of proteins in living cells including microbial ones. In this method, CDIA is a platform for extracting quantitative information from the images and to automate data acquisition and treatment (on general aspects of this issue see the review [99]). By this method, the subcellular localization and movement of three proteins, GFP-tagged Lac Repressor protein (LacR) inserted into a gene cluster, named Chr1, mCherry-labeled Cut11 protein of the nuclear membrane (NM) and the Spindle Pole body (SPB) compound Sid4 fused to Red Fluorescent Protein (Sid4-mRFP) in fission yeast Schizosaccharomyces pombe was studied [100]. The distances between the proteins were assessed on the fluorescence microscopy images of the cells by the measurement tools of Zeiss Zen Lite and ImageJ programs (both are free Internet-accessible) before and after depletion of the nitrogen source. It was shown, in particular, that there was a statistically significant shift in the localization of the gene cluster Chr1 moving away from the NM towards the SBP.
One more example of this methodology implementation is the development of computational methods to automatically analyze the images created by the UCSF yeast GFP fusion localization project [101] [102]. The system was trained to recognize the same location categories that were obtained visually. Testing the system have shown that when the highest confidence assignments were considered, 94.7% agreement was observed. It was concluded that the automated method provides an objective, quantitative and repeatable assignment of protein locations that can be applied to new collections of yeast images (e.g. for different strains or the same strain under different conditions).
4. Conclusion
To summarize briefly the presented survey of literature published for the last 15 years, it can be concluded that the use of CDIA of optical images in microbiology is a promising way to a higher level of many traditional microbiological techniques by making them more objective and suitable for statistical analysis and automation. Also, this methodology made a remarkable improvement of the basic research toolbox, especially for studies of single microbial cells. Nevertheless, it is just a tool, and the success of its use in future will entirely depend on how skillfully it will be implemented for the investigation and the exploration of the microbial world. Anyway, it should be kept in mind that CDIA, as any advanced tool, does not substitute for a qualified researcher, but assists him!