Quantitative Measurement of the Exposure Response of Digital Cameras

Digital cameras span a large range in price and performance. Consumers of-ten focus mainly on the resolution in pixels when shopping for a camera. Of equal importance is the quality of the optics and the exposure response. Digital cameras generally have a linear exposure response, but the amount of noise and the dynamic range vary. It is difficult to obtain quantitative information on these parameters to make an informed assessment. This work ex-plores and demonstrates first-principles methods to measure the exposure response to make meaningful comparisons between different camera models. It also shows how to make the most of a particular camera by measuring its noise level and dynamic range, to understand the limits of its useable ISO amplification. The methods only require a computer and free software to download images and extract their RGB pixel values. The analysis, based on the RGB values, uses standard spreadsheet software. The procedures are therefore accessible to anyone with a digital camera and computer, and will help to reduce speculation in comparing cameras, and help consumers make an informed decision.

tio of the focal length to the aperture diameter is called the aperture value A V (also called f-number or f-stop). The relationship between E V and A V is given by: A camera's shutter opens to let in light for a controlled period called the time value T V (also called the exposure time or shutter speed). L V , A V , and T V together control the light energy falling on the sensor per area called the radiant exposure H (also called the luminous exposure or photometric exposure): The radiant exposure should remain constant for different A V and T V values as long as the ratio T A is kept constant. This is called the reciprocity principle. The camera amplifies H by a factor proportional to the ISO setting, and stores it as a binary number, the camera exposure C, for each pixel. Thus we expect: and ISO C ∝ (4) For the 8-bit jpeg images used in this work, C can have values between 0 and (2 8 − 1) = 255. Further background on photographic parameters can be found in [1] [2] [3] and [4].

Digital Imaging
Digital cameras achieve imaging by having a mosaic of pixels, each of which contains a photo circuit that produces a voltage representing the light energy received by that pixel. We will concentrate here on the CMOS (complementary metal oxide semiconductor) type of sensor, which is currently the most common type and has replaced the CCD (charge-coupled device) in most consumer cameras. Figure 1 shows an example of a simplified functional diagram for what happens in a single pixel's photo circuit leading to its camera exposure value being stored in the camera's memory.
Before exposure, the upper switch (which is also a transistor) closes momentarily to charge the gate electrode of a MOSFET (metal oxide semiconductor  effect transistor), which behaves like one plate of a capacitor. The reverse biased photodiode is almost non-conductive when no light shines on it, so that the charge sits on the MOSFET gate until exposure begins. Then, photoelectrons are excited and make the diode conductive causing charge to leak away, reducing the gate voltage. The total charge that leaks away and the lowering of the gate voltage is proportional to the light energy that falls on the pixel. Before the exposure, the gate at full voltage pinches off the MOSFET cutting off the current through it. As the gate voltage V G falls with exposure, the MOSFET begins to conduct and the current through the resistor and its voltage V R increase. Thus V R is proportional to the light energy collected by the pixel. This is now amplified (ISO amplification), digitized, and stored along with the values for the other pixels that make up the image.
For determining color, each pixel has sub-pixels for each of the three primary colors: red (R), green (G), and blue (B), resulting in a set of three camera exposure values C R , C G , and C B for each pixel. Ideally, C H ∝ , but there is an upper limit H max above which C stops increasing. This happens when all the charge stored on the MOSFET's gate ( Figure 1) leaks away. Then V G drops to its minimum value and V R saturates to its maximum value. There is also a minimum value H min below which noise and dark current leakage through the photodiode start to cover up differences in H.

Methods
Two DSLR (digital single lens reflex) cameras were used in these experiments: a Canon Rebel XSi12-mega-pixel consumer level DSLR (with a 55 mm lens) and a Canon 6D20-mega-pixel professional DSLR (with an 85 mm lens). The cameras were mounted on a tripod pointing straight at a white piece of paper illuminated by a uniform source of incandescent lighting. The camera lens was focused at infinity to blur the image and further even out the illuminance over the entire view. At the beginning, an image of the paper was taken and this was set as the custom white balance. For the starting point of some experiments, the meter in the camera was used to determine the "correct" exposure, which appears to be near the middle of the DR with a camera exposure value of C ≈ 255/2.
After downloading the images from each camera as jpeg files, a free software called GetRGB [7] was used to extract the camera exposure values for each of the three colors (RGB) from each pixel. The mean and standard deviation for each color type for all pixels were calculated using a spreadsheet software. The mean Journal of Analytical Sciences, Methods and Instrumentation indicated the average camera exposure and the standard deviation indicated the noise. An example of the first few rows for a sample spreadsheet is shown in Figure 2.
Four experiments were conducted on both cameras as explained below.

Reciprocity
For testing reciprocity, the ISO was set to 100 and the time value was set to 1/8 s, which is in the middle of the 18 stops of T V available on both cameras. Using this time value, a picture was taken at the aperture value that resulted in a correct exposure as indicated by the camera's meter. Keeping the ISO fixed, pictures were taken at different combinations of aperture and time values while keeping the exposure correct as indicated by the camera (which tries to satisfy Equation (2)). The actual camera exposure values obtained from the images were then compared to see if they were in fact constant for all the exposures.

Camera Exposure Dependency on ISO
For testing how the camera exposure depends on ISO, the ISO was set to 100, the aperture value was set to the largest setting (smallest hole) and the time value to 2.5 stops below the correct exposure to allow room for growth and to be able to see a larger range of camera exposures achieved by changes in ISO. Pictures were taken while increasing the ISO from 100 to 1600. The measured camera exposures were then plotted versus ISO to see how linearly the ISO amplifier works (Equation (4)).

Exposure Response Curve
The exposure-response curves were measured at the highest ISO value of 1600 (common to both cameras) where noise becomes more apparent. T V was first set to the middle value of 1/8 s and A V was adjusted so that a correct exposure was indicated by the camera's meter.  (2)) to see the exposure response curve (Equation (3)).

Dark Exposure and Noise
If the same amount of light energy falls on two pixels (i.e. they receive the same radiant exposure H) then the camera should record the same camera exposure C for these pixels. However, the presence of noise will cause C to depend irregularly on H. This noise was measured by taking the standard deviation in C for all pixels, which have received the same H. The noise will be most noticeable if H = 0, i.e. for a dark exposure. These dark exposures were taken in a dark room, with the lens cap on, while keeping A V at its highest setting (A V = 38 for the Rebel and A V = 22 for the 6D), so that no light energy reached the sensor. The standard deviation in C for each color was calculated to determine the level of noise. Figure 3 shows results of the reciprocity test as per the procedure described above. Both cameras have an excellent reciprocity (constant camera exposure for a variety of A V and T V combinations that satisfy

Data and Analysis
. Notice that the white balance is well maintained as well (no separation of R, G, and B data points). In this respect, the professional 6D camera shows no advantage over the Rebel. Figure 4 shows results of the dependence of the camera exposure on ISO.
Here the radiant exposure (energy captured) is kept constant by keeping T V and A V fixed. The ISO setting changes the gain after the sensor. As can be seen, there is an almost linear relationship and the two cameras have similar performances. The tops of the graphs start to saturate as they approach 255, which is the maximum value that C can have. Figure 5 shows the complete exposure response (dependence of the camera   the noise level whose measurement is described below) are used to define the usable dynamic range. The much lower noise level of the 6D gives it a more expanded DR = 9.4 compared to the Rebel's DR = 7.8. Thus the 6D will be particularly superior when shooting in low light because of its low noise level and when shooting scenes that have a large luminance (scene brightness) range. Figure 6 shows the noise plotted against the time value. As explained in the C. R. Kunchur methods section, these are dark exposures so that no light energy fell on the sensor. However, even without light, the photodiode conducts because of thermal energy. This is referred to as the dark current and it leads to a camera exposure that is proportional to the time value. Two other sources of noise are read noise in the sensor and the circuit noise that comes after the ISO amplifier. These will lead to randomness in the C values between different pixels. Quantitatively, the noise was taken to be the standard deviation σ C in camera exposure values for different pixels. Of these three components of noise, the dark current and read noise coming before the ISO amplifier will appear to increase with ISO.
Of those two, only the dark current component will depend sensitively on time value and it will have a smaller contribution at short time values. There can also be a few defective "hot pixels" that have fixed positions and will be part of the pre-ISO noise. A general discussion of noise can be found in [8] and [9]. The plotted σ C was the average of the σ C values for the three colors.
From the graphs, it appears that the Canon 6D has very little sensor read noise (almost no dependence on ISO at short time values) and the noise level is a factor of 5 lower than the Canon Rebel. At ISO = 100, dark current noise starts to increase beyond T V ≈ 100 s for both cameras. Circuit noise after the ISO amplifier (dominant at low ISO and T V ) is about twice as much in the Rebel as in the 6D. Thus the two cameras should perform similarly for pictures taken at low ISO, short T V < 30 s, and for bright scenes. The advantage of the more advanced 6D camera will become apparent for pictures taken at high ISO in low light.

Summary and Conclusion
This work explored and established simple first-principles methods to quantitatively evaluate digital cameras, taking the concrete example of two Canon DSLR cameras. The study measured the entire exposure response from H min to H max , tested reciprocity and ISO gain, and obtained estimates of the noise and dynamic range. Furthermore, it was able to separate the various components of noise: dark current, sensor read noise, and post ISO circuit noise. The procedures re-quire no special equipment and use free or readily available softwares, allowing any consumer to quantitatively compare digital cameras, and find the best combination of settings of A V , T V , and ISO for a particular camera and photographic situation. It was not possible to find comparable information on these or other cameras published in the literature or the internet, including Canon's own https://www.usa.canon.com/ website. This work, therefore, provides new information and an objective approach to evaluate other digital cameras. An interesting future experiment would be to measure the noise at various temperatures.
The dark current increases with temperature; hence a more advanced DSLR might be needed if shooting is mostly done in a warmer climate, especially at high values of ISO and T V .