Empirical Noise Model Calibration

Owned by Former user (Deleted)
Last updated: Jan 20, 2017 by John Medalen
10 min read

Many algorithms in the kernel use data from the noise model of the sensor for proper functioning. Sharpen, Filter, Flatten Bayer(SFFB), edge sensitive noise removal algorithm, for example, use the information in the noise model to determine whether a given pixel will be flattened or sharpened. The noise model calculates the noise versus pixel value for the minimum analog gain and the maximum analog gain. A sample plot of the SOC3140 noise model is shown below.

Setup and Test Conditions

  • Studio lights controlled by a dimmer
  • DevWare.exe
  • Gretag Macbeth 24 patch color checker that fills approximately 50% of the scene.(Be sure there are at least 32X32 pixels for each channel per patch for best statistical data)
  • Lens Shading Correction

Capture Images for the Noise Profile

Images at the sensor minimum and maximum recommended analog gain must be captured in order to determine the noise profile curves for the two conditions. At each gain setting, two sets of 10 images will be taken. This is done to ensure that enough data points are present to obtain a good noise model. It is important that the only thing that changes between the two sets of images at each gain is the incident light and the only thing that changes between the two gain settings is the analog gain. Please follow the instructions carefully. ALL DIGITAL GAINS MUST BE SET TO UNITY.

Capture images at the minimum analog gain setting

  1. Adjust the lighting such that the green analog gain is at the minimum analog gain value. Must be in preview mode (context A)
    CAM1 Var. 0x71 cam1_ctl_min_analog_gain = 64
CAM1 Var. 0x69 cam1_ctl_analog_green_gain = 64
  2. Switch to full resolution mode.
  3. Use DevWare Sensor Control Panel to put the system in full resolution bypass mode with Lens Correction.
  4. Put the firmware in Test Mode.
    MON Var. 0x02 mon_mode [0] mon_mode_test = 1
  5. Force all the analog gains to the minimum gain and the digital gains to unity
    CAM1 Var. 0x71 cam1_ctl_min_analog_gain = 508
CAM1 Var. 0x69 cam1_ctl_analog_green_gain = 508
CAM1 Var. 0x6B cam1_ctl_analog_red_gain = 508
CAM1 Var. 0x6D cam1_ctl_analog_blue_gain = 508
SOC1 Reg. 0x32D4  digital_gain_1_red = 128
SOC1 Reg. 0x32D6  digital_gain_1_green1 = 128
SOC1 Reg. 0x32D8  digital_gain_1_green2 = 128
SOC1 Reg. 0x32DA  digital_gain_1_blue = 128
SOC1 Reg. 0x32DC  second_digital_gain = 128
  6. Using DevWare graphing functions, view the histogram of an ROI containing the colorchecker only. Right click on the histogram and uncheck logarithmic scale.
  7. Increase the integration time such that the histogram covers the whole range but does not clip the higher pixel values. This is done to have more data points with higher pixel values. Do not touch the integration time until all pictures have been taken for the current section, Capture images at the minimum analog gain setting.
    CORE Reg. 0x3012 coarse_integration_time

  8. Take 10 images with the histogram evenly spread. Using DevWare Sensor Control, Still Capture interface, change the following parameters.
    • Set Total Frames to 10
    • Select Capture —> Current Displayed Image
    • Use the following naming convention.
      gain_1_02x_       This is case sensitive!!
    • Check the box for Auto File Name Increment and Save.
    • Check the boxes for .bmp, .txt(registers), tab(dec), and .png(raw) files.
    • Press capture button
    • Once the 10th frame has been captured, select Done

  9. Turn down the lighting until the histogram is in the lower 50% of the range. Do not let values drop below zero.
  10. Take 10 images with the histogram in the lower 50%.
     
    • For these images with histogram in the lower 50%, use the following naming convention.
      gain_0_02x_
    • Check the box for auto-naming.
    • Check the boxes for .bmp, .txt(registers), tab(dec), and .png(raw) files.
    • Press capture button
    • Once the 10th frame has been captured, select Done
  11. Disable the firmware Test Mode.
    MON Var. 0x02 mon_mode [0] mon_mode_test = 0
  12. Take part out of bypass mode.
  13. Go to Preview mode.

Capture images at the maximum analog gain setting

  1. Adjust lighting until the analog gain on any of the channels the maximum analog gain (When using A light, the blue channel will peak first; When using D65 light, the red channel will peak first), while the digital gain is still set to unity and the frame rate is 5FPS (or the minimum frame rate that the part will be allowed to go down to in low light).
    CAM1 Var. 0x6F cam1_ctl_max_analog_gain 	= 508
CAM1 Var. 0x69 cam1_ctl_analog_green_gain 	= 508
CAM1 Var. 0x6B cam1_ctl_analog_red_gain 	= 508
CAM1 Var. 0x6D cam1_ctl_analog_blue_gain 	= 508
  2. Switch to full resolution mode.
  3. Use DevWare Sensor Control Panel to put the system in full resolution bypass mode with Lens Correction.
  4. Set the firmware in Test Mode.
    MON Var. 0x02 mon_mode [0] mon_mode_test = 1
  5. Force all the analog gains to the maximum gain and the digital gains to unity
    CAM1 Var. 0x71 cam1_ctl_min_analog_gain   = 508
CAM1 Var. 0x69 cam1_ctl_analog_green_gain = 508
CAM1 Var. 0x6B cam1_ctl_analog_red_gain   = 508
CAM1 Var. 0x6D cam1_ctl_analog_blue_gain  = 508
SOC1 Reg. 0x32D4  digital_gain_1_red 	  = 128
SOC1 Reg. 0x32D6  digital_gain_1_green1   = 128
SOC1 Reg. 0x32D8  digital_gain_1_green2   = 128
SOC1 Reg. 0x32DA  digital_gain_1_blue 	  = 128
SOC1 Reg. 0x32DC  second_digital_gain 	  = 128
  6. Using DevWare graphing functions, view the histogram of an ROI containing the colorchecker only.
  7. Take 10 images with the histogram mostly contained in the lower 50% of the range.
    • For these images in the lower 50%, use the following naming convention.
      gain_0_16x_
    • Check the box for auto-naming.
    • Check the boxes for .bmp, .txt(registers), tab(dec), and .png(raw) files.
    • Press capture button
    • Once the 10th frame has been captured, select Done
  8. Adjust the lighting so the histogram is more evenly spread but not clipping.
  9. Take 10 images with the histogram more evenly spread
    • For these images, use the following naming convention.
      gain_1_16x_
    • Check the box for auto-naming.
    • Check the boxes for .bmp, .txt(registers), tab(dec), and .png(raw) files.
    • Press capture button
    • Once the 10th frame has been captured, select Done
  10. Disable the firmware Test Mode.
    MON Var. 0x02 mon_mode [0] mon_mode_test = 0
  11. Take part out of bypass mode.
  12. Go to Preview mode.

Calculate the Noise Profile

  1. In SensorTune, Follow the instructions in the Initialization wizard for selecting your sensor and tuning type, then click Start.
  2. Click on the Noise Characterization tab
  3. Choose the directory where the txt images are located.
  4. Choose the output directory.
  5. Input the required Data Exchange Parameters.
  6. Select Produce Report
  7. Wait.....this may take a couple of minutes
  8. Follow the instructions on the screen to select the registration points around the Color-Checker, you can move individual region if you have severe lens distortion. (like fish-eye lens).
  9. Wait....This may take a long time (~15 minutes), depending on the size of the dataset.The program will now run unassisted and compile a report.

Output Report Explained

After the Noise Model has been derived, an HTML report will be created in the folder specified by the users input in Calculate the Noise Profile. The following describes the contents of the HTML report. Along with the HTML report, an excel spreadsheet of the extracted data is generated. The extracted data will remove outliers from the dataset therefore pixel defects should not affect the noise curves. More specifically, values that are +/- 2 standard deviations from the mean will be removed from the dataset.

Extract Image Data

At the beginning of this section is an image of the Color-Checker with the mask used to extract the patch data. The boundaries of the mask should be completely inside the patches.

Calculate Offset & Slope from Extracted Image Data

Following the mask image, the calculated linear offset and slope of the noise data are output as well as the corresponding plots. The values are in terms of LSB.

SOC3140 - K46A
 
coeffs_StdDev = 
    'Gain'    'Slope'     'Offset' 
    [   2]    [0.0277]    [16.3808]
    [  16]    [0.0322]    [26.9356]
 
coeffs_Variance = 
    'Gain'    'Slope'     'Offset'  
    [   2]    [2.0100]    [-28.4065]
    [  16]    [3.4062]    [262.5067]

Calculate SFFB parameters

The noise model is used to determine the required start and stop values for the definition of an edge in terms of the SFFB algorithm. The values from the report will be entered into the SFFB parameter variables in the SFFB section of the tuning document.

SFFB_params =
'Gain' 'Ramp' 'Slope'       'MaxThresh'   'LowThresh1'  'LowThresh2''LowThresh3'
[   2] [  86] [   42]       [        8]   [         2]  [         3][         4]
[  16] [  34] [   42]       [        9]   [         2]  [         3][         4]

The green dots represent the raw image data (scaled to 8 bit), the blue dots represent the best fit line, and the red line represents the SFFB HW parameter settings. The SFFB parameter settings are derived by finding the optimal solution that closely matches the blue best fit line. The SFFB algorithm will use this line to determine if the pixel under test is a flat field or if it is an edge.

Calculate GRB parameters

The noise model of the G1 and G2 data is used to determine the required start and stop values for the definition of an edge in terms of the Green Re-Balance (GRB) algorithm. The parameters for GRB will entered in the GRB section of the tuning document.

GRB_params = 
 
    'Gain'    'aPos'    'bPos'    'aNeg'    'bNeg'
    [   2]    [  19]    [  13]    [  18]    [  12]
    [  16]    [  31]    [  21]    [  31]    [  21]

The green dots represent the raw image G1 data (scaled to 10 bit), the cyan dots represent the negative raw image G2 data (scaled to 10 bit), the red line represents the GRB a-pos b-pos FW settings, and the blue line represents the GRB a-neg b-neg FW settings. The SFFB algorithm will use the positive line and the negative line to determine if the pixel under test is an edge. If the pixel under test lies outside of the positive and negative line then the GRB algorithm will assume the pixel is an edge and will not do the green imbalance correction. If the pixel under test lies within the positive and negative line then the GRB algorithm will correct the pixel for green imbalance.

Calculate x40 Demosaic parameters

The report calculates the required start and stop values needed for the x40 architecture Demosaic algorithm.In this architecture, the decision of whether to use the flat field Demosaic algorithm or the edge aware Demosaic algorithm is based on a threshold curve that is programmable (with an offset and slope, named thresh_y and thresh_ygain, respectively). The current noise model code sets the curve 3 sigma away from the noise seen by the Demosaic algorithm.

Since this particular part does not share the x40 architecture, these values will not be used.

Demosaic_params = 
 
    'Gain'    'thresh_y'    'thresh_ygain'    'mix_thresh_gain'
    [   2]    [ 49.1424]    [         255]    [            255]
    [  16]    [ 49.1424]    [         255]    [              0]

Calculate x40 Aperture parameters

The report calculates the required start and stop values needed for the x40 architecture Aperture algorithm. This architecture allows for a variable knee value (one for positive edges and one for negative edges) to be subtracted from the aperture value calculated for a given pixel.>The noise can effectively be canceled while the edges are sharpened by setting the knee value to be three sigma from the noise curve.

Since this particular part does not share the x40 architecture, these values will not be used.

Aperture_params = 
 
    'Gain'    'a_knee'     'knee_delta'    'kp_gain'    'kn_gain'
    [   2]    [49.1424]    [        -5]    [10.6324]    [10.6324]
    [  16]    [80.8067]    [        -5]    [12.3584]    [12.3584]

Calculate x30 Kernel parameters

The offset of the linear noise model of the raw image data (scaled to 10 bit) is used to determine the required start and stop values for the definition of an edge in terms of the Demosaic algorithm, Noise Reduction algorithm, and the Aperture algorithm.

The Demosaic threshold is derived by using the offset value of 2 sigma of the noise model. The parameters for Demosaic will be entered in the Demosaic section of the tuning document.

The Noise Reduction Minimum Maximum threshold is derived by using the offset value of 3 sigma of the noise model. The Noise Reduction Color Channel thresholds are derived by using offset value of 6 sigma of the noise model. The parameters for Noise Reduction will be entered in the Noise Reduction section of the tuning document.

The Aperture threshold is derived by using the offset value of 2 sigma of the noise model. The parameters for GRB will entered in the GRB section of the tuning document.

Aperture_params = 
 
    'Gain'    'a_knee'     'knee_delta'    'kp_gain'    'kn_gain'
    [   2]    [49.1424]    [        -5]    [10.6324]    [10.6324]
    [  16]    [80.8067]    [        -5]    [12.3584]    [12.3584]