Overview and Features

The Lens Distortion Correction tool is a DevWare plug-in. It is intended for use with the MT9V126, MT9V128,  and ASX343AT System on Chip (SOC).
The Distortion Correction tool can be used to generate Distortion Correction configuration settings for wide angle lens sensor module. It provides:

• Wide angle lens calibration tool
• Image Distortion Correction configuration tool
• Perspective correction configuration tool
• Miscellaneous: Import/Export configuration settings etc.
• Simulation mode which enables the user to apply distortion and perspective correction settings on static images.

Basic operation

1. To start the Distortion Correction tool GUI window, follow the following steps are:

Run DevWare, go to the main menu->Plug-ins-> Lens Distortion Correction Tools (Ver 2.0), and the Lens Distortion Correction Tools dialog is shown as followings:

Fig. 1 Distortion Correction DevWarePlugin

The tool can operate in one of two modes: MT9V126 Hardware or Software Simulation. In MT9V126 Hardware mode the process of correcting the lens distortion and adjusting the perspective of the image are done by the system hardware. In the Software Simulation mode, these processes are done by a simulation of the system hardware by DevWare. In order to operate in MT9V126 Hardware an Aptina Imaging Demo camera has to be installed connected to your computer.
When selecting one of these modes, DevWare will automatically switch the sensor to the output format which corresponds with your selected mode. For MT9V126 Hardware DevWare will set the sensor to output Dewarp+Overlay CCIR656 Interlaced image. For Software Simulation DevWare will set the sensor to output 640x480 YCbCr/RGB Progressive image.
You can also change the sensor output mode directly by clicking the Control button in the DevWare toolbar, which brings up the Sensor Control dialog. In there access Digital Video Output and make sure 640x480 YCbCr/RGB Progressive is selected if you want to apply Distortion Correction settings to Devware software simulator, Or Dewarp+Overlay CCIR656 Interlaced if you want to apply the current Distortion Correction setting to sensor.

1. In Software Simulation mode, the Distortion Correction output format can be specified via Format: in the Output Mode group. Three modes supported: NTSC, PAL and Progressive. In MT9V126 Hardware the output format is determined by the sensor settings and it can be modified by the DevWare sensor control pages.

1. In Software Simulation mode the Distortion Correction effect can be seen more clearly by selecting Circle Overlay, ten circles with different image heights will be shown on the current frame sensor output, check Enable and you can see what will happen to those circles after Distortion Correction processing. Check Open picture-in-picture if you want to see both images before and after Distortion Correction at the same time. These options are available in Software Simulation mode only.

1. The Distortion Correction Tool provides a default distortion correction setting for the Marshal wide angle lens. You can import a different setting by click Load Settings… or save the current settings for later use by click Save Settings…

You can find default settings which come with the tool in the folder
<installation-path>\Plugins\Default Lens Correction
You can apply the current Distortion Correction tool setting to either the software simulator or to the sensor by clicking the corresponding radio box at the bottom of Distortion Correction Tools, and then click on Apply button. If Software Simulation is selected, make sure Enable gets checked so that the new Distortion Correction setting can take effect in the Devware display window.

Lens Calibration Tool

The Lens calibration tool supports two modes: mathematical model based calibration and measurement based calibration.

Measurement based lens calibration

Check Calibration Lens to enable measurement based calibration. This mode allows the user to capture a set of images and use the information to create a text file.

Hardware/environment

Computer with two monitors with each monitor on a separate table are required, one is for displaying target images and the other is for Devware operations. Two tables should be used in order to minimize the table vibration during image capturing. The vibration needs to be less than 1 pixel in the sensor.
Other hardware includes Demo2 system with fine adjustment for height control and SOC 360 or MI360 with a wide angle lens mount.
The key issues to consider are [1]:
The monitor must be normal to the optical bench (no tilt)
The camera must be parallel to the monitor
An accurate measurement of the optical center of the lens is very important; the Cross-Hair image target can be used for this purpose.
An accurate flat-fielding of the image, make sure the experiment setup is not touched during image capturing.
The illumination of the room can be controlled ( lights need to be turned off during data collections)

Workflow

Specify the directory under which captured images will be stored in the edit box next to Images button. Or click Images button to browse to the target location. Enter Lens Name, and it will affect names of all captured images (ex: Largan).
Four target images are provided in this process:
crossHairWithLensFallOff_9Rings.bmp; captured image crossHairImage_largan.bmp
radialRingsWithLensFallOff_9Rings.bmp; captured image targetImage_largan.bmp
flatFieldWithLensFallOff_9Rings.bmp; captured image ffImage_largan.bmp
In this mode, seven steps are required in order to complete the procedure.

1. Click on Load Crosshair Target button, the cross-hair target image will be displayed on one of the monitors, double click to maximize the image.
2. Adjust lens focus; adjust camera height so that the image of the crosshairs shows up as straight lines at the center (co-ordinates [319,239]) of the Devware display. If it is not possible to adjust the camera so that the straight –lines line up the center of the display, then the optical center of the lens is not at the center of the sensor. Record the values of the horizontal and vertical co-ordinates where the cross-hair lines are straight (from Devware) and enter them in the Horizontal and Vertical edit boxes respectively, click on Capture Crosshair Target button.
3. Click on Load Flatfield Target button, double click to maximize the image.
4. Adjust Devware gains and exposure so that the average intensity in the image is around 200 digital counts and no part of the image is saturated. To check the average digital count, draw a graph through a column / row near the center of the image. Make sure aperture correction is disabled in Devware. Click on Capture Flatfield Target button.
5. Click on Load Scene Target button, double click to maximize the image
6. Click on Capture Scene Target button.
7. Click on Calibrate Button to initiate the distortion measurements.

Please note all the finished steps will get checked so the user does not have to do it again. If all the required images have already been captured before and stored at the specified location, you can directly go to step 7.





Example of an image of the cross-hair target, notice that the lines should be straight enough. See figure 2 (step 2).

Fig. 2 Captured crosshair target

Figure 3 is an example of captured flat-field image (step 4).
Fig. 3 Captured flat field image















Figure 4 shows an example of the calibration target image captured by an wide angle lens (step 6).

Fig. 4 Captured calibration target image














Notice
A grid line image may be used to locate the optical center of the lens. Find the point where both the horizontal and vertical line straight up and enter their coordinates to Horizontal and Vertical edit boxes respectively. Other steps are the same, instead of maximizing the flat-field image and the calibration target image display window; you may need to resize and move the window so that the tiny dot in the center of the image should be positioned at the optical center of the lens.








Fig. 5 Grid Line image

Mathematical model based lens calibration

A simplified mathematical model is introduced to simulate lens distortion. Given scene radius rScene, β parameter, focal length and pixel size, sensor radius can be computed as following:





Click Model Based Calibration radio box and … button, in Model Based Calibration dialog box, enter required information, and then click Apply, the generated lens model will be plotted in the distortion curve graph on Lens Calibration Tool page.

Load lens data from a different source

You may import lens calibration data from an outside text file which may have been saved earlier. The Distortion Correction tool comes with default settings for certain lenses, and the files can be found in the folder <installation-path>\Plugins\Default Lens Correction where <installation-path> is the directory you chose to install the DevSuite in.
The text file has the following format:
Coxccoyc
Xxxx xxxx (float point value)
rScene rSensor (in the unit of pixel numbers)
xxxxxxxx
xxxxxxxx
example:
coxc coyc
286 154
rScene rSensor
0 0
25 42
45 72
68 106
94 142
121 175
148 203
176 226
207 248
246 269
295 289
355 307
423 321
503 334
610 370
658 370
706 370
754 370
802 370



Loaded calibration data will be plotted in the distortion curve graph on Lens Calibration Tool page. See Figure 6.

Fig. 6 Import calibration data from a text file


Lens calibration data generated by the Lens Calibration Tool page will be exported to the Distortion Correction page automatically.

Distortion Correction Tool

Distortion correction fine tuning with known lens distortion profile

This tool is designed to help the user fine tune the Distortion Correction setting provided a lens distortion profile is already known. Three curves are plotted in the graph at the upper-left corner of the Distortion Correction page. The green one is the given lens distortion curve which is used as a reference for further fine tuning. Ten control points are exposed to the user: 40, 80,…,800 (radius in the Distortion Corrected image, in the unit of pixels). The user may either move the slider or enter a desired value in the edit box and then enter a space key, all those operations will cause the Desired Gain blue curve to change. Keep adjusting the blue curve until the Fitted Gain red curve matches your expectation.
The big red dot at coordinates (240,240) is an anchor point which the red curve should pass through.
Distortion Correction center horizontal (vertical) coordinate can be specified in Distortion Correction Center Horizontal (Vertical) edit box.
Perspective Correction can be enabled and disabled during Distortion Correction fine tuning process.
Two curve fitting options are available: manual and automatic, if the user selects Automatic, all the control points within Maximum Radius will participate Fitted Gain curve computation. You can decide how many control points are needed for Fitted Gain derivation by selecting Manual and enter a desired value in Maximum Radius edit box (recommend entering 800).
The user may move all the sliders up (down) by a scaling factor, for example if you enter 1.1, and then click on Scaling Factor button, all the new slider values will be 1.1 times of the current values.

Fig. 7 Distortion Correction Page

Perspective Correction Tool

Keystone correction is a useful technology in scenarios where the camera can not be mounted directly in front of the ROI (region of interest) of the scene. This tool allows user to define the keystone effect by eight parameters: Actual Angle, Actual Distance, Actual FOV, Virtual Angle, Virtual Distance, Virtual FOV, Virtual Shift and Virtual Aspect.
On the Perspective Correction page, the keystone effect is illustrated in Actual and Virtual Camera Position graph; both actual camera position (blue) and virtual camera position (green) are adjustable by moving sliders or entering desired values in the edit boxes. X perspective gain and Y perspective curves are plotted on the right part of this page in blue. Polynomial fitting results are plotted as red dotted curves.
Fig. 8 Perspective Correction Page

Advanced Tool

The Advanced tool gives the user the option to export and import the Distortion Correction configuration in several formats.
Import/Export Dewarp Settings is an option which is useful for capturing and restoring the mathematical model of the lens distortion and perspective correction.
Write to INI File allows exporting the configuration into a presets file which can then be used in order to program the sensor using DevWare Presets.
To Flash Image produces a binary file which is compatible with the SPI flash tool requirement for the Distortion Correction data.
Fig. 9 Perspective Correction Page

Known Issues / Workarounds

A list of known issues and their workarounds are presented in this section.

Slight Artificat in Image in MT9V126 or MT9V128

Issue; the bottom of the image has a few lines repeated.

Workaround; add "1" to the value of register 0x400A.