Imaging Whiteboard

The new blob counter control in the Imaging Whiteboard (2.5.7) can be used for more advanced image analysis algorithms.

Here we see an image of M&Ms and we want to know how many blue ones are visible. The threshold control is used to separate the blue component of the image. The morphology controls are used to filter out spurious noise and partially visible M&Ms. The blob counter will identify the blobs and allow the user to select the blobs or interest. The selected blobs count is the answer.

Here we can see the results of two methods applied to the same image shown on the monitor simultaneously. The split screen feature will be available in version 2.5 of the Imaging Whiteboard.

Noise is added to the image and the set memory control will write the noisy image to the secondary memory. The temporal filter is applied to the primary memory. Swap memory switches the primary and secondary memories. The 3×3 median filter is applied. The monitor shows the primary image (morphology result) on the left, and the secondary (temporal filter) on the right.

White noise will contain all frequencies. By applying filters to white noise and viewing the resulting spectrum the effects can be viewed. Here we see the test signal generator producing white noise on 2 channels and the resulting spectrum. The high pass filter is applied to the signal and the resulting spectrum with low frequencies eliminated is shown. The low pass filter is then applied eliminating the high frequencies.

Tracking a target in a raw image may not provide the best tracking performance in all cases. Often tracking a target in a processed image is better. The pre-processing may be edge detection such as a Sobel filter (shown here), or setting a threshold, noise filtering etc.

Using the Set Memory control the original image is saved to memory. The pre-processing (convolution) is performed on the main pixels. The target is tracked in the processed image, but the crop that is passed to the next control is extracted from the image in memory (if it exists). The type of pre-processing will vary depending on the video used.

The latest version (2.3.2) released on this web site (not in the MS store yet) has a new “Low Frequencies Center” option added to the FFT filter control. This option is more convenient for common FFT filters.

Low Pass FFT filter

By including the low frequencies in the center of the FFT mask window a low pass filter can be implemented.

High pass FFT filter

By excluding the same low frequencies a high pass filter can be implemented.

Keeping the resolution low (256×256 in this case) will provide a reasonable live experience; still not quite real-time.

Now that Imaging Whiteboard 2.0 has been released this would be a good time to blog about some of the new features. The Warper has been improved to include variable warps. Previously only fixed warps were implemented; that is warps such as zoom and rotate where the warp factor is the same for every pixel, see previous blog . Version 2.0 includes variable warps; that is warps where the warp factor is dependent on the target image location (specifically distance from the center of the target image).

Complex number arithmetic is still used, but, the fixed warp factor is split into two components: Fixed Zoom Factor and Fixed Rotate Factor. These are calculated once. Then for each pixel the variable part of the warp factor is calculated from the user input and the target pixel address. The full warp factor is calculated:

(FixedZoomFactor + variableZoom) * (FixedRotateFactor * variableRotate)

Fish-eye correction can be achieved using the variable zoom. This works well enough, but, is wrong in a number of ways. First it is too simple, real fish-eye correction will take account of the source fish-eye lens properties see Fisheye lens correction ( if you really want the details. Also, this technique is based on the target pixel location as it implements backward mapped warping (more efficient), real fish-eye correction will use the source image location to determine the correction.

Fish-eye correction is shown in the user manual and further in the cookbook. Here we see biased rotation applied to a chequerboard:

A new control will allow the user to add random noise to the video.

The slider control will modify the amount of noise added from none to only noise.

This control will allow the user to evaluate various noise reduction techniques.

The temporal filter will do a pretty good job with high filter values, but, the problem of movement blurring will be worse at higher levels. This technique is only applicable to video, not single images.

The most obvious spatial technique would be a low pass convolution. This will result in a loss of detail, but is applicable to video or still images.

A couple of morphological operations will do a better job of removing noise, but will result in some image blocking.

The choice of noise reduction technique will depend on the application, the amount of noise anticipated, and personal taste. The Imaging Whiteboard will now allow the user to experiment with various techniques and combinations; varying the parameters and noise level to find the preferred approach.

The Mandelbrot generator has been added to the Imaging Whiteboard. The whole point of the Mandelbrot is infinite complexity from a simple algorithm. The plot points are generated with only 5 lines of code:

                    Complex z = new Complex(0.0, 0.0);
                    Complex c = new Complex(((double)w - MandelbrotSize / 2 + 
                                             HOffsetSliderValue) / (ZoomSliderValue * 64), 
                                            ((double)h - MandelbrotSize / 2 + 
                                             VOffsetSliderValue) / (ZoomSliderValue * 64));
                    uint iteration = 0;
                    for (; iteration < maxIterations && z.Magnitude < 2; iteration++)
                        z = (z * z) + c;

The control allows the user to zoom into details of the Mandelbrot and generate details.

The detail shown above is from the very left tip of the full scan. It seems that this detail repeats the full scan, but, not exactly. At first glance the Mandelbrot seems to contain much repetition; unlike fractals these repetitions are not exact.

The full un-zoomed plot looks like:

If you take a 3-dimensional object, rotate it 180 degrees in each of 3 dimensions in turn, the object will return to it’s original position.

I don’t know of an elegant mathematical proof of this; it would require the use of 3D complex numbers which do not exist (see the previous blog post ‘Using complex arithmetic to perform combination warps‘). There is an abundance of empirical evidence though.

Using the warp control of the Imaging Whiteboard we can perform this 3D transformation.

Flip Vertical, Flip Horizontal, and rotate 180 degrees. The image will return to it’s original not warped position.

So if I am using regular 2D complex numbers to perform the warp, how do I perform 3D warping?

To better explain this I have created a proof of concept that implements full horizontal and vertical rotation. This is not published (I’m not sure that it is useful).

After applying the warp factor:

sourceAddress = targetAddress * WarpFactor;

The result is modified:

sourceAddress.Real = sourceAddress.Real  / Math.Cos(fhRadians);  //Horizontal rotation

sourceAddress.Imag = sourceAddress.Imag  / Math.Cos(fvRadians);  //Vertical rotation

Manipulating the real and imaginary components is not really a correct use of complex numbers

Warps are generally backward mapped, that is for each pixel in the target image the address of the required pixel in the source image is calculated. Usually co-ordinate geometry is used to calculate the source address.

In the Imaging Whiteboard complex number arithmetic is used. The code to calculate the source pixel becomes:

sourceAddress = targetAddress * WarpFactor;

Yup that is the code!

The trick is to calculate the warp factor.

No warp is (1, 0), here the source is equal to the target address.

Zoom becomes (zoom, 0).

Rotation becomes (Cos (radians), Sin (radians))

So zoom and rotate becomes (zoom, 0) * (Cos (radians), Sin (radians))

Once the warp factor is calculated the warp is almost trivial!


If it’s that easy, why not use a three dimensional complex number to perform a time warp on a captured sequence? That could be fun.

Not so fast, according to the laws of mathematics there is no such thing as a 3 dimensional complex number. 2 sure, 4 OK, even 8.

So, use 4 dimensional complex numbers (quaternions) and ignore one dimension. Turns out that will not work either, mathematics is not so easily fooled.

So I could use co-ordinate geometry, which would be horribly slow and complicated. Or just introduce a DVR control, which is what anybody else would do. The DVR control solution would be limited in its functionality i.e. the same time would have to be applied to all pixels in a frame before the image warp.

None of this is very practical as the memory requirements would be prohibitive.

If you really want to know how to do a time warp you should watch this video.  TimeWarp