Use Fast Level-Sets

Start by using a fast level-sets implementation that minimizes the number of required computations [code]. This will already save a huge number of computations per iteration and speed things up quite a bit!

Create better initializations.

The farther the initial contour is from its final position, the more computations must be done for the contour to converge. Hence, if you can start the contour in almost the right place, you’ll drastically reduce the time needed for segmentation. You can use prior knowledge, user input, or other segmentation techniques to create a rough guess that is close to the right answer. Another initialization that can leads to quick initialization is ‘bubbles’ on an evenly-spaced grid.

Use a multi-scale approach.

This is a way to quickly get good initializations using active contours. Say your data is MxN. Instead of segmenting the full data set, downsample the data so that you are dealing with an (M/8)x(N/8) volume. The segmentation should run much quicker on the smaller volume. Next, upsample the result back to MxN and use this as an initialization for the full data. The idea is that the time saved on the full segmentation by having a good estimate based on downsampled data will make up for the time needed to downsample, segment on the small data, and upsample.

Use approximate active contours.

Using an approximate solution for all or part of your segmentation can be helpful. As in 2 and 3, you can use an approximate active contour technique to quickly get close to the right answer. Then you can use an accurate level sets implementation to get the right answer quickly. Alternatively, the discrete methods can work quite well alone! James Malcolm proposed a nice method in “Fast Approximate Surface Evolution in Arbitrary Dimension” [code].

Use another technique entirely.

Active contours are “variational,” so they give nice, principled solutions with analytic geometry, etc. However, if you just want fast segmentations, other techniques such as thresholding/morphology, graph cuts, region growing, etc. can all be viable solutions.

Any other tips or links to good implementations? Leave them in the comments.

Fast Level Sets Demo

The links below point to the technical report and a demo written in C++/MEX that can be run directly in MATLAB. The demo implements the Chan-Vese segmentation energy, but many energies can be minimized using the provided framework.

Sparse Field Method – Technical Report [pdf]
Sparse Field Method – Matlab Demo [zip]

To run the MATLAB demo, simply unzip the file and run:
>>sfm_chanvese_demo
at the command line. On the first run, this will compile the MEX code on your machine and then run the demo. If the MEX compile fails, please check your MEX setup. The demo is for a 2D image, but the codes work for 3D images as well.

My hope is that other researchers wishing to quickly implement Whitaker’s method can use this information to easily understand the intricacies of the algorithm which, in my opinion, were not presented clearly in Whitaker’s original paper. Personally, these codes have SUBSTANTIALLY sped up my segmentations, and are allowing me to make much faster progress towards completing my PhD!

Thanks to Ernst Schwartz and Andy for helping to find small bugs in the codes and documentation. (they’re fixed now!)

For more information regarding active contour, segmentation, and computer vision, check here: Computer Vision Posts

Edge Finding

Using Contours to Detect and Localize Junctions in Natural
Images. Michael Maire, Pablo Arbeláez, Charless Fowlkes and Jitendra Malik. CVPR, 2008. [paper]

Maire et al. Make significant progress in edge detection and junction detection by using very clever insights into the way humans perceive and draw images. They also demonstrate using convincing metrics that their methods are the state of the art and rapidly approaching human-level of accuracy.

Stereo and 3D

I was impressed with the amount of focus on 3D reconstruction from either stereo cameras or monocular video cameras. This is something that I have been interested in for some time. Due to the apparent focus on this I may pursue this further in the coming months and try to add myself to this list!

Recovering Consistent Video Depth Maps via Bundle Optimization. Guofeng Zhang, Jiaya Jia, Tien-Tsin Wong and Hujun Bao. CVPR, 2008. [project website]

This was one of the first talks I saw and I was floored by the results. The authors use a single video stream to compute beautiful 3D models of the scene. The algorithm makes use of the temporal consistency of video as well as visual similarities between subsequent frames.

Stereoscopic Inpainting: Joint Color and Depth Completion from Stereo Images. Liang Wang, Hailin Jin, Ruigang Yang and Minglun Gong. CVPR, 2008.

Here, authors combine the ideas of in-painting (filling missing areas in an image automatically) and stereo depth estimation to dramatically improve both disciplines. By using in-painting methods, occlusions in stereo pairs can be accurately completed, and by supplementing visual in-painting with depth estimates, better results can be obtained.

3D Point Clouds

The analysis of shapes and 3D objects often relies on the use of a cloud of points on the surface of an object. Although this wasn’t as popular as stereo at CVPR, it was still a noteworthy discipline… and one that is likely to grow.

Three-Dimensional Point Cloud Recognition via Distributions of Geometric Distances. Mona Mahmoudi and Guillermo Sapiro. CVPR, 2008. [google cache of paper]

Shape recognition is a very complicated problem. Here, Mahmoudi and Sapiro take a very refreshing new approach. By computing histograms of features such as pair-wise distances, and curvature they create a system capable of matching shapes very well to ones in large databases. Their method simplifies this hard task and yields excellent results.

Particle Filtering for Registration of 2D and 3D Point Sets with Stochastic Dynamics. Romeil Sandhu, Samuel Dambreville and Allen Tannenbaum. CVPR, 2008.

Cry nepotism if you like, but this is excellent paper from my colleague Rome Sandhu. Lining up point clouds is a task that crops up all over. This technique allows registration of small bits of data onto known models and even bits of data onto other bits of data. The results are simply incredible considering it would be a challenge to do this even for a human!

Visual Summary

Summarizing Visual Data Using Bidirectional Similarity. Denis Simakov, Yaron Caspi. Eli Shechtman and Michal Irani. CVPR, 2008. [project website]

I was amazed last year by a technique called Seam Carving used to re-size images without losing or shrinking important information. This technique accomplishes the same goal, and sometimes does a far better job. The approach is two-fold: 1) make sure the smaller image has as much data as possible from the original, 2) make sure the smaller image doesn’t have any *new* data. The second constraint ensures that no artifacts develop.

Skeletonization

Geometric Modeling of Tubular Structure. Huseyin Tek and M. Akif Gulsun. MMBIA(CVPR), 2008.

I am currently working on vessel analysis myself, and have been reviewing different methods of construction a “skeleton” or stick-like model of a vessel structure. Tek and Gulsun of Siemens Corporate Research have a very clever solution wherein they find the best line through a structure by measuring the “roundness” of the surrounding data.

My Work

Localized Statistics for DW-MRI Fiber Bundle Segmentation. Shawn Lankton, John Melonakos, James Malcolm, Samuel Dambreville and Allen Tannenbaum. MMBIA(CVPR), 2008. [pdf paper]

Just in case you were wondering, this is the paper that brought me to Alaska to take part in this conference. I use a method that looks locally at image differences in brain scans to find the shape of neuron bundles that run through the brain.

The pdf, presentation material, and citation information will be available on the publications page after the conference. Below are videos of the experiments shown in the paper:

 
LEAVES Sequence (High Resolution Download – 11.2Mb)

 
VEHICLE Sequence (High Resolution Download – 34.8Mb)

 
BOAT Sequence (Hi Resolution Download – 2.34Mb)

The two functions that I couldn’t find, and missed the most (especially when writing hack-y code for class projects) were median filtering and morphological dilation. So, in hopes of sparing other the pain of writing them… here they are! The function medfilt_dilate.py has both functions.

medfilt_dilate.py

The medfilt() function uses the PIL filtering code. The dilate() function was written from scratch with NumPy.

Here is a download-able Matlab demo, which should work on any pre-aligned stereo image pairs:

stereo_modefilt.zip

The entire code is written in Matlab/C++/MEX. The stereo matching is all in Matlab, and the selective mode filter is coded in C++ and callable from Matlab (meaning it must be compiled before it can run). Currently, the correspondence is the major bottleneck, so anyone who can improve this, please let me know! Enjoy.

This idea arose when I was trying to de-noise some images as well as do some in-painting of “bad” pixels (that have no value). Consider the image below. The darkest-blue areas are bad pixels. We have no information for those pixels. The other pixels are colored to show how far that pixel is from the camera. (see the post on Stereo Vision) However, some of the good pixels still have the wrong value. These are the noise pixels.

[Initial Image]

In the rest this post I talk about how we use a selective mode filter to convert the above image into the one below. (There’s also download-able Matlab/C++ code)

[Final Result of Selective Mode Filter]

First Try

First, we compare a standard median filter along-side a standard mode filter. We see that the two results are very similar. I deem the mode filter to be slightly better, but this is pretty subjective. As you can see, both methods do a good job of eliminating noise although neither filter removes the bad pixels.

  
[Standard Median Filter, Standard Mode Filter]

Being Selective

The solution is to ignore bad pixels so that they can not contribute to the computation of the median or mode (depending on which filter you use). In my example, bad pixels are value ’0′ while all the other pixels are positive and non-zero. Therefore, I re-wrote the mode filter to ignore pixels under a threshold when computing the local mode around each pixel. The result is very nice. Some of the fine-scale features are lost, but overall, the number of noise pixels and bad pixels are greatly reduced.

Conclusion & Download

This is a simple idea, but it had very nice results. A good next step would be to create a selective median filter. I implemented this in mex so the operation is very quick. In the demo below I include binaries for most platforms, but if doesn’t work on your computer you may need to re-compile.

modefilt.zip

UPDATE:
My new post: Sparse Field Active Contours
implements quicker, more accurate active contours.

Today, I added demo code for the Hybrid Segmentation project. This segmentation algorithm (in the publications section) can be used to find the boundary of objects in images. This approach uses localized statistics and sometimes gets better results than classic methods. For an example, see the video below: The contour begins as a rectangle, but deforms over time so that it finally forms the outline of the monkey.

This can be used to segment many different classes of image. To try it out, download the demo below and run >>localized_seg_demo

localized_seg.zip

This code is based on a standard level set segmentation; it just optimizes a different energy. I’ve also made a demo which implements the well-known Chan-Vese segmentation algorithm. This technique is similar to the one above, but it looks at global statistics. This makes it more robust to initialization, but it also means that more constraints are placed on the image. Download it and see what you think! Again, unzip the file and run >>region_seg_demo

sfm_chanvese_demo.zip (New! Described Here)

regionbased_seg.zip (old and slow)

For another Matlab implementation of Active Contours check out: James Malcolm’s Webpage. He has some codes for very fast approximate implementations as well as a full numerical implementation.

GrowCut Region Growing Algorithm

This algorithm is presented as an alternative to graph-cuts. The operation is very simple, and can be thought of with a biological metaphor: Imagine each image pixel is a “cell” of a certain type. These cells can be foreground, background, undefined, or others. As the algorithm proceeds, these cells compete to dominate the image domain. The ability of the cells to spread is related to the image pixel intensity.

The authors give some pseudocode that very concisely describes the algorithm.


//for every cell p
for all p in image
  //copy previous state
  labels_new = labels;
  strength_new = strength;
  // all neighbors q of p attack
  for all q neighbors
    if(attack_force*strength(q)>strength_new(p))
      labels_new(p) = labels(q)
      strength(p) = strength_new(q)
    end if
  end for
end for


Segmentation Results

Once implemented, this is a nice way to get segmentations. It is quite fast, and the initialization is very intuitive. Consider this picture of a lotus flower:

I made an initialization by clicking 20 points in the flower and 30 points outside. I then made a “label map” where unlabeled pixels are 0 (gray), foreground pixels are 1 (white) and background pixels are -1 (black).

Based on this simple initialization, we obtain a very decent segmentation:

As you can see, it isn’t perfect, but it is quite good. Its possible to interactively refine the seed points to improve the segmentation, but I didn’t do that here.

Matlab Code Downloads

I implemented this code in Matlab (using mex files due to the extensive use of for loops). You can download this below with compiled binaries for mac, linux, and windows. Unzip the file and run >>growcut_test for a demo.

UPDATE: I’ve fixed some bugs thanks to reader, Lin. The code works much better now!

Source & Compiled Binaries (96k) [zip]
“GrowCut” Paper [pdf]

Please let me know if you find this useful, and if you make improvements! Also, check out these related segmentation posts:

Related Segmentation Posts

• Mean-Shift Image Segmentation
• Localized Active Contours
• Fast Level Sets for Segmentation
]]> http://www.shawnlankton.com/2008/03/growcut-segmentation-in-matlab/feed/ 27