Archive for the ‘Vision’ Category

Lesion Segmentation — Patent Granted!

April 10th, 2012 No comments

MRI_Body.261135135_stdBack in 2008, I (along with colleagues at Siemens Corporate Research) invented a system to find and segment tumors in full-body MRI scans. It’s challenging to find all types of tumors across the entire body, but the ability to automatically detect tumors wherever they are can aid early detection and save lives.

We patented the findings in 2008 and received confirmation today that the patent has been granted (#8,155,405). Read on for a quick overview of the approach and a few useful links if you’re interested in seeing how it works!

System and Method For Lesion Segmentation In Whole Body MRI.
Gozde Unal, Gregory G. Slabaugh, Tong Fang, Shawn Lankton, Valer Canda, Stefan Thesen, and Shuping Qing. US Patent Number: 8,155,405. Filed March 2008. Granted April 2012.

   Click to continue →

Categories: Academic, Publication, Vision Tags:

Making Active Contours Fast

July 2nd, 2009 2 comments

Active contours are a method of image segmentation. They are well-loved for their accuracy, ease of implementation, and nice mathematical underpinnings. However, a full level-set implementation can be quite slow, especially when dealing with large data! Here are some tips to speed things up. By combining these ideas and solid programming techniques I’ve been able to get active contour trackers running at hundreds of frames per second!

  1. Use Fast Level-Sets
  2. Start by using a fast level-sets implementation that minimizes the number of required computations [code]. This will already save a huge number of computations per iteration and speed things up quite a bit!

  3. Create better initializations.
  4. The farther the initial contour is from its final position, the more computations must be done for the contour to converge. Hence, if you can start the contour in almost the right place, you’ll drastically reduce the time needed for segmentation. You can use prior knowledge, user input, or other segmentation techniques to create a rough guess that is close to the right answer. Another initialization that can leads to quick initialization is ‘bubbles’ on an evenly-spaced grid.

  5. Use a multi-scale approach.
  6. This is a way to quickly get good initializations using active contours. Say your data is MxN. Instead of segmenting the full data set, downsample the data so that you are dealing with an (M/8)x(N/8) volume. The segmentation should run much quicker on the smaller volume. Next, upsample the result back to MxN and use this as an initialization for the full data. The idea is that the time saved on the full segmentation by having a good estimate based on downsampled data will make up for the time needed to downsample, segment on the small data, and upsample.

  7. Use approximate active contours.
  8. Using an approximate solution for all or part of your segmentation can be helpful. As in 2 and 3, you can use an approximate active contour technique to quickly get close to the right answer. Then you can use an accurate level sets implementation to get the right answer quickly. Alternatively, the discrete methods can work quite well alone! James Malcolm proposed a nice method in “Fast Approximate Surface Evolution in Arbitrary Dimension” [code].

  9. Use another technique entirely.
  10. Active contours are “variational,” so they give nice, principled solutions with analytic geometry, etc. However, if you just want fast segmentations, other techniques such as thresholding/morphology, graph cuts, region growing, etc. can all be viable solutions.

Any other tips or links to good implementations? Leave them in the comments.

Sparse Field Active Contours

April 21st, 2009 95 comments

Active contour methods for image segmentation allow a contour to deform iteratively to partition an image into regions. Active contours are often implemented with level sets. The primary drawback, however, is that they are slow to compute. This post presents a technical report describing, in detail, the sparse field method (SFM) proposed by Ross Whitaker [pdf], which allows one to implement level set active contours very efficiently. The algorithm is described in detail, specific notes are given about implementation, and source code is provided.

Fast Level Sets Demo

The links below point to the technical report and a demo written in C++/MEX that can be run directly in MATLAB. The demo implements the Chan-Vese segmentation energy, but many energies can be minimized using the provided framework.

Sparse Field Method – Technical Report [pdf]
Sparse Field Method – Matlab Demo [zip]

To run the MATLAB demo, simply unzip the file and run:
at the command line. On the first run, this will compile the MEX code on your machine and then run the demo. If the MEX compile fails, please check your MEX setup. The demo is for a 2D image, but the codes work for 3D images as well.

My hope is that other researchers wishing to quickly implement Whitaker’s method can use this information to easily understand the intricacies of the algorithm which, in my opinion, were not presented clearly in Whitaker’s original paper. Personally, these codes have SUBSTANTIALLY sped up my segmentations, and are allowing me to make much faster progress towards completing my PhD!

Thanks to Ernst Schwartz and Andy for helping to find small bugs in the codes and documentation. (they’re fixed now!)

This code can be used according to the MIT license. As long as this work is appropriately cited and attributed, and not being used for proprietary or commercial purposes, I’m fully supportive of you using it. Please drop me a line if it helps you!

For more information regarding active contour, segmentation, and computer vision, check here: Computer Vision Posts

CVPR 2008 Wrap-Up and Selected Papers

June 29th, 2008 2 comments

I return today from a week-long trip to Anchorage, Alaska. I spent the week enjoying the beautiful mountains, and the exciting science being presentented at the Conference for Computer Vision and Pattern Recognition (CVPR 2008) [here are some links to lots of papers from the conference]. This was my first trip to this conference, and I must say that I was impressed with the quality of the work presented. Below, I list some of my favorite papers and give a (very) brief overview:

   Click to continue →

Categories: Academic, Vision Tags: ,

Tracking Through Changes in Scale

May 5th, 2008 1 comment

I will be presenting “Tracking Through Changes in Scale” at the International Conference on Image Processing (ICIP) in San Diego in October, 2008. This tracker uses a two-phase template matching algorithm in conjunction with a novel template update scheme to keep track of objects as their appearance and size changes drastically over the course of a video sequence.

The pdf, presentation material, and citation information will be available on the publications page after the conference. Below are videos of the experiments shown in the paper:

LEAVES Sequence (High Resolution Download – 11.2Mb)

VEHICLE Sequence (High Resolution Download – 34.8Mb)

BOAT Sequence (Hi Resolution Download – 2.34Mb)

Tracking and Surveillance Projects

May 3rd, 2008 5 comments

I took a special topics course in Spring 2008 at Georgia Tech, ECE 8893: Embedded Video Surveillance Systems. The course included three projects, each shown below. Detailed information about the algorithm is in the source code comments. (All the source is in Python)

Project 1: Activity Density Estimation

Use background subtraction to find moving foreground objects in a video sequence. Then, color-code regions with the most activity. Here is the result:


Project 2: Styrofoam Airplane Tracking

Find all white styrofoam planes in the scene and track them throughout the scene. We used color thresholding and simple dynamics to do the tracking.


Project 3: Pedestrian Tracking

Count and track the pedestrians that cross on a busy sidewalk. We use a combination of motion estimation via background subtraction and feature matching using the Bhattacharyya measure.

Final Report: p3.pdf

Most of this code is very hack-y because it was done quickly. However, it was
fun to learn Python, and the class was enjoyable overall.

Median Filter and Morphological Dilation in Python

May 1st, 2008 No comments

Python is a very nice programming language. Fast. Simple. Free. I recently spent some time learning it for a class on computer vision. I was using the PIL and numpy packages to make Python feel more like my old friend Matlab.

The two functions that I couldn’t find, and missed the most (especially when writing hack-y code for class projects) were median filtering and morphological dilation. So, in hopes of sparing other the pain of writing them… here they are! The function has both functions.

The medfilt() function uses the PIL filtering code. The dilate() function was written from scratch with NumPy.

Categories: Matlab, Vision Tags: ,

Fast 3D Stereo Vision

April 14th, 2008 47 comments

Recently, I started looking at faster ways to perform dense stereo matching for some work with 3D video. After some experimentation, I found out that by using a selective mode filter paired with naive correspondence matching, I was able to get satisfactory results very quickly. Check out the slide show below for some results!

[red indicates close, blue indicates far away]


Here is a download-able Matlab demo, which should work on any pre-aligned stereo image pairs:

The entire code is written in Matlab/C++/MEX. The stereo matching is all in Matlab, and the selective mode filter is coded in C++ and callable from Matlab (meaning it must be compiled before it can run). Currently, the correspondence is the major bottleneck, so anyone who can improve this, please let me know.

This code can be used according to the MIT license. As long as this work is appropriately cited and attributed, and not being used for proprietary or commercial purposes, I’m fully supportive of you using it. Please drop me a line if it helps you!

Selective Mode filter in Matlab

April 12th, 2008 3 comments

The median filter is a well-known image processing filter. It provides a very nice way to smooth an image while preserving edges. The median filter replaces each pixel in the image with the median value of its neighboring pixels. A similar non-linear filter with slightly different properties is the mode filter which replaces each pixel with the mode of its neighboring pixels. I additionally make a slight modification so that “bad” pixels are ignored entirely in the computation of the mode.

This idea arose when I was trying to de-noise some images as well as do some in-painting of “bad” pixels (that have no value). Consider the image below. The darkest-blue areas are bad pixels. We have no information for those pixels. The other pixels are colored to show how far that pixel is from the camera. (see the post on Stereo Vision) However, some of the good pixels still have the wrong value. These are the noise pixels.

original data
[Initial Image]

In the rest this post I talk about how we use a selective mode filter to convert the above image into the one below. (There’s also download-able Matlab/C++ code)

selective modefilt
[Final Result of Selective Mode Filter]

   Click to continue →

Categories: Matlab, Vision Tags: ,

Active Contour Matlab Code Demo

April 7th, 2008 20 comments

My new post: Sparse Field Active Contours
implements quicker, more accurate active contours.

Today, I added demo code for the Hybrid Segmentation project. This segmentation algorithm (in the publications section) can be used to find the boundary of objects in images. This approach uses localized statistics and sometimes gets better results than classic methods. For an example, see the video below: The contour begins as a rectangle, but deforms over time so that it finally forms the outline of the monkey.

This can be used to segment many different classes of image. To try it out, download the demo below and run >>localized_seg_demo

This code is based on a standard level set segmentation; it just optimizes a different energy. I’ve also made a demo which implements the well-known Chan-Vese segmentation algorithm. This technique is similar to the one above, but it looks at global statistics. This makes it more robust to initialization, but it also means that more constraints are placed on the image. Download it and see what you think! Again, unzip the file and run >>region_seg_demo (New! Described Here) (old and slow)

For another Matlab implementation of Active Contours check out: James Malcolm’s Webpage. He has some codes for very fast approximate implementations as well as a full numerical implementation.

This code can be used according to the MIT license. As long as this work is appropriately cited and attributed, and not being used for proprietary or commercial purposes, I’m fully supportive of you using it. Please drop me a line if it helps you!