Shawn Lankton Online

I came across a cute segmentation idea called “Grow Cut” [pdf]. This paper by Vladimir Vezhnevets and Vadim Konouchine presents a very simple idea that has very nice results. I always feel that the simplest ideas are the best! Below I give a brief description of the algorithm and link to the Matlab/C/mex code.

GrowCut Region Growing Algorithm

This algorithm is presented as an alternative to graph-cuts. The operation is very simple, and can be thought of with a biological metaphor: Imagine each image pixel is a “cell” of a certain type. These cells can be foreground, background, undefined, or others. As the algorithm proceeds, these cells compete to dominate the image domain. The ability of the cells to spread is related to the image pixel intensity.

The authors give some pseudocode that very concisely describes the algorithm.


//for every cell p
for all p in image
  //copy previous state
  labels_new = labels;
  strength_new = strength;
  // all neighbors q of p attack
  for all q neighbors
    if(attack_force*strength(q)>strength_new(p))
      labels_new(p) = labels(q)
      strength(p) = strength_new(q)
    end if
  end for
end for


Segmentation Results

Once implemented, this is a nice way to get segmentations. It is quite fast, and the initialization is very intuitive. Consider this picture of a lotus flower:

I made an initialization by clicking 20 points in the flower and 30 points outside. I then made a “label map” where unlabeled pixels are 0 (gray), foreground pixels are 1 (white) and background pixels are -1 (black).

Based on this simple initialization, we obtain a very decent segmentation:

As you can see, it isn’t perfect, but it is quite good. Its possible to interactively refine the seed points to improve the segmentation, but I didn’t do that here.

Matlab Code Downloads

I implemented this code in Matlab (using mex files due to the extensive use of for loops). You can download this below with compiled binaries for mac, linux, and windows. Unzip the file and run >>growcut_test for a demo.

UPDATE: I’ve fixed some bugs thanks to reader, Lin. The code works much better now!

Source & Compiled Binaries (96k) [zip]
“GrowCut” Paper [pdf]

Please let me know if you find this useful, and if you make improvements! Also, check out these related segmentation posts:

Related Segmentation Posts

• Mean-Shift Image Segmentation
• Localized Active Contours
• Fast Level Sets for Segmentation

Video tracking is widely used for surveillance, security, and defense purposes. In cases where the camera is not fixed due to pans and tilts, or due to being fixed on a moving platform, tracking can become more difficult. Camera motion must be taken into account, and objects that come and go from the field of view should be continuously and uniquely tracked. We propose a tracking system that can meet these needs by using a frame registration technique to estimate camera motion. This estimate is then used as the input control signal to a Kalman filter which estimates the target’s motion model based on measurements from a mean-shift localization scheme. Thus we decouple the camera and object motion and recast the problem in terms of a principled control theory solution.

Our experiments show that using a system built on these principles we are able to track videos with multiple objects in sequences with moving cameras. Furthermore, the techniques are computationally efficient and allow us to accomplish these results in real-time. Of specific importance is that when objects are lost off-frame they can still be uniquely identified and reacquired when they return to the field of view.

This work was published in the Proceedings of the SPIE on Electronic Imaging in this paper: Improved Tracking by Decoupling Camera and Target Motion.

See this paper and more on the publications page.

2D is nice, but these days I’m getting interested in doing computer vision in 3D. One way to get 3D data is to use two cameras and determine distance by looking at the differences in the two pictures (just like eyes!). In this project I show some initial results and codes for computing disparity from stereo images.    Click to continue →

Recently I wrote about some startup companies in computer vision. However, this is only part of a good industry analysis. I also want to explore some of the interesting research going on in the field. Below is a list of some of the vision research that I’ve come across that seems most interesting (and applicable/marketable).

Seam Carving

This is brilliant (and brilliantly simple work). It solves a problem, and in doing so gives us tools to solve problems we didn’t even know we had! Its hard to explain, check the video out.

Dr. Ariel Shamir has a host of other interesting research as well: link.

Read on for more great research:    Click to continue →

I have spent some time researching startup companies involved in computer vision. This has largely been in an effort to understand the marketability of computer vision research (which I spend much of my time learning about and contributing to). In this post, you’ll find a list of some notable companies. Let me know if you know of some other good ones. (Of course this doesn’t include the big, big companies like Siemens, GE, Phillips, and HP that are working on medical image processing every day!    Click to continue →

The super short answer: It’s what I study at Georgia Tech.
The longer answer: below…

As part of a program I’m involved in I needed to describe my research succinctly and from a very high level to forty of my colleagues. Many of the people who I was addressing did not have a technical background. I drafted the following short essay to describe what Computer Vision is, and what I do with it as of right now:

Computer Vision: A Summary

Sight, in my opinion, is the most incredible of our five senses. I conduct research in a field known as “Computer Vision.” This area focuses on the higher-level parts of a fascinating overall problem: Teach computers to see like people.

Teaching computers to see can be considered in three layers. The first is image acquisition. Here, I use the term image to mean data that a computer can interpret. This includes pictures like those from a camera, three-dimensional brain scans, and much more. These images are fantastic tools for people, and people alone. Computers cannot understand them without something more.

The next phase is image processing. This step makes the acquired images ‘nicer.’ This includes signal processing problems like removing noise, enhancing colors, and making important features more apparent. A processed image, though, still holds no meaning for the computer… only the humans that look at them.

The third layer is where my research is focused. The computer vision layer takes these processed images and assigns meaning to them. Three key activities here are segmentation, registration, and tracking. Segmentation involves finding the boundary of an object or objects of interest in a scene; registration is the process of lining up two images; and tracking is the process of determining the position of an object over time in a video sequence.
   Click to continue →

UPDATE:
My new post: Sparse Field Active Contours
implements quicker, more accurate active contours.

The well-known Chan-Vese segmentation algorithm from the paper “Active Contours Without Edges,” is a great example of active contours. This technique deforms an initial curve so that it separates foreground from background based on the means of the two regions. The technique is very robust to initialization and gives very nice results when there is a difference between the foreground and background means.

In this video, the curve begins as a square. As time goes on the square changes shape so that it does a better and better job of separating the image into a light area and a dark area.

Below is a download-able Matlab demo. The code is very easy to read, and could be the foundation for lots of other active contour segmentation techniques.

sfm_chanvese_demo.zip (New! Described Here)

regionbased_seg.zip (old and slow)

I recently added some new active contour stuff based on a more complex (and sometimes more capable energy). Check out the latest results, and the full project writeup which is a little older!

In this project I look at engineering from a different perspective… the business perspective. My team and I started with an idea: Track cars in a parking lot and use the information to help the whole operation run smoother. From there, we researched, though, and schemed until we had a sturdy business plan for the new venture.    Click to continue →

I’ve been working lots this past week… You could say I’m working too much because I’m choosing to work on research rather than do homework and write papers for my classes… oops! Anyway, the object of my obsession is some cool new visual tracking algorithms. I’ve been terribly remiss in putting up some details on my projects page, but in the mean time I’ll taunt my internet audience with some results I presented to my group at a meeting this Thursday: enjoy!

These are some people walking around outside my office.

Don’t know much about this video, but I’ll bet those are cars.

They were all tracked with a technique called “Kernel Tracking.” I can get these results pretty fast (real-time). Right now I’m working on some even cooler stuff with “Particle Filters.” More coming soon.

My newest research focus has moved away from medical images and towards moving images… And instead of finding organs and tumors, I’m tracking moving objects in the videos. Check out some of my latest successes (and failures).    Click to continue →