Shawn Lankton Online

Did you have accomplished stereo calibration and rectification?

I didn’t do the calibration and rectification. In fact, I used images from the Middlebury database and some other pre-aligned sources. If anyone has good information on how to accomplish rectification, please post it!

I did the calibration and rectification of my own images and this information is available here:
http://grauonline.de/wordpress/?p=5
I don’t know exactly why it does not work on all of my images so far, but I’ll post it on my blog once I found that out :-)
-Alexander

I tried d/l’ing from the MATLAB site and the demo wouldn’t run (Undefined command/function ‘edison_wrapper_mex’). Now I downloaded this version and I the demo, again, will not run (Undefined command/function ‘modefilt2_mex’). How do I fix this?

John: I tried to include pre-compiled binaries for the mex file. However, I don’t have binaries for all OS’s. Try:
>>mex modefilt2_mex.cpp
Then give it another shot.

Great work.

Hi,

I am working on a project on simulating the descent of a lander onto a planet’s `surface, so I was very interested to come across your MATLAB project for Fast 3D Stereo Vision.
My project is to do with analyzing simulated images of a planetary surface, & calculating a good landing site, for a space craft. We can do this with techniques like shadow mapping & LIDAR.
I would like to look at Stereo Vision & I was hoping that I could incorporate your MATLAB project for Fast Stereo Vision into my program. Would this be OK with you?

I have extracted images from my proram & fed them into your project but my early results have been mixed. No fault of your program though! The images that I am using from our planetary surface simulator, are not great. They are quite dark & there is not much contrasting distances – mainly dunes & rocks. The resulting disparity maps have been somewhat unclear, however I am working on it. Any advice/suggestions you might have for alterations to your program, to make it more suitable for planetary surfaces would be greatly appreciated!!

Thanks again,

Carmen

Thanks so much for the rapid response and advice!!

Hi,

Some cleaning of my images & tweaking of the window size parameter have improved my results. I just have 1 issue that has been driving me mad. I am trying to save the filtered disparity map to a bitmap file, but the colors are all wrong, as compared to the image displayed in MATLAB. I presume its something to do with colormaps.
The code I added to your demo.m is:

colormap(‘default’);
cmap = colormap;
imwrite(d, cmap, ‘filtered.bmp’,'bmp’);

This produces a bitmap but the wide rage of colors that are displayed in the MATLAB image are all shifted to various shades of blue. Have you any ideas?

Great advice, it finally works.
Thanks very much.

Hello Shawn,

I have recently started working on 3D vision using Stereo Disparity.
I have found you code and your website really useful.

I will get in touch with you when i need some help.

many thanks
naga

Hi Shawn,

I would like to use your algorithm as a source for computing a depth map-based metric to evaluate the quality of an algorithm I am submitting for publication. You mention that your algorithm is based upon (“Segment-Based Stereo Matching Using Belief Propogation and a Self-Adapting Dissimilarity Measure” by Klaus, Sormann, and Karner). Is there any way that your matlab implementation of this algorithm is cite-able?

Thank you,
Kyle

Hi Shawn,

Although, I can get a depth map finally, but how can I find the largest disparity value(offset value) between Left and Right image?

Thanks

Hi,i still can not understand how to use displarity plane to fit each segmention region…would you please explain to me? and how to programm it using matlab?
thank you very much

it is very work , and i need your help,
f = modefilt2_mex(img,win,ignore);
this function Undefined and i want what is the meaning by this function and i will try to pre-compiled it.
please send to me as fast as possible.
regards
emy

Hi Shawn! I found ur code very useful for my projects but I’m having problem here I tried ur codes with images from Middlebury database the teddy and cone stereo images. I found put it dosen’t work with your code. I tried changing the window size and set the min and max disparity still fails me.can you help me out? thank you

May I know how I can set the maxs parameters correctly??

Hi,shawn lankton

i still can not understand how to use displarity plane to fit each segmention region…would you please explain to me? and how to programm it using matlab? which part is diparity plane code in your code?

thank you very much

thank you so much about fast response but , i don’t know how i can compile this file in matlab , when i write this
>>mex modefilt2_mex.cpp
the result is
Error: Could not detect a compiler on local system
which can compile the specified input file(s) at C:\MATLAB7/bin/win32/mexsetup.pm line 519.
??? Error using ==> mex
Unable to complete successfully

i think that bacause i type of operating system i used , iam using window xp
what is your opinion?
regards
emy

@emy

You have to set up MEX so that it knows what compiler to use to compile the code. Type in mex -setup in MATLAB to set that up.

Hope this helps,
- Ray.

@ Ray
thank you so much about your answer, i setup mex and when i write this line
mex f:\modefilt2_mex.cpp the resulte is
Error F:\modefilt2_mex.cpp: 82 illegal expression
Error F:\modefilt2_mex.cpp: 82 found `int’ expected a function
Error F:\modefilt2_mex.cpp: 82 operands of = have illegal types `int’ and `void’
Error F:\modefilt2_mex.cpp: 100 illegal expression
Error F:\modefilt2_mex.cpp: 100 found `int’ expected a function
Error F:\modefilt2_mex.cpp: 100 operands of = have illegal types `int’ and `void’
6 errors, 0 warnings

C:\MATLAB7\BIN\WIN32\MEX.PL: Error: Compile of ‘F:\modefilt2_mex.cpp’ failed.

??? Error using ==> mex
Unable to complete successfully

so what can i do??!! , i can’t deal with c.
regards
emy

Hellow Yawn,
I have not found the matrix.h, mex.h and except.h files. Please help me where I can find these files to run the programm. I have search these files on google and download, but when I run the programm it gives errors.

Hello Shawn,
I am a student working on 3D video coding project. Initially I have to get some depth maps. I am using your code to calculate the disparity maps of stereo pairs. I am using the test sequences from http://www.3dtv-research.org/publicSwLibrary.php website. I am not getting good results but I noticed a very strange thing that my results are better when i interchange the input images – Right and left.
I guess it has something to do with trying out different value for MAXS – I did, but it is not working. Can you advise me on this please? Thank you so much.

Here is the result with the correct inputs – http://picasaweb.google.com/karthik.divakaruni/DisparityMapRoom3D#5467026926067829202

Here is the result with right and left interchanged –
http://picasaweb.google.com/karthik.divakaruni/DisparityMapRoom3D#5467027926374654338

I compared the images that you used with my test sequences and I noticed that I need to give the inputs as i did in the image of the second link above. However, the output lacks detail and if I decrease the window size, it does not improve. Also, if you notice the roof of the room in the image(second link), it has blue stripes that follow the texture in the original image. Any ideas how to avoid this?
Thanks a lot.
-Karthik.

Dear Shawn, thank you very much for your rapid response. I guess, this accuracy is enough for my work. However, now that I have the disparity map, I want to calculate the depth map from it. I have all the camera parameters. So how do I calculate the depth maps? I know some formulas that use the disparity value to calculate the depth map,but is there a place where I can find the code to do the same? Or if it is easy to code, can I do it myself? Please advise me. Here are the camera parameters I have:

f = 24,287 mm (focal length)
Znear = 0 m
Zfar = 30m
Distance between left and right camera: 68mm

Hi, thanks for your code.
When I tried using the Tsukuba pictures, I get the same result as your picture.
But if I use your code with my pictures, there are a few problems.
Matlap give my images the Value 382×512 uint8, in Tsukuba, it is 288x284x3 uint8
That’s a problem.
Can you tell me how can I convert my images so that the code is running?

Thanks for your help.

Hi, thanks for your code.

Can you tell me how can I compile with 64 bits so that the code is running?

Because when I try to run “demo”, I have this error :

??? Undefined function or method ‘edison_wrapper_mex’ for input arguments of type ‘struct’.

Error in ==> edison_wrapper at 64
[fimage labels modes regSize grad conf] = edison_wrapper_mex(fim, rgbim, p);

Error in ==> msseg at 42
[fimg labels modes regsize grad conf] = edison_wrapper(I,@RGB2Luv, …

Error in ==> total_stereo at 58
[segs labels] = msseg(i1,hs,hr,M); %– mean shift segmentation

Error in ==> demo at 16
[d p s l] = total_stereo(i1,i2, hs,hr,M,mins, maxs);

And when I run “compile_edison_wrapper”
I have this error:

C:\PROGRA~1\MATLAB\R2009A\BIN\MEX.PL: Error: ‘edison_wrapper_mex.cpp’ not found.

??? Error using ==> mex at 218
Unable to complete successfully.

Error in ==> compile_edison_wrapper at 5
mex -O edison_wrapper_mex.cpp …

I don’t know what I must change here

mex -O edison_wrapper_mex.cpp …
segm/ms.cpp segm/msImageProcessor.cpp segm/msSysPrompt.cpp segm/RAList.cpp segm/rlist.cpp …
edge/BgEdge.cpp edge/BgImage.cpp edge/BgGlobalFc.cpp edge/BgEdgeList.cpp edge/BgEdgeDetect.cpp
fprintf(1, ‘Done edison_wraper compilation\n’);

so that the code run with 64 bits !!!!!!!!!!!

Thanks for your help.

Thanks a lot for your helpful code. I am wondering if it is possible to extract the labelling code from the segmentation code.

I look forward to hearing from you soon

thanks for your help.

@Sonia: It sounds like you don’t have MEX properly setup. The main reason why is because when it tries to execute the MEX command, the error outputs indicate that it can’t find MEX, which is only found after you set MEX up. MEX requires that MATLAB knows which C/C++ compiler that you’re using. In the MATLAB command prompt, type in the following (without quotes) “mex -setup”, and go from there. You need to have a C/C++ compiler installed on your computer, or you won’t be able to run the code. Once you do that, run the compile_edison_wrapper again so that the code compiles on your 64-bit machine. It should work after. I’m running Windows 7 64-bit Ultimate, and the code works fine with me.

@Joseph: The original EDISON code is actually poorly written. It can get a bit obfuscated. However, the labelling is actually not too bad. Once the mean shift segmentation algorithm finishes, be sure to save all of the centroids for each cluster that has been identified. Once you’ve identified all of the centroids, look for points where the joint Euclidean distance (both colour and distance to the centroid) is less than some number (I chose 0.5). If a point’s distance is less than this amount, it belongs to that cluster. Basically, use a loop, have a variable that stores a centroid at a particular iteration, then loop through all points in the image and check to see if points are close to the centroid in question. This brute force technique takes a very long time, so the EDISON authors use efficient nearest neighbour techniques to decrease the amount of time that this executes.

Hope this helps,
- Ray.

Hey Shawn,

May I ask you why there are many wrong small regions at the top right of tsukuba image and can you guess any method to just conceal these errors?
Thanks

hi
i have looked ur blog long time…It is a good work…at least i think so…
rencently i am focusing on stereo matching about Belief Propagation …so do you have touched this method? i have some confused things …
would you give me an email address…..thx
best wishes…

hello shawn….
I am working on project 3D object recognition using stereovision.
I am having problems in getting the disparity map. can u please tell me how to select the maximum disparity value. Is their some formula to get it.
And one thing, what should be the maximum distance between the two cameras while making a stereo vision system….
Please help me…..i am dying to have results….
thank u….

hi shawn
Nice work.shawn can u plz guide how can i set maxs and mins accurately according to pick .Any way to find that .i m not getting good resultsss..helpp meeeee outttttt

hi shawn
Nice work.Shawn plzz Guide how can i set Maxs and mins Accuratelly.Any specific method to calculate that.I m not getting Good Results plzzzzz help me outtt

Hi,
What paper did you use to do this code, Segment-based stereo matching using beleif propagation and self-adopting dissimilarity measure, or another paper?? If so, can you tell me which, and its link, if possible??

Thanks for your help

Hi Shawn,

Great work. I really like the excellent performance of your code.

However I have two questions concerning the function “stereo.m”:

1) In line 59
“pd = shift_image(pixel_dsp,5); %– shift to match i1”
the disparity map is shifted by 5 pixel. As far as I understood the disparity map should already be aligned according to image i1. So why does one still need to shift the disparity map?

2) The subfunction “function I = shift_image(I,shift)” (line 98) does not shift the image for a parameter setting of shift=1. In fact it provides the same result as shift=0. Does the subfunction need some modification?

I hope you have some time to take a look at my questions.