Wednesday, September 17, 2008

A bug hard to find...

During the last evenings, I have been coding a little tool to convert Avi files to 4:2:0 Yuv files. I just wanted it to have more videos to test with my H.264 encoder I work with in my diploma thesis. Well, it was no big deal programming it, just diving into the Video for Windows Api on MSDN to get familar with its interface so it didn't take long till I finished it. It's a console application and I always tested inside Visual Studio, passing the commandline parameters in the project settings. So everything worked fine till I tested it inside a cmd window outside Visual Studio. The converting loop was successful but it freezed after printing "Done!". Looking at the code I was surprised cause that printf("Done!\n") is the very last command!
That's the pseudocode:

for all frames
   pData = (BYTE*)AVIStreamGetFrame(pFrame, i-1);

Inside Visual Studio, I found at first no hint what was wrong, even tracing it showed no problem. Than I noticed that it does not freeze when the for-loop is not entered. Again reading in msdn, more specific AviStreamGetFrame, I read following:


The returned frame is valid only until the next call to this function or the AVIStreamGetFrameClose function.

Then I suddenly noticed, I called every necessary Avi-release and Avi-close function except the AviStreamGetFrameClose one. Inserting it just before my program ends and the bug was gone... and I was happy!

So what we learn from this: when using the Video for Windows Api, make sure to call the corresponding close-function for every Avifunction you use, although MSDN does not explicitly state that you have to before your program finishes!

But the open question remains: how to find this bug in a structured straightforward way? Normal debugging did not help and I was not willing the spend all my time to debug internal system calls within Ollydbg.
So if you know a way or you have just a suggestion, feel free to tell it me!

No comments: