That's the pseudocode:
for all frames
pData = (BYTE*)AVIStreamGetFrame(pFrame, i-1);
CreateYuvfromFrame(pData);
CloseAviFunctions();
printf("Done\n");
return;
Inside Visual Studio, I found at first no hint what was wrong, even tracing it showed no problem. Than I noticed that it does not freeze when the for-loop is not entered. Again reading in msdn, more specific AviStreamGetFrame, I read following:
Remarks:
The returned frame is valid only until the next call to this function or the AVIStreamGetFrameClose function.
Then I suddenly noticed, I called every necessary Avi-release and Avi-close function except the AviStreamGetFrameClose one. Inserting it just before my program ends and the bug was gone... and I was happy!
So what we learn from this: when using the Video for Windows Api, make sure to call the corresponding close-function for every Avifunction you use, although MSDN does not explicitly state that you have to before your program finishes!
But the open question remains: how to find this bug in a structured straightforward way? Normal debugging did not help and I was not willing the spend all my time to debug internal system calls within Ollydbg.
So if you know a way or you have just a suggestion, feel free to tell it me!