Hooking CreateFile() Vista SP1's explorer.exe crashes explorer


Explorer process in Vista keep crashing when CreateFile() function is hooked, however it works fine in XP. Same problem for the FileMon example provided. The exception is thrown from shell32.dll.. any idea what's wrong?
Closed Aug 15, 2015 at 6:21 AM by spazzarama
old version


ChristophHusse wrote Mar 23, 2009 at 8:43 PM

I know this error... But I don't know why it occurs... From the debugging information I can see that this is something very strange. To my surprise the bug won't disappear when disabling the hook or the thread deadlock barrier. It seems as if the error automatically appears when you install the hook. A reason might be that, since explorer is a system application, that somewhere in shell32.dll CreateFile() is called under special assumptions (whole program optimization) and the hook srews up the whole thing by changing registers that shell32.dll did not assume to change. After all, I simply have no idea... The thing is, that it works for me on Windows Server 2008 (always), Vista 64-Bit (sometimes), Windows XP (sometimes), Windows 2000 (always). I think it heavily depends on the updates you have installed or explorer extensions or whatever...

wrote May 21, 2010 at 2:45 PM

wrote Feb 22, 2013 at 12:15 AM

wrote Aug 15, 2015 at 6:21 AM