Unmanaged API - disabling API hooks for child thread calls to same API function

Aug 28, 2012 at 1:24 PM

I'm injecting a custom DLL into an EXE to wrap 3 non-M$ API functions exposed by a third party DLL.  The EXE I'm targeting is essentially a network API server - so after DLL initialization it will spawn threads as required to handle incoming requests from clients.

Unfortunately the 3rd party API doesn't work efficiently against modern storage architectures, so in order to have the API call complete in non-geological time I wrote a hook function handler that manages a threadpool within the process, taking work coming in from the API hook handler function and splitting the workload into a number of jobs allocated to an internal threadpool.  These then execute in parallel using the original API function and in testing do a far better job of getting the answer - about 600% faster.

I implemented the above using Detours - and everything works - we hook the API function and keep a pointer to the 'old' function.  Within our hook function - and any child threads it creates - we use this pointer to call the unadulterated original API.

Detours license kinda sucks as I guess everyone on here knows - so I've ported my DLL to EasyHook.  Mostly painless too, but for one thing - I'm having trouble with the "call the unadulterated original API function" aspect - there's no such thing in current EasyHook, at least as far as I can see.

Reading the Easyhook API guide the only way I can dream up to handle this is

1. At Worker Thread startup add CurrentThreadID to the "do not hook" Exclusive ACL of the function I'm using.

2. Call the API function - Easyhook will capture the call and check the thread ID against the ACL's - since we're 'forbidden' this will then pass through the call to the original API function

3. When the thread is ready to terminate we remove CurrentThreadID from the Exclusive ACL - just in case it gets recycled.

Is this the only way to handle this scenario or is there a better way anyone has used/can think of?

Cheers

Wullie

Coordinator
Sep 5, 2012 at 10:26 AM

Hi Wullie,

Yes I believe this would be the safest way to achieve this outcome with EasyHook.

Cheers,