How to protect native usermode hooks?

Mar 4, 2009 at 10:42 AM
Hi

Its a common trick to directly call system calls by implementing the system call stub (load eax with sysnbr, load edx with stackptr, sysenter) in your own code and set up all the native stuff by your own. Is there a way to protect usermode native api hooks (e.g. NtCreateFile) from this simple bypass attack with easyhook?

Regards

Joe
Mar 4, 2009 at 1:49 PM
Yes, but I can't imagine that any productive application will use such native invocation, because it is very hard to maintain. This is typically used by malware! Mostly it is enough just hooking the NtXxx-APIs but even that can become very hard to maintain...

The only way to "catch" such invokations is by using kernel mode hooks, which EasyHook DOES support for 32- and 64-Bit kernels. The problem with 64-Bit is PatchGuard so I think only 32-Bit kernel mode hooks will work for productive applications...

regards
chris
Mar 4, 2009 at 2:17 PM
Hm is there a way to hook kernel functions from usermode through easyhook or another framework, with a callback like mechanism?

Regards

Joe
Mar 5, 2009 at 11:33 AM
Edited Mar 5, 2009 at 11:44 AM
I thought about this but then I decided not to do this because of the following:

1) THE MOST IMPORTANT: If you apply kernel hooks you should know what you are doing and if you want to hook the kernel via callback this is probably a sign that you don't know what you are doing, without saying that YOU don't know what you are doing but in general... Making mistakes in kernel code will cause the system to BSOD...
2) The approach will only work for a subset of kernel methods running at PASSIVE_LEVEL and maybe also at APC_LEVEL. Plus, you can easily deadlock your whole system.
3) There is a huge amount of security issues with the callback approach.

I have to admit that I did this myself when working at a software-firewall, but you can't provide such a mechanism in any secure, stable or reliable general purpose framework...

MY ADVISE:

1) Look which kernel mode routines to hook.
2) Hook them and queue calls in a stack.
3) Use a user-mode polling thread to query new calls, let's say every 100 ms. (I remember that there is also something called "Event-Pairs". You can create them in kernel mode and also in user mode (but the latter is something hacky) With these pairs you can set the kernel side of the event when new calls arrive and the corresponding user mode polling thread will IMMEDIATELY take over execution without any yield time! This approach is the most performant one I can imagine, but also not that easy to implement)
4) Process them in user mode and send ALLOW or DISALLOW back to kernel mode.

regards
chris
Mar 5, 2009 at 11:36 AM
Of course you will have to suspend the calls after queueing them into the stack (there is some equivalent to WaitForSingleObject in kernel mode but I forgot). If you get the result back from user mode you can continue execution...