Vista 64 SP1 Gdi32 StartDocW hook help please

May 12, 2009 at 7:59 PM

Hi All,

I am trying to hook the Gdi32->StartDocW under Vista 64 SP1. The Hook itself is fired up but cause the crash of application (Notepad for example) after return back the int parameter. It's OK under XP SP2.

Thanks!

Igor

//hook install

                    StartDocWHook = LocalHook.Create(
                        LocalHook.GetProcAddress("gdi32.dll", "StartDocW"),
                        new DStartDocW(StartDocW_Hooked),
                        this);

                    StartDocWHook.ThreadACL.SetExclusiveACL(new Int32[] { 0 });

// Hook staff itself

        // DELEGATES
        delegate int DStartDocW( System.IntPtr hdc, ref DOCINFOW lpdi);

        /// Return Type: int
        ///hdc: HDC->HDC__*
        ///lpdi: DOCINFOW*
       [System.Runtime.InteropServices.DllImportAttribute("gdi32.dll", EntryPoint = "StartDocW")]
        static extern int StartDocW(System.IntPtr hdc,ref DOCINFOW lpdi);

        public static int StartDocW_Hooked( System.IntPtr hdc, ref DOCINFOW lpdi)
        {
            int ret = 0;
            EdgeMonInject This = (EdgeMonInject)HookRuntimeInfo.Callback;  
           
            This.Queue.Push("***StartDocW_Hooked " + DateTime.Now.ToShortTimeString() + "[" + RemoteHooking.GetCurrentProcessId() + ":" +
                RemoteHooking.GetCurrentThreadId() + "]: \" CALL BLOCKED File:" + lpdi.lpszDocName +"\"");

            //ret = StartDocW(hdc, ref lpdi);

            return ret;
        }

May 13, 2009 at 12:21 AM

Interesting. I can reproduce this behaviour. Could be related to this: http://social.msdn.microsoft.com/Forums/en-US/clr/thread/ff1415f3-b47c-40f5-a73d-4fd7f4aa1187 perhaps? The .net interop releasig a DOCINFO string when it should not? Don't know though. If you replace public static int StartDocW_Hooked( System.IntPtr hdc, ref DOCINFOW lpdi) with public static int StartDocW_Hooked( System.IntPtr hdc, IntPtr lpdi) the crash doesn't happen.

A workaround is marshalling manually:

    public static int StartDocW_Hooked(System.IntPtr hdc, IntPtr lpdi) {

...

DOCINFOW foo = (DOCINFOW)Marshal.PtrToStructure(lpdi, typeof(DOCINFOW));

...

return 0 }

Perhaps Christoph knows whats going wrong.

Ben Schwehn

Coordinator
May 13, 2009 at 8:03 AM
Edited May 13, 2009 at 8:06 AM

No I don't ;-).

 

But if your solution with marshalling manually works fine then I can only assume two things. Either the NET version of the DOCINFOW structure itself is wrong, or you shouldn't use a "ref" for passing it as parameter. Instead you could try to omit the marshalling by using "static extern unsafe int StartDocW(System.IntPtr hdc, DOCINFOW* lpdi);" and "unsafe delegate int DStartDocW( System.IntPtr hdc, DOCINFOW* lpdi);".

But then I think you would also have to convert the DOCINFOW structure to an unsafe one (without NET types). Which means that "String" for example would be "char*". You can access such pointers with Marshal.UnicodeFromPtr() or something like that....

regards chris

Coordinator
May 13, 2009 at 8:09 AM

After all it is always a little bit tricky to provide C# hook handlers. The better way, and this is what I would do in almost any case, is to provide a C++.NET library doing the hooking part and use a C# library for the "high level" NET work. This way you are able to provide clean hook handlers by using exactly the same definition that is stated in the windows native headers. The NET compiler will then do all the dirty work to convert the "native code" to NET. So you got some kind of native C++ code compilation to NET, which is a great utility! But of course if you have no clue about C++ then you shouldn't go this road, because C++.NET is even harder to understand ;-).

May 13, 2009 at 11:29 AM

Thank you so much!

Ben: I've used your solution and it's work! Oh, thanks!

Christoph: Thanks, and regarding C++: of course, you're right, but C# help us to try to keep the project development in scopes and timerange.

Igor

-------- Sorry, I've missed out the DOCINFO been used in my request. You may found the struct below:

        //=======================================================
        [System.Runtime.InteropServices.StructLayoutAttribute(System.Runtime.InteropServices.LayoutKind.Sequential)]
        public struct DOCINFOW
        {

            /// int
            public int cbSize;

            /// LPCWSTR->WCHAR*
            [System.Runtime.InteropServices.MarshalAsAttribute(System.Runtime.InteropServices.UnmanagedType.LPWStr)]
            public string lpszDocName;

            /// LPCWSTR->WCHAR*
            [System.Runtime.InteropServices.MarshalAsAttribute(System.Runtime.InteropServices.UnmanagedType.LPWStr)]
            public string lpszOutput;

            /// LPCWSTR->WCHAR*
            [System.Runtime.InteropServices.MarshalAsAttribute(System.Runtime.InteropServices.UnmanagedType.LPWStr)]
            public string lpszDatatype;

            /// DWORD->unsigned int
            public uint fwType;
        }

Coordinator
May 13, 2009 at 12:15 PM

>of course, you're right, but C# help us to try to keep the project development in scopes and timerange.

I think you missunderstood my comment... C++.NET is NOT C++... C++.NET provides the same advantages as C# but with a different syntax. Microsoft states that C# and C++.NET differ only 2% in syntax but I don't believe it ;-).

After all you can save much time with C++.NET for writing hook handlers. You can link C# libraries to the C++.NET library also link the C++.NET library to C# applications and so on. It should be no problem to outsource hook handlers with C++.NET and still write all other code in C#...

 

Coordinator
May 13, 2009 at 12:20 PM

The big advantage with C++.NET is that, as I said, you can merge unmanaged (usual C++ code) and managed code (C++.NET code). The NET compiler will take care of the unmanaged code sections and do all the work for you.

You don't have to think about how to express a call signature in managed code anymore, because with C++.NET you can just copy&paste the unmanaged code from the SDK headers and attach a method body in managed or unmanaged code or both, as you wish... All the unmanaged C++ structures in the SDK headers will be automatically translated to their NET equivalent by the NET compiler. So your work is done without thinking one second about marshalling, which can be a burden...

May 13, 2009 at 1:09 PM

>I think you missunderstood my comment...

Christoph, I am understood. :-) Yes, the marshalling is a really hard but there is a tools might help.

Anyway, sure, the way you mentioned is really resonable. BTW: Can you also provide some simple example next time you will have a chance? I.E. something like C++.NET/Hooks/Injectors with C#/Business Logic?

Thank you very much,

Igor