The packer-unpacker game is a game that I disapprove of, however this doesn't stop me from documenting a trick I remembered after reading the following blog post: Anti-Debugging Series - Part II.
To detect if a user mode debugger is attached to your process, do the following steps:
- Allocate some memory with HeapAlloc.
- Write something to the allocated memory
- Free the allocation with HeapFree.
- Check if the pattern you've written is still present at the given address. If a debugger is attached, it is overwritten with a fixed pattern when HeapFree gets called.
This is probably because attaching a debugger activates the "debug mode" in the heap allocator.
Whether a debugger is attached or not, the freed memory is modified. Tested with MSCV 2005 SP1/SDK 6.1.
ReplyDelete@Anonymous: if I recall correctly I've done the test with Delphi 7 under Windows XP SP2. Maybe Visual C initializes the heap allocator in a different way.
ReplyDeletePS. I just tried this with MSVC 2008 under Windows 7, and it always seems to overwrite the freed region (tried it with malloc/free and HeapAlloc/HeapFree with GetProcessHeap and HeapCreate). Maybe it is a new protection scheme introduced in Vista/7. Also, it is possible that the behavior changes depending on the allocation size/pattern.