Windows Kernel Debugger booster for Virtual Machines

Overview

Welcome to the VirtualKD homepage. This project allows speeding up (up to 45x) Windows kernel module debugging using VMWare and VirtualBox virtual machines. If you have ever noticed that the standard debugging over virtual COM port is irritatingly slow, this software is for you!

Normally, debugging over a virtual COM port involves the following scheme:
1. Windows uses a virtual COM port to exchange data with host
2. WinDBG/KD uses a named pipe provided by VM to communicate with debugging OS

The weakest link in this scheme is the virtual COM port, as its speed is limited to 115200 bits per second, i.e. around 10 KB/s. VirtualKD replaces the virtual COM port-related functionality, significantly raising the exchange rate (!irpfind produces ~450KB/s traffic on VirtualBox and ~150KB/s on VMWare, significantly outperforming standard ~10KB/s COM Port). Basically, VirtualKD consists of a Windows KD extension DLL and a patch to VMWare and VirtualBox processes, that create named pipes on host. This allows debugging Windows drivers on virtual machines using stadard tool set (WinDBG/KD), but with significantly lower response times.

New in version 2.1: Now fully compatible with Windows 7 (both host and guest), supports VirtualBox 3.x, and includes a fully automatic guest machine installer.

New in version 2.0: Added support for VirtualBox and fixed rare x64 loading bugs.

Attention! To use VirtualKD with VirtualBox 3.1.x, you need the following patch.

Features

  • Includes an automatic guest machine installer.
  • Significantly improves debugger-to-kernel connection speed.
  • Supports all modern VMWare and VirtualBox versions.
  • Supports all modern 32-bit and 64-bit Windows version from Windows XP to Windows 7.
  • Fixes truncated Driver Verifier load messages.
  • Automatically reconnects debugger to a VM without restarting, when a VM is restored from a snapshot, or restarted forcibly (via emulating the Kernel Restart notification).
  • Allows creating detailed HTML logs of sent and received KDCOM packets to perform comprehensive protocol analysis and simplify diagnosing problems.

Screenshots




Downloading

To download VirtualKD, see this page. The packages available to download include release build, debug build with PDB files for source-level debugging, and full source archive. Note that you need to download BazisLib library and VirtualBox sources to build VirtualKD.

Compatibility

VirtualKD supports both x86 and x64 guest operating systems and was tested with the following OSes:
  • Windows 7 64-bit
  • Windows Vista 32-bit
  • Windows Vista 64-bit
  • Windows XP 32-bit
  • Windows XP 64-bit
  • Windows Server 2003 32-bit
Normally, all modern versions of VMWare are supported. The following versions were tested:
  • VMWare Server 1.0.5
  • VMWare Server 2.0.0
  • VMWare Workstation 6.5.1

VirtualBox: the latest 3.0 branch is supported. You can rebuild VirtualKD from sources using VirtualBox headers to support other versions. The following version was tested:

  • VirtualBox 3.0.4
Note: VirtualKD was tested on both 32-bit and 64-bit host and guest systems.

Installation

If you are using VirtualBox, please do the following steps prior to doing anything with your VMs:

  • Rename the VBoxDD.dll file in your VirtualBox folder to VBoxDD0.dll.
  • Copy modified VBoxDD.dll from VirtualKD archive to VirtualBox directory.
  • Ensure that you have selected correct version (x86 or x64) of VBoxDD.dll
To install VirtualKD on a guest machine, simply copy the target/vminstall.exe to the virtual machine and run it. You will see a similar dialog box:

VirtualKD installer works with both pre-Vista operating systems (Windows XP, Windows Server 2003) and Vista-family (Windows Vista, Windows 7). After installing VirtualKD to the virtual machine, simply run the vmmon.exe on host machine and restart the Virtual Machine. VMMon will launch WinDbg with the corresponding parameters automatically.

Manual installation

If automatic installation has failed, there still are 2 manual ways of installing VirtualKD on a virtual machine:
A) Dynamic patching (not recommended).
Copy both KDPATCH.SYS and KDVM.DLL to SYSTEM32\DRIVERS directory of your virtual machine (yes, both DLL and SYS in the same place) and apply the KDPATCH.REG file (or manually create a legacy driver running KDPATCH.SYS). Then, perform the following sequence:
1) Start the virtual machine
2) BEFORE OS starts to load, ensure that VMMON.EXE is running (or use VMXPATCH.EXE) [VMWare only]
3) Start your virtual Windows in a normal debug mode with a virtual COM port
4) Ensure that WinDBG establishes connection (our driver is not involved now)
5) Let the OS boot
6) Wait for the KDPATCH.SYS driver to load (if you set it to manual mode, run "net start kdpatch"). This will redirect all debugging activity from the COM port simulated by VMWare to our fast debugging interface (VMWare needs to be already patched at this moment).
7) Use VMMON.EXE to start WinDbg, or do it manually(The pipe name for the fast debugging is the following: \\.\pipe\kd_<dirname>, when your VM is located at X:\something\<dirname>\filename.vmx).
8) You can close the first instance of WinDBG.
Alternatively, you can stop the KDPATCH.SYS driver at any moment to direct debug activity back to the VMWare COM port.

This method is useful if you want to play with the VirtualKD sources and to load different versions of KDVM.DLL without rebooting the virtual machine. Just start the patcher service, and the KDVM.DLL is used instead of KDCOM.DLL. Stop it and KDCOM.DLL gains control back.
B) Static patching (recommended)
1) Copy KDVM.DLL to SYSTEM32 directory of your guest OS. There should already exist KDCOM.DLL and KD1394.DLL files.
2) Open your boot.ini file. If you are using the COM debugging, you should have a line like this:

multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Microsoft Windows XP Professional" /noexecute=optin /fastdetect /DEBUG /DEBUGPORT=COM1
3) Change it to this:
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Microsoft Windows XP Professional" /noexecute=optin /fastdetect /DEBUG /DEBUGPORT=VM
Attention! On Windows Vista and later, use the following BCDEDIT command to enable KDVM.DLL manually:
bcdedit /set dbgtransport kdvm.dll

4) Reboot your virtual machine and wait for the OS selection dialog.
5) Ensure that VMMON.EXE is running (or use VMXPATCH.EXE) [VMWare only]
6) Use VMMON.EXE to start WinDbg, or do it manulally (use \\.\pipe\kd_<dirname> as pipe name (see above)).
7) Start your guest OS
The /DEBUGPORT=VM parameter in boot.ini instructs Windows to use the KDVM.DLL file as the packet layer driver for kernel debugging. This is more flexible than patching KDCOM.DLL, as it does not require booting in COM debug mode.

Note that if you used KDVMWare 1.x on your VMWare machine, you can continue using old driver and kernel DLL, or use install the new one. In that case, you will need to change /DEBUGPORT=VMWare to /DEBUGPORT=VM.

Patching

To ensure that all host-side code is loaded correctly, please keep VMMON.EXE running while you are using your virtual machines (it is really important to have it running when a VMWare VM is started). If you are using VirtualBox, VMMON.EXE is not required, as all patching is done manually by renaming VboxDD.dll and copying modified one on its place (see above).

If you have problems with VMMON.EXE, you can try simple console-based VMXPATCH.EXE.

Bugs

None known in v2.1. If you find any, write me an e-mail.

Troubleshooting

If something goes wrong, I strongly recommend you to download the sources and see everything in a debugger. Feel free to ask any questions on the SysProgs forum.

Tweaking

You can modify some parameters in registry under SOFTWARE\BazisSoft\KDVMWare\Patcher:

  • AllowPatchingAtTableStart. Set it to 0 if your VMWare crashes when being patched.

  • AllowReplacingFirstCommand. Set it to 1 if patching fails (and debug output indicates something like "0 free entries").

  • DefaultPatchingAtTableStart. You can try setting this to increase the performance (just a bit), but in can make VMWare crash on patching. Feel free to try ;) Additionally, you can set the WaitForOS to 0 in VirtualKD\Monitor to let the debugger be started immediately when a VM is detected (without waiting for OS to load).

Source structure

The information about source code structure and some hints for experimenting with it can be found at the following address: http://virtualkd.sysprogs.org/dox.

Change log

v1.0
Initial version
  
v1.1
Fixed handler loss after Virtual Machine reset
Fixed bug with VMWare hanging when no debugger is connected
Added patcher/packet level log displaying in VMMON
Added support for KDCLIENT.DLL unloading from VMMON
Added advanced statistics reporting to VMMON
Added permissive SECURITY_ATTRIBUTES to statistics-related objects to support non-admin VMWare instances
Added debugger command line customization
Added proxy mode support for debug VMMON builds
Added TraceAssist feature
Implemented buffered VMWare GuestRPC resulting in ~1.7x communication speedup
  
v1.2(Only host-side part changed)
 Fixed rare bug, when disconnecting debugger in the middle of a KdSendPacket() call caused hanging
 Reduced CPU usage from 100% to 0% when a VM is active and no debugger is connected
 Added workaround for truncated Driver Verifier messages
 Added on-demand packet logging feature for easy packet-level KD protocol analyzis
 Added API for detecting & patching VMs to KDCLIENT.DLL to support VisualDDK integration
 Pipe name is now generated based on VMWARE-VMX.EXE command line, instead of current directory (fix for rare "kd_" pipes)
 Added support for sending generated 'Target OS Shutdown' packet when VM is stopping to force debugger to disconnect
 Added ",reconnect" option to WinDbg command line, instructing it to reconnect a pipe, when it is closed
 (Due to 2 previous features, debugger does not need to be restarted when a VM is restored from a snapshot, while OS is running)
v2.0(KDVMWare was renamed to VirtualKD)
 Added support for VirtualBox
 Fixed bug with incorrect x64 VMWare patching
v2.1Added support for Windows 7 host and guest machines
 Added test signature to KDVM.DLL allowing it to run on Vista x64 in TESTSIGNING mode
 Added automatic guest machine installer
v2.2Improved integration with VisualDDK
 Added support for VMWARE-VMX-DEBUG.EXE etc.
v2.3Fixed memory leak in VMMON
 Added support for patching VMs from non-administrator accounts via IPC with VMMON
 Fixed bug when only first running VM was reported
 Added support for VirtualBox 3.1.x

Acknowledgments

I would like to thank the following people for making the creation of this tool possible: