Jungo WinDriver  
Official Documentation
Chapter 12: Understanding the Kernel PlugIn

This chapter provides a description of WinDriver's Kernel PlugIn feature.

12.1. Background

The creation of drivers in user mode imposes a fair amount of function call overhead from the kernel to user mode, which may cause performance to drop to an unacceptable level. In such cases, the Kernel PlugIn feature allows critical sections of the driver code to be moved to the kernel while keeping most of the code intact. Using WinDriver's Kernel PlugIn feature, your driver will operate without any degradation in performance. The Kernel PlugIn is available for Windows and Linux, and it is an integral part of the WinDriver PCI/ISA toolkit that does not require additional licensing.

Writing a Kernel PlugIn driver provides the following advantages over a standard OS kernel mode driver:

  • All the driver code is written and debugged in the user mode.
  • The code segments that are moved to the kernel mode remain essentially the same, and therefore typically no kernel debugging is needed.
  • The parts of the code that will run in the kernel through the Kernel PlugIn are platform independent, and therefore will run on every platform supported by WinDriver and the Kernel PlugIn. A standard kernel-mode driver will run only on the platform it was written for.

Not every performance problem requires you to write a Kernel PlugIn driver. Some performance problems can be solved in the user-mode driver by better utilization of the features that WinDriver provides.

12.2. Expected Performance

Since you can write your own interrupt handler in the kernel with the WinDriver Kernel PlugIn, you can expect to handle about 100,000 interrupts per second without missing any one of them.

12.3. Overview of the Development Process

Using the WinDriver Kernel PlugIn, you normally first develop and debug the driver in the user mode, using with standard WinDriver tools. After identifying the performance-critical parts of the code (such as the interrupt handling or access to I/O-mapped memory ranges), you can create a Kernel PlugIn driver, which runs in kernel mode, and drop the performance-critical portions of your code into the Kernel PlugIn driver, thus eliminating the calling overhead and context switches that occur when implementing the same tasks in the user mode.

This unique architecture allows the developer to start with quick and easy development in the user mode, and progress to performance-oriented code only where needed, thus saving development time and providing for virtually zero performance degradation.

12.4. The Kernel PlugIn Architecture

12.4.1. Architecture Overview

A driver written in user mode uses WinDriver's API (WDC_xxx and/or WD_xxx ) to access devices. If a certain function that was implemented in the user mode requires kernel performance (the interrupt handler, for example), that function is moved to the WinDriver Kernel PlugIn.

Generally it should be possible to move code that uses WDC_xxx / WD_xxx function calls from the user mode to the kernel without modification, since the same WinDriver API is supported both in the user mode and in the Kernel PlugIn.

Kernel PlugIn Architecture

12.4.2. WinDriver's Kernel and Kernel PlugIn Interaction

There are two types of interaction between the WinDriver kernel and the WinDriver Kernel PlugIn:

  • Interrupt handling: When WinDriver receives an interrupt, by default it will activate the caller's user-mode interrupt handler. However, if the interrupt was set to be handled by a Kernel PlugIn driver, then once WinDriver receives the interrupt, it activates the Kernel PlugIn driver's kernel-mode interrupt handler.

Your Kernel PlugIn interrupt handler could essentially consist of the same code that you wrote and debugged in the user-mode interrupt handler, before moving to the Kernel PlugIn, although some of the user-mode code should be modified. We recommend that you rewrite the interrupt acknowledgment and handling code in the Kernel PlugIn to utilize the flexibility offered by the Kernel PlugIn (see 12.5.5. Handling Interrupts in the Kernel PlugIn).

  • Message passing: To execute functions in kernel mode (such as I/O processing functions), the user-mode driver simply passes a message to the WinDriver Kernel PlugIn.

The message is mapped to a specific function, which is then executed in the kernel. This function can typically contain the same code as it did when it was written and debugged in user mode. You can also use messages to pass data from the user-mode application to the Kernel PlugIn driver.

12.4.3. Kernel PlugIn Components

At the end of your Kernel PlugIn development cycle, your driver will have the following components:

  • User-mode driver application (<application name>/.exe), written with the WDC_xxx / WD_xxx API.
  • The WinDriver kernel module — windrvr1630.sys/.o/.ko, depending on the operating system.
  • Kernel PlugIn driver (<Kernel PlugIn driver name>/.sys/.o/.ko/.kext), which was also written with the WDC_xxx / WD_xxx API, and contains the driver functionality that you have selected to bring down to the kernel level.

12.4.4. Kernel PlugIn Event Sequence

The following is a typical event sequence that covers all the functions that you can implement in your Kernel PlugIn.

12.4.4.1. Opening a Handle from the User Mode to a Kernel PlugIn Driver

Event

Windows loads your Kernel PlugIn driver.

This takes place at boot time, by dynamic loading, or as instructed by the registry.

Callback

Your KP_Init Kernel PlugIn routine is called.

KP_Init informs WinDriver of the name(s) of your KP_Open routine(s). WinDriver calls the relevant open routine when there is a user-mode request to open a handle to your Kernel PlugIn driver.

Event

Your user-mode driver application requests a handle to your Kernel PlugIn driver, by calling one of the following functions:

Callback

The relevant KP_Open Kernel PlugIn callback routine is called.

The KP_Open callback is used to inform WinDriver of the names of all the callback functions that you have implemented in your Kernel PlugIn driver, and to initiate the Kernel PlugIn driver, if needed.

12.4.4.2. Handling User-Mode Requests from the Kernel PlugIn

Event

Your application calls WDC_CallKerPlug(), or the low-level WD_KernelPlugInCall() function.

Your application calls WDC_CallKerPlug() / WD_KernelPlugInCall() to execute code in the kernel mode (in the Kernel PlugIn driver). The application passes a message to the Kernel PlugIn driver. The Kernel PlugIn driver will select the code to execute according to the message sent.

Callback

Your KP_Call Kernel PlugIn routine is called. KP_Call executes code according to the message passed to it from the user mode.

12.4.4.3. Interrupt Handling — Enable/Disable and High Interrupt Request Level Processing

Event

Your application calls WDC_IntEnable() with the fUseKP parameter set to TRUE (after having opened a handle to the Kernel PlugIn), or calls the low-level InterruptEnable() or WD_IntEnable() functions with a handle to a Kernel PlugIn driver (set in the hKernelPlugIn field of the WD_INTERRUPT structure passed to the function).

Callback

Your KP_IntEnable Kernel PlugIn routine is called. This function should contain any initialization required for your Kernel PlugIn interrupt handling.

Event

Your hardware creates an interrupt.

Callback

Your high-IRQL Kernel PlugIn interrupt handler routine — KP_IntAtIrql (legacy interrupts) or KP_IntAtIrqlMSI (MSI/MSI-X) — is called.

KP_IntAtIrql and KP_IntAtIrqlMSI run at a high priority, and therefore should perform only the basic interrupt handling, such as lowering the HW interrupt signal of level-sensitive interrupts to acknowledge the interrupt. If more interrupt processing is required, KP_IntAtIrql (legacy interrupts) or KP_IntAtIrqlMSI (MSI/MSI-X) can return TRUE in order to defer additional processing to the relevant deferred processing interrupt handler — KP_IntAtDpc or KP_IntAtDpcMSI .

Event

Your application calls WDC_IntDisable() , or the low-level InterruptDisable() or WD_IntDisable() functions, when the interrupts were previously enabled in the Kernel PlugIn (see the description of the interrupt enable event above).

Callback

Your KP_IntDisable Kernel PlugIn routine is called. This function should free any memory that was allocated by the KP_IntEnable callback .

12.4.4.4. Interrupt Handling — Deferred Procedure Calls Event/Callback Notes

Event

The Kernel PlugIn high-IRQL interrupt handler — KP_IntAtIrql or KP_IntAtIrqlMSI — returns TRUE. This informs WinDriver that additional interrupt processing is required as a Deferred Procedure Call (DPC) in the kernel.

Callback

Your Kernel PlugIn DPC interrupt handler — KP_IntAtDpc (legacy interrupts) or KP_IntAtDpcMSI (MSI/MSI-X) — is called. Processes the rest of the interrupt code, but at a lower priority than the high-IRQL interrupt handler.

Event

The DPC interrupt handler — KP_IntAtDpc or KP_IntAtDpcMSI — returns a value greater than 0. This informs WinDriver that additional usermode interrupt processing is required.

Callback

WD_IntWait() returns. Your user-mode interrupt handler routine is executed.

12.4.4.5. Plug-and-Play and Power Management Events

Event

Your application registers to receive Plug-and-Play and power management notifications using a Kernel PlugIn driver, by calling WDC_EventRegister() with the with the fUseKP parameter set to TRUE (after having opened the device with a Kernel PlugIn), or calls the low-level EventRegister() or WD_EventRegister() functions with a handle to a Kernel PlugIn driver (set in the hKernelPlugIn field of the WD_EVENT structure that is passed to the function).

Event

A Plug-and-Play or power management event (to which the application registered to listen) occurs.

Callback

Your KP_Event Kernel PlugIn routine is called. KP_Event receives information about the event that occurred and can proceed to handle it as needed.

Event

KP_Event returns TRUE. This informs WinDriver that the event also requires user-mode handling.

Callback

WD_IntWait() returns. Your user-mode event handler routine is executed.

12.5. How Does Kernel PlugIn Work?

The following sections take you through the development cycle of a Kernel PlugIn driver.

It is recommended that you first write and debug your entire driver code in the user mode. Then, if you encounter performance problems or require greater flexibility, port portions of your code to a Kernel PlugIn driver.

12.5.1. Minimal Requirements for Creating a Kernel PlugIn Driver

To build a Kernel PlugIn driver you need the following tools:

  • On Windows:

The Windows Driver Kit (WDK), including its C build tools. The WDK is available as part of a Microsoft Development Network (MSDN) subscription, or from Microsoft Connect. For more information, refer to Microsoft's Windows Driver Kit (WDK) page.

  • On Linux:

GCC, gmake or make.

While this is not a minimal requirement, when developing a Kernel PlugIn driver it is highly recommended that you use two computers: set up one computer as your host platform and the other as your target platform. The host computer is the computer on which you develop your driver and the target computer is the computer on which you run and test the driver you develop.

12.5.2. Kernel PlugIn Implementation

12.5.2.1. Before You Begin

The functions described in this section are callback functions, implemented in the Kernel PlugIn driver, which are called when their calling event occurs — see 12.4.4. Kernel PlugIn Event Sequence for details. For example, KP_Init is the callback function that is called when the driver is loaded.

The name of your driver is given in KP_Init. The Kernel PlugIn driver's implementation of this callback must be named KP_Init. The names of the other Kernel PlugIn callback functions (which are passed to KP_Init) are left to the discretion of the driver developer.

It is the convention of this reference guide to refer to these callbacks using the format KP_<Functionality> — for example, KP_Open.

When generating Kernel PlugIn code with the DriverWizard, the names of the callback functions (apart from KP_Init) conform to the following format: KP_<Driver Name>_<Functionality>. For example, if you named your project MyDevice, the name of your Kernel PlugIn KP_Call callback will be KP_MyDevice_Call.

12.5.2.2. Write Your KP_Init Function

Your KP_Init function should be of the following prototype: BOOL __cdecl KP_Init (KP_INIT *kpInit);

This function is called once, when the driver is loaded. The function should fill the received KP_INIT structure with the name of the Kernel PlugIn driver, the name of the WinDriver Kernel PlugIn driver library, and the driver's KP_Open callback(s) (see example in WinDriver/samples/c/pci_diag/kp_pci/kp_pci.c).

The name that you select for your Kernel PlugIn driver — by setting it in the cDriverName field of the KP_INIT structure that is passed to KP_Init — should be the name of the driver that you wish to create; i.e., if you are creating a driver called XXX.sys, you should set the name "XXX" in the cDriverName field of the KP_INIT structure.

You should verify that the driver name that is used when opening a handle to the Kernel PlugIn driver in the user mode (see 13.5. Open a Handle to the Kernel PlugIn) — in the pKPOpenData parameter of the WDC_KernelPlugInOpen() or WDC_PciDeviceOpen()/WDC_IsaDeviceOpen() (PCI / ISA ) functions, or in the pcDriverName field of the pKernelPlugIn parameter passed to the low-level WD_KernelPlugInOpen() function — is identical to the driver name that was set in the cDriverName field of the KP_INIT structure that is passed to KP_Init.

The best way to implement this is to define the driver name in a header file that is shared by the user-mode application and the Kernel PlugIn driver and use the defined value in all relevant locations.

From the KP_PCI sample (WinDriver/samples/c/pci_diag/kp_pci/kp_pci.c):

/* KP_Init is called when the Kernel PlugIn driver is loaded. This function sets the name of the Kernel PlugIn driver and the driver's open callback function(s). */

BOOL __cdecl KP_Init(KP_INIT *kpInit)
{
/* Verify that the version of the WinDriver Kernel PlugIn library
is identical to that of the windrvr.h and wd_kp.h files */
if (WD_VER != kpInit->dwVerWD)
{
/* Rebuild your Kernel PlugIn driver project with the compatible
version of the WinDriver Kernel PlugIn library (kp_nt<version>.lib)
and windrvr.h and wd_kp.h files */
return FALSE;
}
kpInit->funcOpen = KP_PCI_Open;
kpInit->funcOpen_32_64 = KP_PCI_VIRT_Open_32_64;
strcpy (kpInit->cDriverName, KP_PCI_DRIVER_NAME);
return TRUE;
}
BOOL __cdecl KP_PCI_Open(KP_OPEN_CALL *kpOpenCall, HANDLE hWD, PVOID pOpenData, PVOID *ppDrvContext)
Kernel PlugIn open function.
Definition: kp_pci.c:154
#define TRUE
Definition: kpstdlib.h:264
#define FALSE
Definition: kpstdlib.h:260
char *__cdecl strcpy(char *s1, const char *s2)
CHAR cDriverName[WD_MAX_KP_NAME_LENGTH]
return the device driver name
Definition: wd_kp.h:69
DWORD dwVerWD
version of the WinDriver Kernel PlugIn library
Definition: wd_kp.h:68
KP_FUNC_OPEN funcOpen
returns the KP_Open function
Definition: wd_kp.h:71
KP_FUNC_OPEN funcOpen_32_64
returns the KP_Open function for 32 bit app with 64 bit KP.
Definition: wd_kp.h:72
Definition: wd_kp.h:67
BOOL __cdecl KP_Init(KP_INIT *kpInit)
You must define a KP_Init() function to link to the device driver.
Definition: kp_pci.c:65
#define WD_VER
Definition: wd_ver.h:25

Note that the driver name in the sample is set using a preprocessor definition. This definition is found in the WinDriver/samples/c/pci_diag/pci_lib.h header file, which is shared by the pci_diag user-mode application and the KP_PCI Kernel PlugIn driver:

/* Kernel PlugIn driver name (should be no more than 8 characters) */
#define KP_PCI_DRIVER_NAME "KP_PCI"

12.5.2.3. Write Your KP_Open Function(s)

You can implement either one or two KP_Open functions, depending on your target configuration. The KP_Open function(s) should be of the following prototype:

BOOL __cdecl KP_Open (
KP_OPEN_CALL *kpOpenCall,
HANDLE hWD,
PVOID pOpenData,
PVOID *ppDrvContext);

This callback is called when opening a handle to the Kernel PlugIn driver from the user mode — i.e., when WD_KernelPlugInOpen() is called, either directly or via WDC_KernelPlugInOpen() / ISA ), as explained in 13.5. Open a Handle to the Kernel PlugIn.

In the KP_Open function, define the callbacks that you wish to implement in the Kernel PlugIn.

The following is a list of the callbacks that can be implemented:

Callback Description
KP_Close Called when the WD_KernelPlugInClose() function is called from the user mode — either directly, or via one of the high-level WDC_PciDeviceClose() / WDC_IsaDeviceClose() functions (PCI / ISA ) when called for a device that contains an open Kernel PlugIn handle (see 13.5. Open a Handle to the Kernel PlugIn).
KP_Call Called when the user-mode application calls the WDC_CallKerPlug() function or the low-level WD_KernelPlugInCall() function, which is called by the wrapper WDC_CallKerPlug() function. This function implements a Kernel PlugIn message handler.
KP_IntEnable Called when the user-mode application enables Kernel PlugIn interrupts, by calling WDC_IntEnable() with the fUseKP parameter set to TRUE (after having opened a Kernel PlugIn handle), or by calling the low-level InterruptEnable() or WD_IntEnable() functions with a handle to a Kernel PlugIn driver (set in the hKernelPlugIn field of the WD_INTERRUPT structure that is passed to the function). This function should contain any initialization required for your Kernel PlugIn interrupt handling.
KP_IntDisable Called when the user-mode application calls WDC_IntDisable(), or the low-level InterruptDisable() or WD_IntDisable() functions, if the interrupts were previously enabled with a Kernel PlugIn driver (see the description of KP_IntEnable above). This function should free any memory that was allocated by the KP_IntEnable callback.
KP_IntAtIrql Called when WinDriver receives a legacy interrupt, provided the received interrupt was enabled with a handle to the Kernel PlugIn. This is the function that will handle your legacy interrupt in the kernel mode. The function runs at high interrupt request level. Additional deferred processing of the interrupt can be performed in KP_IntAtDpc and also in the user mode (see below).
KP_IntAtDpc Called if the KP_IntAtIrql callback has requested deferred handling of a legacy interrupt by returning TRUE. This function should include lower-priority kernel-mode interrupt handler code. The return value of this function determines the amount of times that the application's usermode interrupt handler routine will be invoked (if at all).
KP_IntAtIrqlMSI Called when WinDriver receives an MSI or MSI-X, provided MSI/MSI-X was enabled for the received interrupt with a handle to the Kernel PlugIn. This is the function that will handle your MSI/MSI-X in the kernel mode. The function runs at high interrupt request level. Additional deferred processing of the interrupt can be performed in KP_IntAtDpcMSI and also in the user mode (see below).
KP_IntAtDpcMSI Called if the KP_IntAtIrqlMSI callback has requested deferred handling of an MSI/MSI-X interrupt by returning TRUE. This function should include lower-priority kernel-mode MSI/MSI-X handler code. The return value of this function determines the amount of times that the application's usermode interrupt handler routine will be invoked (if at all).
KP_Event Called when a Plug-and-Play or power management event occurs, provided the user-mode application previously registered to receive notifications for this event in the Kernel PlugIn by calling WDC_EventRegister() with the fUseKP parameter set to TRUE (after having opened a Kernel PlugIn handle), or by calling the low-level EventRegister() or WD_EventRegister() functions with a handle to a Kernel PlugIn driver (set in the hKernelPlugIn field of the WD_EVENT structure that is passed to the function).

In addition to defining the Kernel PlugIn callback functions, you can implement code to perform any required initialization for the Kernel PlugIn in your KP_Open callback(s). In the sample KP_PCI driver and in the generated DriverWizard Kernel PlugIn driver, for example, the Kernel PlugIn open callbacks also call the shared library's initialization function and allocate memory for the Kernel PlugIn driver context, which is then used to store the device information that was passed to the function from the user mode.

From the KP_PCI sample (WinDriver/samples/c/pci_diag/kp_pci/kp_pci.c):

/* KP_PCI_Open is called when WD_KernelPlugInOpen() is called from the user mode. pDrvContext will be passed to the rest of the Kernel PlugIn callback functions. */

BOOL __cdecl KP_PCI_Open(KP_OPEN_CALL *kpOpenCall, HANDLE hWD, PVOID pOpenData,
PVOID *ppDrvContext)
{
PCI_DEV_ADDR_DESC *pDevAddrDesc;
WDC_ADDR_DESC *pAddrDesc;
DWORD dwSize;
DWORD dwStatus;
/* Initialize the PCI library */
dwStatus = PCI_LibInit();
if (WD_STATUS_SUCCESS != dwStatus)
{
KP_PCI_Err("KP_PCI_Open: Failed to initialize the PCI library: %s",
PCI_GetLastErr());
return FALSE;
}
KP_PCI_Trace("KP_PCI_Open entered. PCI library initialized.\n");
kpOpenCall->funcClose = KP_PCI_Close;
kpOpenCall->funcCall = KP_PCI_Call;
kpOpenCall->funcEvent = KP_PCI_Event;
/* Create a copy of device information in the driver context */
dwSize = sizeof(PCI_DEV_ADDR_DESC);
pDevAddrDesc = malloc(dwSize);
if (!pDevAddrDesc)
goto malloc_error;
COPY_FROM_USER(pDevAddrDesc, pOpenData, dwSize);
dwSize = sizeof(WDC_ADDR_DESC) * pDevAddrDesc->dwNumAddrSpaces;
pAddrDesc = malloc(dwSize);
if (!pAddrDesc)
goto malloc_error;
COPY_FROM_USER(pAddrDesc, pDevAddrDesc->pAddrDesc, dwSize);
pDevAddrDesc->pAddrDesc = pAddrDesc;
*ppDrvContext = pDevAddrDesc;
KP_PCI_Trace("KP_PCI_Open: Kernel PlugIn driver opened successfully\n");
return TRUE;
malloc_error:
KP_PCI_Err("KP_PCI_Open: Failed allocating %ld bytes\n", dwSize);
if (pDevAddrDesc)
free(pDevAddrDesc);
PCI_LibUninit();
return FALSE;
}
BOOL __cdecl KP_PCI_IntAtIrqlMSI(PVOID pIntContext, ULONG dwLastMessage, DWORD dwReserved)
High-priority Message-Signaled Interrupts (MSI) / Extended Message-Signaled Interrupts (MSI-X) handle...
Definition: kp_pci.c:643
DWORD __cdecl KP_PCI_IntAtDpcMSI(PVOID pIntContext, DWORD dwCount, ULONG dwLastMessage, DWORD dwReserved)
Deferred processing Message-Signaled Interrupts (MSI) / Extended Message-Signaled Interrupts (MSI-X) ...
Definition: kp_pci.c:688
DWORD __cdecl KP_PCI_IntAtDpc(PVOID pIntContext, DWORD dwCount)
Deferred processing legacy interrupt handler routine.
Definition: kp_pci.c:602
BOOL __cdecl KP_PCI_IntEnable(PVOID pDrvContext, WD_KERNEL_PLUGIN_CALL *kpCall, PVOID *ppIntContext)
Called when WDC_IntEnable() / WD_IntEnable() is called from the user mode with a Kernel PlugIn handle...
Definition: kp_pci.c:428
BOOL __cdecl KP_PCI_IntAtIrql(PVOID pIntContext, BOOL *pfIsMyInterrupt)
High-priority legacy interrupt handler routine, which is run at high interrupt request level.
Definition: kp_pci.c:495
void __cdecl KP_PCI_Close(PVOID pDrvContext)
Called when WD_KernelPlugInClose() (see the WinDriver PCI Low-Level API Reference) is called from use...
Definition: kp_pci.c:326
void __cdecl KP_PCI_Call(PVOID pDrvContext, WD_KERNEL_PLUGIN_CALL *kpCall)
Called when the user-mode application calls WDC_CallKerPlug() (or the low-level WD_KernelPlugInCall()...
Definition: kp_pci.c:377
void __cdecl KP_PCI_IntDisable(PVOID pIntContext)
Called when WDC_IntDisable() / WD_IntDisable() is called from the user mode for interrupts that were ...
Definition: kp_pci.c:456
BOOL __cdecl KP_PCI_Event(PVOID pDrvContext, WD_EVENT *wd_event)
Called when a Plug-and-Play or power management event for the device is received, provided the user-m...
Definition: kp_pci.c:712
void *__cdecl malloc(unsigned long size)
void __cdecl free(void *buf)
KP_FUNC_EVENT funcEvent
Definition: wd_kp.h:58
KP_FUNC_CLOSE funcClose
Definition: wd_kp.h:50
KP_FUNC_INT_AT_IRQL funcIntAtIrql
Definition: wd_kp.h:54
KP_FUNC_INT_AT_DPC_MSI funcIntAtDpcMSI
Definition: wd_kp.h:57
KP_FUNC_INT_ENABLE funcIntEnable
Definition: wd_kp.h:52
KP_FUNC_INT_AT_IRQL_MSI funcIntAtIrqlMSI
Definition: wd_kp.h:56
KP_FUNC_INT_AT_DPC funcIntAtDpc
Definition: wd_kp.h:55
KP_FUNC_CALL funcCall
Definition: wd_kp.h:51
KP_FUNC_INT_DISABLE funcIntDisable
Definition: wd_kp.h:53
Address space information struct.
Definition: wdc_defs.h:27
@ WD_STATUS_SUCCESS
[0] Operation completed successfully
Definition: windrvr.h:1066

The KP_PCI sample also defines a similar KP_PCI_Open_32_64 callback, for use when opening a handle to a 64-bit Kernel PlugIn from a 32-bit application.

12.5.2.4. Write the Remaining PlugIn Callbacks

Implement the remaining Kernel PlugIn routines that you wish to use (such as the KP_Intxxx functions — for handling interrupts, or KP_Event — for handling Plug-and-Play and power management events).

12.5.3. Sample/Generated Kernel PlugIn Driver Code Overview

You can use DriverWizard to generate a skeletal Kernel PlugIn driver for your device, and use the generated code as the basis for your Kernel PlugIn driver development (recommended); alternatively, you can use one of the Kernel PlugIn WinDriver samples as the basis for your Kernel PlugIn development.

The Kernel PlugIn documentation in this manual focuses on the generated DriverWizard code, and the generic PCI Kernel PlugIn sample — KP_PCI, located in theWinDriver/samples/c/pci_diag/kp_pci directory.

If you are using the a PCI Express card with the Xilinx Bus Master DMA (BMD) design, you can also use the KP_BMD Kernel PlugIn sample as the basis for your development; the WinDriver/samples/c/xilinx/bmd_design directory contains all the relevant sample files — see the Xilinx BMD Kernel PlugIn directory structure note at the end of 12.5.4.1. pci_diag and kp_pci Sample Directories.

The Kernel PlugIn driver is not a standalone module. It requires a user-mode application that initiates the communication with the driver. A relevant application will be generated for your driver when using DriverWizard to generate Kernel PlugIn code. The pci_diag application (found under the WinDriver/samples/c/pci_diag directory) communicates with the sample KP_PCI driver.

Both the KP_PCI sample and the wizard-generated code demonstrate communication between a user-mode application (pci_diag / xxx_diag — where xxx is the name you selected for your generated driver project) and a Kernel PlugIn driver (kp_pci.sys/.o/.ko/.kext / kp_xxx.sys/.o/.ko/.kext — depending on the OS).

The sample/generated code demonstrates how to pass data to the Kernel PlugIn's KP_Open function, and how to use this function to allocate and store a global Kernel PlugIn driver context that can be used by other functions in the Kernel PlugIn.

The sample/generated Kernel PlugIn code implements a message for getting the driver's version number, in order to demonstrate how to initiate specific functionality in the Kernel PlugIn from the user mode and how to pass data between the Kernel PlugIn driver and a user-mode WinDriver application via messages.

The sample/generated code also demonstrates how to handle interrupts in the Kernel PlugIn. The Kernel PlugIn implements an interrupt counter and interrupt handlers, including deferred processing interrupt handling, which is used to notify the user-mode application of the arrival of every fifth incoming interrupt.

The KP_PCI sample's KP_IntAtIrql() functions demonstrate legacy level-sensitive PCI interrupt handling. As indicated in the comments of the sample KP_IntAtIrql function, you will need to modify this function in order to implement the correct code for acknowledging the interrupt on your specific device, since interrupt acknowledgment is hardware-specific. The sample KP_IntAtIrqlMSI() and KP_IntAtDpcMSI() functions demonstrate handling of Message-Signaled Interrupts (MSI) and Extended Message-Signaled Interrupts (MSI-X) (see detailed information in 10.4. Single Root I/O Virtualization (SR-IOV)).

The generated DriverWizard code will include sample interrupt handler code for the selected device (PCI/ISA). The generated KP_IntAtIrql function will include code to implement any interrupt transfer commands defined in the wizard (by assigning registers read/write commands to the card's interrupt in the Interrupt tab).

For legacy PCI interrupts, which need to be acknowledged in the kernel when the interrupt is received (see 10.4. Single Root I/O Virtualization (SR-IOV)), it is recommended that you use the wizard to define the commands for acknowledging (clearing) the interrupt, before generating the Kernel PlugIn code, so that the generated code will already include the required code for executing the commands you defined.

It is also recommended that you prepare such transfer commands when handling interrupts for hardware that supports MSI/MSI-X, in case enabling of MSI/MSI-X fails and the interrupt handling defaults to using level-sensitive interrupts (if supported by the hardware).

⚠ Attention

‍Memory allocated for the transfer commands must remain available until the interrupts are disabled .

In addition, the sample/generated code demonstrates how to receive notifications of Plug-and-Play and power management events in the Kernel PlugIn.

We recommend that you build and run the sample/generated Kernel PlugIn project (and corresponding user-mode application) "as-is" before modifying the code or writing your own Kernel PlugIn driver. Note, however, that you will need to modify or remove the hardware-specific transfer commands in the sample's KP_IntAtIrql function, as explained above.

12.5.4. Kernel PlugIn Sample/Generated Code Directory Structure

12.5.4.1. pci_diag and kp_pci Sample Directories

The KP_PCI Kernel PlugIn sample code is implemented in the kp_pci.c file. This sample driver is part of the WinDriver PCI diagnostics sample — pci_diag — which contains, in addition to the KP_PCI driver, a user-mode application that communicates with the driver (pci_diag) and a shared library that includes APIs that can be utilized by both the user-mode application and the Kernel PlugIn driver. The source files for this sample are implemented in C.

Following is an outline of the files found in the WinDriver/samples/c/pci_diag directory:

  • kp_pci — Contains the KP_PCI Kernel PlugIn driver files:
    • kp_pci.c: The source code of the KP_PCI driver.
    • Project and/or make files and related files for building the Kernel PlugIn driver.

The Windows project/make files are located in subdirectories for the target development environment (msdev_<version>/win_gcc) under x86 (32-bit) and amd64 (64-bit) directories and arm64 (for ARM64 platform).
The Linux makefile is generated using a configure script, located directly under the kp_pci directory.

  • A pre-compiled version of the KP_PCI Kernel PlugIn driver for the target OS:

Windows x86 32-bit: WINNT.i386\kp_pci.sys — a 32-bit version of the driver. Windows x64: WINNT.x86_64\kp_pci.sys — a 64-bit version of the driver. Linux: There is no pre-compiled version of the driver for Linux, since Linux kernel modules must be compiled with the header files from the kernel version installed on the target — see 15.3. Linux Driver Distribution).

  • pci_lib.c: Implementation of a library for accessing PCI devices using WinDriver's WDC API. The library's API is used both by the user-mode application (pci_diag.c) and by the Kernel PlugIn driver (kp_pci.c).
  • pci_lib.h: Header file, which provides the interface for the pci_lib library.
  • pci_diag.c: Implementation of a sample diagnostics user-mode console (CUI) application, which demonstrates communication with a PCI device using the pci_lib and WDC libraries.

The sample also demonstrates how to communicate with a Kernel PlugIn driver from a user mode WinDriver application. By default, the sample attempts to open the selected PCI device with a handle to the KP_PCI Kernel PlugIn driver. If successful, the sample demonstrates how to interact with a Kernel PlugIn driver, as detailed in 12.5.3. Sample/Generated Kernel PlugIn Driver Code Overview. If the application fails to open a handle to the Kernel PlugIn driver, all communication with the device is performed from the user mode.

  • pci.inf (Windows): A sample WinDriver PCI INF file for Windows. NOTE: To use this file, change the vendor and device IDs in the file to comply with those of your specific device.

To use Message-Signaled Interrupts (MSI) or Extended Message-Signaled Interrupts (MSI-X) on Windows (for PCI cards that support MSI/MSI-X) you will need to modify or replace the sample INF file so that your INF file includes specific MSI information; otherwise WinDriver will attempt to use legacy level-sensitive interrupt handling for your card, as explained in 10.1.7.1. Windows MSI/MSI-X Device INF Files.

  • Project and/or make files for building the pci_diag user-mode application.

The Windows project/make files are located in subdirectories for the target development environment (msdev_<version>/win_gcc) under x86 (32-bit) and amd64 (64-bit) directories. The msdev_<version> MS Visual Studio directories also include solution files for building both the Kernel PlugIn driver and user-mode application projects. The Linux makefile is located under a LINUX subdirectory.

  • A pre-compiled version of the user-mode application (pci_diag) for your target operating system:

Windows: WIN32\pci_diag.exe Linux: LINUX/pci_diag

  • files.txt: A list of the sample pci_diag files.
  • readme.pdf: An overview of the sample Kernel PlugIn driver and user-mode application and instructions for building and testing the code.

12.5.4.2. Xilinx BMD Kernel PlugIn Directory Structure

The structure of the sample directory for PCI Express cards with the Xilinx Bus Master DMA (BMD) design — WinDriver/samples/c/xilinx/bmd_design — is similar to that of the generic PCI sample's pci_diag directory, except for the following issues: the bmd_diag user-mode application files are located under a diag subdirectory, and the kp subdirectory, which contains the Kernel PlugIn driver's (KP_BMD) source files, currently has make files only for Windows.

12.5.4.3. The Generated DriverWizard Kernel PlugIn Directory

The generated DriverWizard Kernel PlugIn code for your device will include a kernel-mode Kernel PlugIn project and a user-mode application that communicates with it. As opposed to the generic KP_PCI and pci_diag sample, the wizard-generated code will utilize the resources information detected and/or defined for your specific device, as well as any device-specific information that you define in the wizard before generating the code.

As indicated in 12.5.3. Sample/Generated Kernel PlugIn Driver Code Overview, when using the driver to handle legacy PCI interrupts, it is highly recommended that you define the registers that need to be read/written in order to acknowledge the interrupt, and set up the relevant read/write commands from/to these registers in DriverWizard, before generating the code, thus enabling the generated interrupt handler code to utilize the hardware-specific information that you defined. It is also recommended that you prepare such transfer commands when handling interrupts for hardware that supports MSI/MSI-X, in case enabling of MSI/MSI-X fails and the interrupt handling defaults to using level-sensitive interrupts (if supported by the hardware).

⚠ Attention

‍Memory allocated for the transfer commands must remain available until the interrupts are disabled .

Following is an outline of the generated DriverWizard files when selecting to generate Kernel PlugIn code (where xxx signifies the name that you selected for the driver when generating the code).

The outline below relates to the generated C code, but on Windows you can also generate similar C# code, which includes a C Kernel PlugIn driver (since kernel-mode drivers cannot be implemented in C#), a .NET C# library, and a C# user-mode application that communicates with the Kernel PlugIn driver.

  • kermode — Contains the KP_XXX Kernel PlugIn driver files:
    • kp_xxx.c: The source code of the KP_XXX driver.
    • Project and/or make files and related files for building the Kernel PlugIn driver.

The Windows project/make files are located in subdirectories for the target development environment (msdev_<version>/win_gcc) under x86 (32-bit) and amd64 (64-bit) directories. The Linux makefile is generated using a configure script, located in a linux subdirectory.

  • xxx_lib.c: Implementation of a library for accessing your device using WinDriver's WDC API. The library's API is used both by the user-mode application (xxx_diag) and by the Kernel PlugIn driver (KP_XXX).
  • xxx_lib.h: Header file, which provides the interface for the xxx_lib library.
  • xxx_diag.c: Implementation of a sample diagnostics user-mode console (CUI) application, which demonstrates communication your device using the xxx_lib and WDC libraries.

The application also demonstrates how to communicate with a Kernel PlugIn driver from a user-mode WinDriver application. By default, the application attempts to open your device with a handle to the KP_XXX Kernel PlugIn driver. If successful, the application demonstrates how to interact with a Kernel PlugIn driver, as detailed in 12.5.3. Sample/Generated Kernel PlugIn Driver Code Overview. If the application fails to open a handle to the Kernel PlugIn driver, all communication with the device is performed from the user mode.

  • Project and/or make files for building the xxx_diag user-mode application.

The Windows project/make files are located in subdirectories for the target development environment (msdev_<version>/win_gcc) under x86 (32-bit) and amd64 (64-bit) directories. The msdev_<version> MS Visual Studio directories also include solution files for building both the Kernel PlugIn driver and user-mode application projects. The Linux makefile is located in a linux subdirectory.

  • xxx_files.txt: A list of the generated files and instructions for building the code.
  • xxx.inf (Windows): A WinDriver INF file for your device. This file is required only when creating a Windows driver for a Plug-and-Play device, such as PCI.

12.5.5. Handling Interrupts in the Kernel PlugIn

Interrupts will be handled in the Kernel PlugIn driver, if enabled, using a Kernel PlugIn driver, as explained below (see 12.5.5.2. Interrupt Handling in the Kernel (Using the Kernel PlugIn)).

If Kernel PlugIn interrupts were enabled, when WinDriver receives a hardware interrupt, it calls the Kernel PlugIn driver's high-IRQL handler — KP_IntAtIrql (legacy interrupts) or KP_IntAtIrqlMSI (MSI/MSI-X). If the high-IRQL handler returns TRUE, the relevant deferred Kernel PlugIn interrupt handler — KP_IntAtDpc (legacy interrupts) or KP_IntAtDpcMSI (MSI/MSI-X) — will be called after the high-IRQL handler completes its processing and returns. The return value of the DPC function determines how many times (if at all) the user-mode interrupt handler routine will be executed.

In the KP_PCI sample, for example, the Kernel PlugIn interrupt handler code counts five interrupts, and notifies the user mode on every fifth interrupt; thus WD_IntWait() will return on only one out of every five incoming interrupts in the user mode.

The high-IRQL handler — KP_IntAtIrql or KP_IntAtIrqlMSI — returns TRUE every five interrupts to activate the DPC handler — KP_IntAtDpc or KP_IntAtDpcMSI — and the DPC function returns the number of accumulated DPC calls from the high-IRQL handler. As a result, the user-mode interrupt handler will be executed once for every 5 interrupts.

12.5.5.1. Interrupt Handling in the User Mode (Without the Kernel PlugIn)

If the Kernel PlugIn interrupt handle is not enabled, then each incoming interrupt will cause WD_IntWait() to return, and your user-mode interrupt handler routine will be invoked once WinDriver completes the kernel processing of the interrupts (mainly executing the interrupt transfer commands passed in the call to WDC_IntEnable() or the low-level InterruptEnable() or WD_IntEnable() functions) — see Figure below.

Interrupt Handling Without Kernel PlugIn

12.5.5.2. Interrupt Handling in the Kernel (Using the Kernel PlugIn)

To have the interrupts handled by the Kernel PlugIn, the user-mode application shoul open a handle to a Kernel PlugIn driver (as explained in 13.5. Open a Handle to the Kernel PlugIn), and then call WDC_IntEnable() with the fUseKP parameter set to TRUE.

If your are not using the WDC_xxx API, your application should pass a handle to the Kernel PlugIn driver to the WD_IntEnable() function or the wrapper InterruptEnable() function (which calls WD_IntEnable() and WD_IntWait()). This enables the Kernel PlugIn interrupt handler. The Kernel PlugIn handle is passed within the hKernelPlugIn field of the WD_INTERRUPT structure that is passed to the functions.

Interrupt Handling With the Kernel PlugIn

When calling WDC_IntEnable() / InterruptEnable() / WD_IntEnable() to enable interrupts in the Kernel PlugIn, your Kernel PlugIn's KP_IntEnable callback function is activated. In this function you can set the interrupt context that will be passed to the Kernel PlugIn interrupt handlers, as well as write to the device to actually enable the interrupts in the hardware and implement any other code required in order to correctly enable your device's interrupts.

If the Kernel PlugIn interrupt handler is enabled, then the relevant high-IRQL handler, based on the type of interrupt that was enabled — KP_IntAtIrql (legacy interrupts) or KP_IntAtIrqlMSI (MSI/MSI-X) — will be called for each incoming interrupt. The code in the high-IRQL handler is executed at high interrupt request level. While this code is running, the system is halted, i.e., there will be no context switches and no lower-priority interrupts will be handled.

Code running at high IRQL is limited in the following ways:

  • It may only access non-pageable memory.
  • It may only call the following functions (or wrapper functions that call these functions):
  • WDC_xxx() read/write address or configuration space functions.
  • WDC_MultiTransfer() , or the low-level WD_Transfer(), WD_MultiTransfer(), or WD_DebugAdd() functions.
  • Specific kernel OS functions (such as WDK functions) that can be called from high interrupt request level. Note that the use of such functions may break the code's portability to other operating systems.
  • It may not call malloc(), free(), or any WDC_xxx or WD_xxx API other than those listed above.

Because of the aforementioned limitations, the code in the high-IRQL handler (KP_IntAtIrql or KP_IntAtIrqlMSI ) should be kept to a minimum, such as acknowledgment (clearing) of level-sensitive interrupts. Other code that you want to run in the interrupt handler should be implemented in the DPC function (KP_IntAtDpc or KP_IntAtDpcMSI ), which runs at a deferred interrupt level and does not face the same limitations as the high-IRQL handlers. The DPC function is called after its matching high-IRQL function returns, provided the high-IRQL handler returns TRUE.

You can also leave some additional interrupt handling to the user mode. The return value of your DPC function — KP_IntAtDpc() — determines the amount of times (if any) that your user-mode interrupt handler routine will be called after the kernel-mode interrupt processing is completed.

12.5.6. Message Passing

The WinDriver architecture enables a kernel-mode function to be activated from the user mode by passing a message from the user mode to the Kernel PlugIn driver using WDC_CallKerPlug() or the low-level WD_KernelPlugInCall() function.

The messages are defined by the developer in a header file that is common to both the user-mode and kernel-mode plugin parts of the driver. In the pci_diag KP_PCI sample and the generated DriverWizard code, the messages are defined in the shared library header file — pci_lib.h in the sample or xxx_lib.h in the generated code.

Upon receiving the message from the user mode, WinDriver will execute the KP_Call Kernel PlugIn callback function, which identifies the message that has been received and executes the relevant code for this message (as implemented in the Kernel PlugIn).

The sample/generated Kernel PlugIn code implement a message for getting the driver's version in order to demonstrate Kernel PlugIn data passing. The code that sets the version number in KP_Call is executed in the Kernel PlugIn whenever the Kernel PlugIn receives a relevant message from the user-mode application.

You can see the definition of the message in the shared pci_lib.h / xxx_lib.h shared header file. The user-mode application (pci_diag.exe / xxx_diag.exe) sends the message to the Kernel PlugIn driver via the WDC_CallKerPlug() function .

12.6. FAQ

12.6.1. Why does my WD_KernelPlugInOpen() call fail?

There are several reasons why the call to WD_KernelPlugInOpen() might fail:

  • You did not set up the parameters correctly in the call to WD_KernelPlugInOpen(). Please check this carefully.
  • You did not set up the driver name correctly in all locations in the code. Please verify that you are using the correct Kernel PlugIn driver name in all the relevant locations in the code. Look specifically at the Kernel PlugIn KP_Init() implementation and at kernelPlugIn.pcDriverName in the user mode project. Please indicate the driver name in capital letters and without the file extension (*.sys/.ko).
  • You did not install the driver properly. Make sure you copied the driver file that was created to the correct location (e.g.,windir%\system32\drivers — to install a SYS Kernel PlugIn driver on Windows) and that you installed the Kernel PlugIn kernel module correctly, as explained in the this Manual.

For Windows, for example, make sure to use the correct wdreg/wdreg_gui/wdreg_frontend syntax. For SYS drivers:

wdreg -name <your KP driver name> install

⚠ Attention

WD_KernelPlugInOpen() is also called from the high-level WDC_KernelPlugInOpen() function, and from the WDC_PciDeviceOpen() / WDC_IsaDeviceOpen() functions — when they are called with the name of a Kernel PlugIn driver.

12.6.2. When handling my interrupts entirely in the Kernel PlugIn, can I erase the interrupt handler in the user mode?

Yes, you can erase the user mode interrupt handler routine.

You can also implement some of the interrupt handling in the Kernel PlugIn and some of it in the user mode. The return value of KP_IntAtDpc() (which is called when the high-priority KP_IntAtIrql() routine returns TRUE) determines the number of times that the user mode interrupt handler routine will be executed (if at all).

12.6.3. How can I print debug statements from the Kernel PlugIn that I can view using a kernel debugger, such as WinDbg?

You can use WinDriver’s WD_DebugAdd() function to print debug messages from your Kernel PlugIn or user-mode code to the Debug Monitor utility, and then view the messages in the Debug Monitor log. WD_DebugAdd() can be called from within any user-mode or Kernel PlugIn function, including KP_IntAtIrql().

You can also select to send the debug information from WinDriver’s Debug Monitor to a kernel debugger, as explained in Chapter 8: Debugging Drivers.

In addition, you can add calls in your Kernel PlugIn code to OS kernel functions that print directly to the kernel debugger — such as KdPrint() on Windows, or printk() on Linux.

12.6.4. My PC hangs while closing my application. The code fails in WD_IntDisable(). Why is this happening? I am using the Kernel PlugIn to handle interrupts.

This might happen if you are enabling the interrupt from your Kernel PlugIn interrupt routines, and simultaneously disabling it from the user mode (using WD_IntDisable() or InterruptEnable() / WDC_IntEnable() — which call WD_IntDisable()). Since the interrupt is active (having been enabled from the Kernel PlugIn), the interrupt cannot be disabled and the PC hangs waiting for WD_IntDisable() to return.

A possible solution, is to call WD_IntDisable() / InterruptEnable() / WDC_IntEnable() as an atomic operation, so that it will disable the interrupts successfully before the Kernel PlugIn interrupt routine enables the interrupt.