Handling Windows Messages In Visual Basic 5/6

by Richard Matthias

Introduction

My background is in C and C++ programming and when I learned windows programming it was with the SDK (raw windows API calls). However, as part of my current job I had to learn Microsoft Visual Basic which is not so bad, but I was frustrated at first by an inability to do a lot of the things an SDK programmer can.

One of the nice things about VB is that the chore of handling the majority of common windows messages is taken care of. Where it is appropriate for your application to respond to a windows message sent to it, VB allows you to have an Event Handler. What about all the (literally hundreds) of other windows messages that are not passed on to VB's Event Handlers? What if you knew a bit about SDK programming and wanted to catch some of them?

There is no general purpose way to catch windows messages in VB. The Component Tools Guide that comes with the Pro and Enterprise versions of VB does hint at a way of doing what I wanted and this document covers the technique in detail.

This document assumes a thorough knowledge of Visual Basic programming. It also assumes a certain amount of knowledge of SDK programming, although the resultant code (downloadable) can be used by any competent VB programmer. As such, this document does not explain the terminology used when discussing SDK programming.

Why Catch Windows Messages?

Most VB programmers know a few Windows API's and how to call them (probably the most popular being GetPrivateProfileString). However, API calls are only half of the windows programming experience - the other half being messages. Messages are either sent by an application to a window belonging to an application (these may be the same applications or different ones).

Two reasons why you might want to send a message in your application are:-

  1. Windows common controls are created using an API call (OK, so usually VB creates them for you, but VB has to do it using CreateWindow() ), but once they are created your application communicates with them using messages. Normally you do not have to worry about this because VB takes care of it for you, but there is sometimes common control functionality which is not covered by VB's properties or methods for a control. An example of this is the ListBox control. The corresponding VB object has no property for setting up a horizontal scroll-bar. You can get a horizontal scroll-bar on your VB ListBox though by sending it a message (I may cover this in a future document).
  2. User and system behavior can be simulated by sending or posting messages to other applications windows.

As previously stated, most of the messages a windows application would want to handle are handled by VB - if necessary VB will then call an Event Handler in your code. There are other messages which you might wish to handle if only you knew they were being sent. The only way to find out about these messages though is to either read through the section on messages in the windows SDK documentation (it is extremely long), or to pick up on particular messages from articles like this.

A good source of information is the Microsoft Developer Network Library. A special edition of this CD comes with the Pro and Enterprise versions of VB and Visual Studio and it is packed with articles and documentation. It also includes the full windows API documentation which is handy because the SDK help file that used to be shipped with VB3 and VB4 is no longer included in VB5. You can also read the MSDN library by registering at the MSDN web site.

Windows sends your application (or more correctly your application's window) messages for various reasons, but they largely fit into two categories:-

  1. To inform the window of some event: resizing; mouse move; key-press.
  2. To ask the window something:
    1. Do you want to be closed?
    2. Did the mouse hit a non-client area of the window?
    3. What is the minimum and maximum a size you want the window to be sized to?

Category 1 is the best covered by the built-in message handling of VB. Some things in category 2 are covered like 2a (see the QueryUnload event). 2b is an interesting one - if this message is handled you can achieve the same effect as the windows clock (i.e. clicking on the background of the clock drags the window the same as clicking on the title bar). It is the message described in 2c that sparked my interest in this issue and so it is discussed throughout the rest of this document.

WM_GETMINMAXINFO

This message is sent to a top-level window after the user clicks on the window frame to start the resizing operation, but before the mouse moves to actually resize the window. Normally a VB form can only be sizeable or fixed in size. Fixed borders are fine for dialogs, but to have fixed borders for all the windows makes your application look unprofessional. Resizable windows are great, but then you have to write code to handle the Form_Resize event. This is unavoidable unless you want to use one of the many ActiveX controls available to handle resize behavior for you. By far the best of these I have come across is KL Group's Olectra Resizer.

If your resize code is fairly simple then your will almost certainly end up in the situation where you want to be able to limit the minimum or maximum size the user is able to size the window to.

The WndProc of the window being resized is sent the WM_GETMINMAXINFO message just before the user is to start moving the mouse to resize the window. The rParam of the message is undefined and the lParam points to a structure: MINMAXINFO. We are only interested in two elements of the structure for the purpose of this exercise: ptMinTrackSize and ptMaxTrackSize. These are POINT structures containing and X and Y dimension as long integers. When the WM_GETMINMAXINFO message is processed, these ptMinTrackSize and ptMaxTrackSize elements must be filled to tell Windows the minimum and maximum size that the user should be able to resize the window to.

Window Subclassing Using AddressOf

So we know why we want to handle Windows messages, and which message would be interesting to handle, but how do we handle them when VB has no facility to handle arbitrary messages?

Subclassing

The key here is a Windows SDK technique called Subclassing. Windows uses a lot of pseudo-object-oriented terminology to describe various concepts. All windows belong to a class and as such share certain attributes. The most important of these attributes is the procedure which handles messages (commonly known as the WndProc). This procedure is important because it not only defines how the window responds to user interaction events, but also how the window appears (since it handles the WM_PAINT message and thus gets to paint the window).

When you subclass a window you substitute a procedure of your choice (usually one in your program) for the WndProc of an existing window. When you do this the procedure of your choice effectively gets full control of the window because it can ignore or deal with all windows messages how it sees fit.

The substitute WndProc will receive all message that would usually be handled by the original WndProc. You will probably only want to deal with one particular message or group of messages when you subclass a window and so the preferred action is to pass off all the messages you are not interested in to the original WndProc. You receive a pointer to the original WndProc when you perform the subclass operation - you must store this so that you can call the windows' original WndProc as required.

AddressOf

To subclass a window you must supply windows with the address of that procedure so that windows can call it. This is something that was not possible with versions of VB before 5, but it is now thanks to the AddressOf operator. When a windows API call requires a pointer to a function (e.g. for event handling) then you just specify the name of the procedure in your program preceded with the AddressOf keyword.

There are some restrictions on the use of AddressOf. The procedure can only be located in a standard module (.bas file) in the current project. It says in the help file that the API you want to pass the address of your procedure to must be declared such that the procedure argument is of type Any. This seems incorrect as the API Text Viewer application gives the address argument As Long. Declaring the argument as Long seems to work, so we'll stick with that for now!

So, the declarations for SetWindowLong() and CallWindowProc() would be:-

Declare Function SetWindowLong Lib "user32" Alias "SetWindowLongA" _
    (ByVal hWnd As Long, ByVal nIndex As Long, ByVal dwNewLong As Long) _
    As Long

Declare Function CallWindowProc Lib "user32" Alias "CallWindowProcA" _
    (ByVal lpPrevWndFunc As Long, ByVal hWnd As Long, ByVal Msg As Long, _
    ByVal wParam As Long, ByRef lParam As MINMAXINFO) As Long

The parameters dwNewLong and lpPrevWndFunc are the procedure addresses in the above declarations.

The Substitute WndProc

There are a number of features common to every substitute WndProc. The first is the definition of the procedure. Essentially it must have the same number and type of arguments as all other WndProc. It must also be a function returning a BOOLEAN. So, a general purpose substitute WndProc would be defined something like this:-

Public Function wndProc(ByVal hWnd As Long, ByVal iMsg As Long, _
                        ByVal wParam As Long, ByVal lParam As Long) As Long
    ....
End Function

I say the arguments will always be the same for all WndProc. This is not quite right. For some windows messages the lParam is a pointer to a structure containing additional information. For the WM_GETMINMAXINFO message lParam points to a structure called MINMAXINFO which is not so much extra information for your WndProc, but rather a place for your WndProc to put information for windows' benefit. This structure (MINMAXINFO) can be represented in VB using a user-defined Type. To tell VB that the lParam is to be treated as a pointer to a structure you have to make it a ByRef parameter of type MINMAXINFO.

The types are defined in VB thus:-

Type POINTAPI
    x As Long
    y As Long
End Type
Type MINMAXINFO
    ptReserved As POINTAPI
    ptMaxSize As POINTAPI
    ptMaxPosition As POINTAPI
    ptMinTrackSize As POINTAPI
    ptMaxTrackSize As POINTAPI
End Type

The whole function is shown below with line numbers to the left:-

1  Public Function wndProc(ByVal hWnd As Long, ByVal iMsg As Long, _
2                          ByVal wParam As Long, ByRef lParam As MINMAXINFO) As Long
3      If hWnd = This_hWnd Then
4         If iMsg = WM_GETMINMAXINFO Then
5             lParam.ptMinTrackSize.x = Min.x
6             lParam.ptMinTrackSize.y = Min.y
7             lParam.ptMaxTrackSize.x = Max.x
8             lParam.ptMaxTrackSize.y = Max.y
9             wndProc = True
10            Exit Function
11        End If
12     End If
13     'Otherwise, pass it to old Window Procedure
14     wndProc = CallWindowProc(procOld, hWnd, iMsg, wParam, lParam)
15 End Function

Windows will call this procedure for every message that would normally be sent to your Form (VB handles the messages for you normally). If you've ever used one of the many message spying programs available you'll know that windows can receive hundreds of messages a second when there is mouse activity over a window and the number of messages sent to the window even for mundane things such as key-presses or maximizing would surprise most non-SDK programmers. For this reason it is a good idea to compile a program that uses sub-classing with the 'native code' option.

Out of the thousands of messages your form will be sent between being created (yes, it gets messages even before it is shown) and being destroyed, you are probably only interested in one (albeit one that can occur a lot if the user resizes the form a lot). Since most VB programmers will not have written a WndProc before I will repeat here the 3 key points of the WndProc:-

  1. If you handle a message successfully return True.
  2. If you handle a message, but not successfully for some reason (the reason would depend on the semantics of the message in question), then return False.
  3. If you do not handle a message you should call the original WndProc and return whatever value that returns.

These three points are embodied in the wndProc procedure shown above. If the message is WM_GETMINMAXINFO then the structure pointed to by lParam is filled in and wndProc returns True. The structures have been declared just like any other user-defined type in VB and so they are accessed like any other type as shown in lines 5 though 8 in the listing. Min and Max in the listing are just some module-level variables of type POINTAPI.

In the listing (above), when the message is not WM_GETMINMAXINFO, the previous WndProc is called using the API function CallWindowProc(). The previous WndProc had been stored in another module-level variable called procOld. procOld is simply declared as a Long since this is what the SetWindowLong() and CallWindowProc() API calls use for WndProc addressing.

Performing the subclass

If you download and take a look at the example project you will see that all the above code, variable declarations and type definitions are in a standard module called ModProc.bas. The form SelfResize uses ModProc.bas to restrict it's resizing. In the Form_Load event the windows API call SetWindowLong() is used to subclass the form (the form has been created by the time the Form_Load event is called even though the form is not shown yet - this is why the hWnd property of the form is valid). You can subclass any window in exactly the same way - you just need some way of getting hold of the window handle (the hWnd).

Private Sub Form_Load()
    procOld = SetWindowLong(SelfResize.hWnd, GWL_WNDPROC, AddressOf ModProc.WNDPROC)
End Sub

Private Sub Form_Unload(Cancel As Integer)
    procOld = SetWindowLong(SelfResize.hWnd, GWL_WNDPROC, procOld)
End Sub

In the example SetWindowLong() is used again in the Form_Unload event to remove the substitute WndProc. The line is almost the same as that in the Form_Load event except that we are now setting the WndProc to be procOld - i.e. what is was originally. This is not strictly necessary in the case of the SelfResize form; the form is about to be unloaded. However, if you are subclassing other windows - maybe those of common controls or common dialogs - then you must be careful that you do not leave your substitute WndProc in place when it is no longer appropriate. You will have to use your own discretion to find what is appropriate and what is not, but suffice to say, you will experience some very odd behavior if you get it wrong!

Putting It In An ActiveX Control

The technique described above works, but that is about all you can say about it! If you have a project with many tens of forms then it becomes painful to manage all the variables necessary for the routines in ModProc to handle WndProc calls on behalf of all the different forms and return different minimum/maximum sizes for each. You also need to add code to the Form_Load event of each form which can then get mixed up with all the other code that inevitably finds its way into a form's Form_Load event.

How much cleaner it would be if you could just drop an ActiveX control onto the form and set the desired minimum/maximum sizes as properties of the control. VB5 makes it easy to create ActiveX controls. The example project MinMaxCtrl contains the actual control form (.ctl) and a supporting standard module (.bas). These work together in pretty much the same way as the ordinary form and module work in the previously discussed project.

When the control is created on a form (in a running program or in the design environment) the UserControl_ReadProperties event is executed. The example control does two things here:

  1. The saved values of the properties are retrieved from the property bag (the owning form). If the control has just been dumped on a form for the first time in the design environment, there will not be any stored values for these properties. In this case the default values of 0 for minimum and screen size for maximum are used.
  2. SetWindowLong() is called to substitute the WndProc in the standard module for the WndProc of this control's parent (the form the control was placed on). The hWnd of the parent form and the address of the old WndProc are stored away. We need to keep hold of the hWnd because the control can be placed on many forms in the same program and the WndProc needs to keep track of which sizes belong to which form.

The UserControl_Terminate event replaces the substitute WndProc with the original one. The UserControl_WriteProperties event writes the new values of the properties back into the property bag (assuming they were modified during the form design. For more detailed information on the nuts'n'bolts of ActiveX control creation take a look at the Component Tools Guide that comes with the Pro and Enterprise versions of VB.

Quite apart from the sheer convenience of having the resizing code packaged as a ActiveX control, another slightly less obvious advantage is visible when the control is placed on a form in the design environment: The code in an ActiveX control is executed within the design environment as well as when the finished program is running so you can see the size of the form being limited at design-time as well as in the final product!

Conclusion

Although at first glance it would seem the VB programmer is the poor relation of the SDK/MFC programmer, there is much that can be achieved given sufficient knowledge of the Windows API and VB tricks.

Download

The VB project used as an example in this document can be downloaded here. The file is a VB project group containing two projects in the same directory. One project has the two forms - one self resizing and the other holding a MinMaxCtrl instance to handle resizing. The second project is the MinMaxCtrl ActiveX control source.

You may experience some errors when loading the SizeTest1 project into the VB environment for the first time. This is because the MinMaxCtrl.ocx file will not be registered on your system until you compile the MinMaxCtrl sub-project. For this reason it is recommended that you load the MinMaxCtrl project first and recompile it and then load the project Group. If you open the ControlResize form before the control is registered the instance of the control on that form will be converted to Picture Box. If this happens, close VB without saving anything; load and compile the MinMaxCtrl project and then you can load the project group and look at the ControlResize form.

Acknowledgement

Windows, Visual Basic, Microsoft Developer Network, MSDN, Visual Studio are trademarks of Microsoft Corporation.