Adventures into VB6 "reflection" and error handling

Submitted by Vidar on

I've started working on a testing framework for VB6.
To be more specific, a testing framework that it should be possible to integrate into even the most hairy of legacy applications.
I tried reading up on existing tools, but there where not many, and the documentation left much to be desired. But the one thing I found was that they all required ActiveX dll's or exe files.
If your legacy application is not designed using this technology, for example if it's just a standard vb6 executable, your pretty much left out in the dark.
So there is my goal, a Visual Basic 6 testing framework that can be inserted into legacy applications.

The blindness

In order to avoid boilerplate code modern testing framework uses reflection to infer a lot of information about the testfixtures and testmethods. It turns out that the old world of vb6 is rather lacking in the reflection department.
The recommended way is to use the type information library, but once again I hit the ActiveX wall. The type information library only deals with dynamic libraries, which a standard VB6 executable is not.
I was left with only two reflection like methods:

  • The TypeName function, that gets the name of the type for a given object.
  • The CallByName function, that can invoke a member on a given object by name given as a string

There seems to be no way to extract the names of public members from a class module.
My solution was to require each test class to provide a public function that returns a collection of all test methods that should be invoked. (Boilerplate, and copy / paste this means.)
With this information I can use CallByName to invoke the test methods.

But this should not be my biggest challenge:

Err, Err.Raise and automation errors

Modern testrunners rely on catching exceptions to get information about what went wrong if a test fails. This could be an exception thrown by the code under test, or an exception thrown by the asserter indicating that a value was not as expected.

Trusty old VB6 does not have a notion of exceptions (nor stack trace for that matter). It does however have a global Err object. If something goes wrong the information in the Err object changes, the code path is interrupted, and guided by "On error Goto" statements you can attempt to handle the Err as best you can.

The combination of having some code in an ActiveX dll (as my testing framework will be), and some code in a standard VB6 exe project have some really obscure effects on the Err object.

As I mentioned, I use CallByName to invoke the test methods.
The test methods are defined in class modules in a standard VB6 exe project.
My test runner however resides in an ActiveX dll.
To add to your mental overflow reading this, there is an additional thing to keep in mind:
I rely on reading information from the Global Err object to get information about what went wrong if a test fails.

The above combination turns out to be rather toxic. If I raise (not throws, this is VB6 remember) an error in my test class (residing in the VB6 exe project), or my asserter (that is defined in my dll, but the instance lives in the VB6 exe project), my error listener in the testrunner only receives 440 Automation errors, no matter what information I feed into the supposedly global Err object.

It turns out This is by design. A feature not a bug. (ref: http://support.microsoft.com/kb/194418 )
VB believes that this can not possibly be worth propagating, something is obviously wrong with the linked library. It absolutely must be an Automation error.

To remedy this, even more boilerplate was needed.
Each test method must now be a function, that returns Nothing on success or an instance of my custom testErr object encapsulating the valuable information I need to present in the testrunner.

sigh
Boilerplate is the death of efficient TDD. But this is VB6, it's supposed to be really verbose, right?

Here is an example of a test class I use to test my Assert.That(item).Equals(otherItem):

Option Explicit

Dim localTests As Collection
Dim helper As TestFixtureHelper
Dim Assert As Asserter

Const ARBITRARY_ERROR_CODE As Long = vbObjectError

Public Property Get Tests() As Collection
  Set Tests = localTests
End Property

Public Function Create() As Assert_equal_tests
  Set Create = New Assert_equal_tests
End Function

Private Function Fail() As TestErr
  Set Fail = helper.Fail
End Function

Private Sub Class_initialize()
  Set localTests = New Collection
  Set Assert = New Asserter
  Set helper = New TestFixtureHelper
 
  localTests.Add "One_should_equal_one"
  localTests.Add "One_should_not_equal_two"
  localTests.Add "Text_should_equal_Text"
  localTests.Add "Text_should_not_equal_text"
  localTests.Add "True_should_equal_true"
  localTests.Add "True_should_not_equal_false"
End Sub

Public Function One_should_equal_one()
On Error GoTo failed
  Call Assert.That(1).Equals(1)
  Set One_should_equal_one = Nothing
Exit Function

failed:
  Set One_should_equal_one = Fail
End Function

Public Function One_should_not_equal_two() As TestErr
On Error GoTo reportError
  Assert.That(1).Equals (2)
  Err.Raise ARBITRARY_ERROR_CODE, "One_should_not_equal_two", "Assertion did not fail"
Exit Function

reportError:
  If Err.Number = Assert.AssertionErrorNumber Then
    Set One_should_not_equal_two = Nothing
    Exit Function
  End If
  Set One_should_not_equal_two = Fail
End Function

Public Function Text_should_equal_Text()
On Error GoTo failed
  Assert.That("Text").Equals ("Text")
  Set Text_should_equal_Text = Nothing
Exit Function

failed:
  Set Text_should_equal_Text = Fail
End Function

Public Function Text_should_not_equal_text()
On Error GoTo failed
  Assert.That("Text").Equals ("text")
  Err.Raise ARBITRARY_ERROR_CODE, "Text_should_not_equal_text", "Assertion did not fail"
Exit Function

failed:
  If Err.Number = Assert.AssertionErrorNumber Then
    Set Text_should_not_equal_text = Nothing
    Exit Function
  End If
  Set Text_should_not_equal_text = Fail
End Function

Public Function True_should_equal_true()
On Error GoTo failed
  Assert.That(True).Equals (True)
  Set True_should_equal_true = Nothing
Exit Function

failed:
  Set True_should_equal_true = Fail
End Function

Public Function True_should_not_equal_false()
On Error GoTo failed
  Assert.That(True).Equals (False)
  Err.Raise ARBITRARY_ERROR_CODE, "True_should_not_equal_false", "Assertion did not fail"
Exit Function

failed:
  If Err.Number = Assert.AssertionErrorNumber Then
    Set True_should_not_equal_false = Nothing
    Exit Function
  End If
  Set True_should_not_equal_false = Fail
End Function

Private Sub Class_terminate()
  Set localTests = Nothing
  Set Assert = Nothing
End Sub

Hopefully it's not too bad, but If anyone have good ideas on how to reduce the boilerplate please do tell me.

Some links, and thoughts about the future

The project is called OldUnit and all the code is on Github: https://github.com/Vidarls/OldUnit
If you feel that you can contribute and help all the developers in the world stuck with maintaining untested and untestable legacy applications, please do fork it.

So far the Asserter only have the Equals assertion implemented, and after listening to Llewellyn Falco (http://twitter.com/isidore_us) on the Herding Code podcast, I'm contemplating to ditch the Asserter and just provide a wrapper for his Approvals library (http://approvaltests.sourceforge.net/)

Any thoughts and comments are welcome.

Categories: