Thursday, March 25th, 2004

Even though Ward Cunningham and I have been living only 4 miles away from each other for some years, I really didn’t know that he lived in in the same state ’til he sold his soul hired on at Microsoft. Since then, we’ve been meeting at a local Starbucks on a regular basis to shoot the shit in a very DevelopMentor/Tektronix-like way, i.e. ramble on about whatever ’til we hit on something fun to try. In one of those meetings, we were talking about requirements for a web services API that I’m working on and he took us off on what I thought was a tangent, but turn out to be very cool.

Before Ward set me straight, I thought that Test-Driven Development (TDD) was when I built a little console app to test my APIs. I routinely do this before I write an API so that I can imagine what I wish I had to program against without being encumbered with an existing implementation. Then, I run my console app(s) afterwards for testing and regression testing, but only sporadically. I knew that NAnt let me run these tests into my build process, but since I’m a VS.NET guy and I haven’t done the NAnt/VS.NET integration work that I know I should do, I haven’t done as much with this as I knew I should be. Plus, as it was only me, it didn’t seem like such a big deal.

What makes TDD a big deal, however, is when you’ve got a team of folks. Of course, every developer is expected to check in tests for their stuff and those would be run as part of the nightly build. But that’s not the cool part. The cool part is when another developer adopts my API and puts his tests into my build process. When someone adopts an API that I build, then make certain assumptions about how it works and what side effects they can expect. TDD lets Joe R. Programmer encode his assumptions as unit test code so that when I break his assumptions during an iteration of my API, I see it as part of my compiler output.

Until I get a build error, I don’t need to know or care about Joe’s test. But when I do, I look at Joe’s unit test code and decide whether he was right. If Joe’s unit test is valid, I need to either fix my breaking change or be prepared to help everyone that uses my API to rewrite and redeploy their code. If Joe’s unit test isn’t valid, I’ve got to help Joe fix his code and prepare to help everyone else that made “invalid” use of my API to begin with. The very nicest things about TDD is that “fixing breaking changes” becomes the least painful thing to do!

This is huge. In fact, it’s so huge, that MS should provide an end point for people to submit their own unit tests against our Windows and .NET APIs so that when we make breaking changes, we can either fix them or know the proportion of folks affected by this breaking change.

Anyway, when Ward told me this, my eyes were opened and I was changed. But he was just getting warmed up. After showing me the power of formal unit tests across a team, he then went on to describe the power of putting the unit tests right into specifications themselves. For example, imagine a table in a spec laying out a series of inputs and expected outputs:

Now imagine a parser that could read through this table and execute the code being specified, checking expected output against actual output, lighting matching output green and mismatched output red:

This scheme lets designers define the spec long before it’s implemented and then check it’s progress as it’s implemented. Likewise, a spec itself can be put into the build process as another set of unit tests. Plus, the readability and maintainability of specs in this format is much improved and very much more accessible then unit test code.

But wait, there’s more. This isn’t some pipe dream. Ward has a .NET implementation of it called the FIT Framework on the web and yesterday he and I played around with it in my kitchen. Our goal was to take Ward’s .NET FIT implementation and add support to it for SOAP endpoints. We started with the ISBN Information web service that we found on xmethods.org and defined the following spec:

We specified the name of the FIT “fixture” (which is the class that implements the tests in FIT), as well as the WSDL URL, the service type and method we’d like to test, the arguments we’d like to test and the results we expected to get. We mixed in some of Christian Weyer’s Dynamic Web Services Proxy and we got this:

Notice that we’re not quite done yet, i.e. nothing is green and we’re not reaching into the retrieved XML and pulling out the Author, Publisher, etc. Still, while FIT requires a matching piece of code for every spec, we dropped a bit more metadata into the tables for the SOAP version, leveraged WSDL and have narrowed the project to a single piece of generic code for any WSDL-described SOAP endpoint (so long as Christian’s code can parse it).

Our plan is to finish up the extensions to FIT for SOAP and to put it into practice on the SOAP API that I’m currently designing and even to push it into our build process if I can. Further, I’d like to build another FIT fixture that, instead of executing the tests, generates sample code to demonstrate example usage and expected output (including expected errors). Rest assured, when we get it working well enough for public consumption, Ward and I will publish the SOAP extensions for your enjoyment.

But don’t wait for that! You can do TDD with custom actions in VS.NET 2003 and with the FIT Framework today!

Discuss

I knew Mindreef had cool testing tools, but I just found one today on their web site that I ended up using all afternoon. You feed it a WSDL URL and it generates a form, allowing you to see the SOAP request and response packet in several formats include raw XML, to pseudo-code, pretty-printed XML and tree. It’s much more flexible then the built in one that ASP.NET provides and the price is right.

Here.

Rory says:

“I mean, here was this guy who managed to make .NET Code Access Security look like fun. It’d be easier to pull a three-headed monkey in a tutu singing barbershop choir out of your ass, but he still accomplished it. Even my girlfriend who didn’t know diddly about coding walked away making thoughtful noises to herself. To this day, she sleeps in a blue oversized Chris Sells t-shirt. How many other coders out there have silk-screened images of themselves draped over the nubile naked bodies of women as hot as my little tart of a girlfriend? Not many, my friends. Not many at all.”

I’m met Rory’s “tart of a girlfriend” and there’s really nothing more I need to consider accomplishing. : )

Tom Barnaby has some tips for preparing today for Indigo tomorrow.

Brad Abrams posts a nice, consise description of two tweaks to the GC that Whidbey provides to enable developers to handle unmanaged resources in their .NET objects better. Check it out.

Tonights Portland Nerd Dinner was a barn-burner. Not only did we have the typical geek banter and laughter, but we spend some time on the question I posed the other day about what should replace character stream-based programming.

The problem is that all of the hand-crafting of solutions doesn’t scale. The answer was posed by a fellow nerd who, who pointed out that we’ve already got a model for self-organizing, self-evolving, self-maintaining systems: biology. God didn’t hand code humans and zebras and platypi — he defined a class and instance encoding method (DNA) and an environment to serve as the runtime (Earth) and let us interact with and improve ourselves and our environment over time to solve the problem. What’s the problem? What species survives best on this planet. So far, we’re the winners, but we’ll see what happens at the next ice age or when the next asteroid hits.

.NET and Longhorn have provided us a big step forward in terms of hand-crafting computer-based solutions to our problems, but only self-organizing, self-evolving, self-repairing systems can really scale. I know that IBM has done some work in this area with their automonic computing, but my personal favorite work on this topic is Genetic Programming III by John Koza. Anyone for a Genetic Algorithm Runtime (GAR)?

I don't think the interview covers how my blog got it’s name, but how can I quibble when I get a rare link from the ever watchful Mary Jo Foley, and one so eloquently titled to boot!

So long as the post office would toss the junk mail and I could switch my utilities to email from the source, I’d happily pay $.22/letter to get my snail mail in email earlier. Come to think of it, I'd pay them just to drop my junk mail...