At Microsoft we have an internal email list called “Spot the Defect” — people mail around buggy code they’ve discovered and we compete to see who can find the most problems with it. It’s fun, and you learn a lot about what other people consider bugs — everything from security holes to lying comments.

I love playing Spot the Defect. Here is the code for the WScript.Sleep method with the comments removed and some serious bugs added. You’ll note that this code has all the required features I mentioned in my previous post.

We go to sleep in one-second (or less) intervals, and tell the operating system to wake us up if COM posts a message to the message queue, because there might be an event handler to dispatch. We also check to see if the host recorded a script error (either due to an event handler or due to the script timeout firing) so that we can abort the sleep. This way we never keep the script alive more than a second after it was shut down due to error.

What bugs did I add?

HRESULT CWScript::Sleep(long Time)
{
  const DWORD TimerGranularity = 1000;
  if (Time < 0)
    return E_INVALIDARG;
  DWORD StartTickCount = ::GetTickCount();
  DWORD EndTickCount = StartTickCount + Time;
  DWORD CurTickCount = StartTickCount;
  while(CurTickCount < EndTickCount)
  {
    MSG msg;
    DWORD CurWaitTime = (DWORD)(EndTickCount - CurTickCount);
    if (CurWaitTime > TimerGranularity)
      CurWaitTime = TimerGranularity;
      ::MsgWaitForMultipleObjects(
        0, NULL, TRUE, CurWaitTime, QS_ALLINPUT | QS_ALLPOSTMESSAGE);
      if (0 != ::PeekMessage(&msg, NULL, 0, 0, PM_REMOVE))
        ::DispatchMessage(&msg);
      if (m_pHost->FErrorPending())
        return S_OK;
      CurTickCount = ::GetTickCount();
  }
  return S_OK;
}

Commentary from 2019:

Readers from 2019 who are used to reading C# may need the hint that long and DWORD are both 32 bit integers.

I don’t recall why we had this strange convention of UpperCasing local variables; the WSH codebase was actually taken over by the scripting team from another team that had developed it for their own use, and there were a number of odd choices in that codebase. I suspect this was one of them; I’m a big believer in keeping new work consistent with old work, even if you wouldn’t use the conventions of the old work in brand-new green-field code.

There were a lot of good responses to this puzzle:

Is it legal to call MsgWaitForMultipleObjects with zero handles?

Yes, in regular desktop or server Windows. A user pointed out that this is not legal in the CE version of Windows.

After MsgWaitForMultipleObjects returns, you should pump all waiting messages, not just one.

Great catch! That bug was actually in the code; that’s not the bug I added for this puzzle.

I sent the bug on to the scripting sustaining engineering team.

It looks like there is an overflow bug at StartTickCount + Time​

There is; can you be more specific?

Suppose the machine has been on for just under 49.7 days, which is when GetTickCount() overflows, and adding Start overflows. The end time will be less than the start time, so it will not sleep at all.

That is a bug, but there is a worse one.

Suppose CurTickCount and EndTickCount are very close to overflowing, and the second time you set CurTickCount it is set to a small value after the overflow. The function will then sleep for 49.7 days.

That’s correct, but it’s actually worse than that; it will sleep for at least 49.7 days, and then it has a possibly very narrow “window” of tick counts that will avoid sleeping for another 49.7 days. If that window is small then we are likely to sleep for a long, long time.

The real code was written to take all of these problems into account; it does all of its timing arithmetic in doubles, which have more than enough range and precision to avoid overflow issues, and detects clock overflows correctly.

Yes, the title is grammatical. A few days ago I was discussing the semantics of data that isn’t there. Today I want to talk a little about programs that do nothing. What do you do when you want a program to pause briefly, for whatever reason?

Those of you who like me cut your programming teeth on Commodore PET Basic know the answer. A for loop will do the trick.

for(x = 0 ; x < 1000000 ; ++x) {}

Of course, that has the problem that the time spent paused is shorter on faster machines. The obvious improvement waits until a given number of milliseconds have passed:

for(var startTime = new Date(); new Date() - startTime > 1000 ; ) {}

The problem with both these techniques is that this is busy waiting.  The program may appear to be paused but it is actually executing as furiously as ever.  The processor is probably pegged to 100% but doing entirely pointless work.  In a multithreaded, multiprocess operating system busy waiting is downright rude — the processor could be tending to other tasks like redrawing the screen or background processing.

Really what you want to do is to ask the operating system to put the process asleep for a bit and then wake it up later, right?  That way the program pauses but the thread scheduler is free to run something more important. And indeed, there’s a Win32 API that does just that, Sleep.

But there is no way to call win32 APIs directly from script, so the scriptable “go to sleep” method needs to either be built in to the language, or (the horror!) in an ActiveX object. So why the heck do VBScript and JScript not have a “go to sleep” method built in?

There were good reasons why we didn’t. The principal reason is because I lied above.  You do not want to put the process to sleep and wake it up again later, because while the script is “sleeping”, you still might want event handlers to run.

The way events work in Apartment Threaded COM objects such as initialized script engines (see my earlier entries for a refresher on how the script engine threading model works) is pretty simple. When the user clicks on a button and raises a button event, what actually happens is a bunch of messages are deposited in the thread’s message queue.  When the message queue is pumped, the messages tell the COM plumbing to call the event sinks listening to the event sources.

What happens when a thread is asleep? Nothing! In particular, no message loop runs on the thread. How could it?  It’s asleep. So you put your script to sleep for ten seconds, a user presses a button, and then waits at least ten seconds for the message loop to get pumped. That seems kind of suboptimal. The sleep method needs to pump a message loop occasionally so that events get handled.

But wait, it gets worse.

What if one of those event sinks causes a script error, which is then reported to the host, and the host decides to shut down the script engine? Ten seconds later, the script wakes up and keeps running! Oh, the pain.

To properly implement a “go to sleep” method you need to know all kinds of details about the desired message processing, event handling, error handling and multi-threading semantics of the host application.  We were very worried that we’d add a method to the script engines which, when used in IE, or ASP, or WSH, or some third-party script host, completely screw up the carefully implemented host.

Hence, there is no “go to sleep” method in the languages.  If the host implementor thinks that a “go to sleep” method is necessary, they can implement one and add it to their object model, as we did for WSH.

Commentary from 2019:

A reader pointed out that you can use a combination of window.setTimeout and window.showModalDialog in the browser to sleep the UI while still allowing event handling and data binding to work.

I’m sure there are better ways to do that in modern asynchronous JavaScript, but I have not kept up with advances in JS asynchrony so I don’t know what they are.

However, the original comment points out exactly what I was talking about in the article; setTimeout knows exactly what the internal message pumping mechanisms are in the browser, and can therefore do it correctly; the sensible place to put any kind of “delay” function is in the code that knows how to manage it. We designed the script engines to be hosted in browsers, servers, games, and so on, and did not want to put anything into the language that could be badly abused in those environments.

I didn’t mention DoEvents in this article, which is often used (badly!) to make busy-loops that handle events; I believe I discussed why that is almost always wrong when I discussed adding asynchrony to C#.

I’m having my kitchen redone.

Yes, I remember that in my very first blog entry I said that I would be talking about only technical matters, not trivia about my life.  Bear with me, people.

When I say “redone” I mean that I took a crowbar, a sledgehammer and a couple of helpful housemates, pulled down all the walls, pulled up the floor, trucked the asbestos-laden vinyl flooring to the dump and then invited over a bunch of big burly guys to build a new kitchen from scratch.

Unfortunately, we found a number of “structural issues” once the walls were apart which necessitated some pretty major infrastructure improvements.  I was hoping to have all the work in the kitchen done in early October, but these various non-scheduled improvements ate up a lot of time.  I got an email from the general contractor the other day saying that the revised schedule now has us finishing up in October — October 45th to be precise.  Ha ha ha.  Very funny guys.

Update: We finished on January 16th, taking exactly twice as long as initially forecast, and confirming the old saying that a poorly planned project takes three times as long as you think, a well planned project only twice as long.

The good news is that this reminded me that when I dissed the VBScript date format a few weeks ago, I never got around to dissing the similarly goofy JavaScript date code.  October 45th?  No problem!

var d = new Date(2003, 9, 45);
print(d);

prints out

Fri Nov 14 00 :00:00 PST 2003

What the heck?  “2003, 9, 45” is the 14th of November?

Yep.  First of all, for reasons which were never adequately explained to me by my cohorts at Netscape, the date constructor numbers dates starting from zero.  So “9”, not “10”, is October.  If you want to know why that is, ask Brendan next time you see him, ’cause I sure don’t know. (UPDATE: See below.)

But what’s more interesting is that any attempt to schedule projects like my contractor made result in real dates.  November 14th is, in some sense, the best candidate for October 45th, as it is 44 days after October 1st.   You can get weirder than that, of course.

var d = new Date(2003, -1, 0);

prints out

Wed Nov 30 00:00:00 PST 2002

Which is only logical — if January was month zero of 2003 then December 2002 was the “negative first” month of 2003.  And if December 1st was the 1st day of December, then November 30th must have been the 0th day. Similarly, November 29th was the -1st day, and so on.

The underlying implementation details of JScript dates are quite a bit more sensible than VBScript’s implementation.  JScript also stores dates as double-precision 64 bit floats, but the JScript implementation stores all dates as the number of milliseconds since midnight, 1 January 1970 UTC.  Universal Time Coordinated is the proper name for what most people call “Greenwich Mean Time” — the time at the prime meridian, with no adjustment for British Summer Time.

For all practical purposes, we treat the 64 bit float as a 53 bit integer plus a sign bit.  This means that JScript has 300000 years of millisecond accuracy on either side of the epoch, as opposed to VBScript where the precision varies over the range.  Also, in this system there are no two dates with the same numeric representation, which makes comparing, subtracting, adding and rounding times straightforward.

Also, JScript, unlike VBScript, adjusts for daylight savings time and time zone differences according to the local information reported by the operating system.

There are some weirdnesses in JavaScript though.  The most bizarre is the getYear method, which returns a one-or-two-digit number for all dates from 1900 to 1999, and a four-digit number for dates before 1900 or after 2000.  Again, why JavaScript has a function that obviously causes Y2K-style errors implemented a few years before Y2K is unknown to me — ask your favourite Netscape language designer next time you meet one at a party.

Microsoft didn’t make the problem any better when we screwed up our implementation of getYear — the original IE implementation returns the year minus 1900, so 2000 was year 100.  Fortunately we corrected that incompatibility in the second version.  You should never use getYear — use getFullYear, which always returns all the necessary digits.

More generally, you might wonder if there are design principles underlying the decision to make “October 45th” a legal date.  Indeed there are (though to be perfectly frank, when it comes to the date handling, I suspect that any “justification” might be somewhat post hoc.)

Here are some thoughts that go through the minds of language designers:

• The sooner you produce an error for bogus behaviour, the sooner the developer will catch the bug.  Therefore, at the first sign of anything out of the ordinary, crash and die immediately.
• But wait a minute — the more errors we produce, the more error handling code the developer will have to write, because some of those bogosities will be produced by end user data.
• Worse, if the developer misses an error handling case then the error will be reported to the end user.
• The end user of a web page script is someone who has absolutely no ability to understand the error.  If they do understand the error, they have no ability to go to the web server and fix it.  Therefore, never produce error messages unless you absolutely have to.

The ability to start with a premise and deduce its opposite is a neat trick, you must admit!  But seriously, the existence of conflicting goals explains why there is more than one programming language. Hard-core application development languages like C# are designed to trap as many bugs as possible at compile time and run time so that you can find them, write the handling code, and ship.  C# demands that you type your variables, that you call reflection explicitly, that you mark unchecked arithmetic, and so on.

But C# is a language designed for building hundred-thousand-line application frameworks and object models, not hundred line web page scripts.  C# is designed for professional developers, not hobbyists and departmental web developers who just want to get “glue code” written as rapidly as possible.

That’s why JavaScript’s attitude towards error cases is not “die at the first sign of trouble” but rather “muddle along as best you can”.  Assign to an undeclared variable?  Declare it dynamically.  Forget a semicolon?  Insert it.  Try to create a date for the 31st of September?  Move it to October 1st.  Divide by zero?  Return a NaN .  Reference an element beyond the end of an array?  Grow the array.  All these things that would be compile time or runtime errors in other languages are just handled for you in JScript, for better or for worse.

And this is why when people come to me and say “my team of twenty developers has written a hundred-thousand-line database program in JScript and now I can’t make head nor tail of it, help!” I tell them to use the right tool for the job!  C# was designed for those tasks.  Heck, JScript .NET was designed for those tasks!  But a weakly-typed, interpreted, fail-silently, late-bound, non-generational-GC language like JScript was designed to add scripting to simple web pages.  If I give you a really great hammer then you can build a decent house but not a decent skyscraper.

Updates from 2019:

This article produced some responses from noted individuals. Raymond Chen pointed out:

Looking back over my entries in the last couple of weeks, I see that I’ve got a bit of a theme going here.  A number of entries have been about programming language features that exist in both JScript and VBScript, but have semantics just dissimilar enough to trip up the unwary.  Well, here’s another one! What’s the difference between this VBScript:

Function Rezrov(Foo)
  ' whatever
End Function
' ...
With Frobber.Blorple
  .Frotz 123, 456
  Rezrov .Cleesh
End With

and this JScript:

function Rezrov(Foo)
{
  // whatever
}
with(Frobber.Blorple)
{
  Frotz(123, 456);
  Rezrov(Cleesh);
}

?  At first glance, both look pretty straightforward.  But there is an important difference. The VBScript example is exactly equivalent to this program:

Set Temp = Frobber.Blorple
Temp.Frotz 123, 456
Rezrov Temp.Cleesh

whereas the JScript is logically equivalent to this pseudo-JScript:

var temp = Frobber.Blorple;
if (temp has property frotz)
  temp.frotz(123, 456);
else
  frotz(123, 456);
if (temp has property Rezrov)
{
  if (temp has property Cleesh)
    temp.Rezrov(temp.Cleesh);
  else
    temp.Rezrov(Cleesh);
}
else
{
  if (temp has property Cleesh)
    Rezrov(temp.Cleesh);
  else
    Rezrov(Cleesh);
}

As you can see, if Frobber.Blorple actually has a Rezrov member, these two programs do completely different things.  In the VBScript program, you know (and the compiler knows) whether you are calling a property on the With object or not.  In the JScript program, you don’t know and the compiler doesn’t know either.  The compiler has to emit code that does the namespace lookup at runtime.

This is extremely costly!  Consider for example what happens to local variables:

function foo(){
  var abc = 123;
  with(Frobber.Blorple)
  {
    Rezrov(abc);
  }
}

Is that a reference to local abc or to Frobber.Blorple.abc?  Like yesterday, because of the potential of expando objects the compiler cannot tell.  Normally in this case the compiler would generate a binding to the exact offset in the local scope table, but since the compiler cannot do that, it must instead generate a by-name lookup of the entire scope chain.

It gets worse.  Suppose you provide the Frobber.Blorple object and one day you want to add a property to it called Blah.  How can you do so safely? You can’t!  Every user of your object who put it in a with block and who has a local or global variable called Blah used in that with block is going to suddenly break.  Every time you add (or remove) a property, you run the risk of changing the variable lookup semantics of an existing program.  And the converse holds as well — every time you use a with block, you run the risk of accidentally binding to the wrong thing when someone changes the object properties.

The with block in JScript is a terrible programming construct.  It makes your programs slower, harder to understand, brittle in the face of changes and generally more bug prone.   Do not use it! 

The VBScript With block on the other hand, by the simple expedient of requiring a leading period on all references to the object, does not have these problems.

For a discussion of the pros and cons of the feature in VB, this discussion on the Fog Creek software forum has input from both me and from the designer of the feature, Joel Spolsky. https://discuss.fogcreek.com/joelonsoftware2/50125.html

Commentary from 2020:

• I stand by my opinion that “with” in JS is harmful; it’s a bad feature that should never have been added in the first place. It is deprecated in ECMAScript 6.
• The tradition of calling out bad programming language features by writing a post titled “X Considered Harmful” is a snowclone. I’m happy to participate in this tradition.
• A commenter reminisced about the pleasures of playing Enchanter, Sorceror and Spellbreaker from Infocom; many of the metasyntactic names in this blog are from those games.
• I also reinforced in the comments that the fundamental problem is not that “with” in JS makes your variable lookups hundreds of times slower — odds are good there is even slower stuff in your JS program! The problem is that it makes programs hard to read and reason about, and it makes them brittle in the face of changes; broken is much worse than slow.
• Another commenter pointed out that the combination of automatic semicolon insertion and ignoring whitespace would make it ambiguous if the VBScript-style “dot prefix” notation were adopted in JS. Excellent point!
• For reasons unknown to me, this article has been deleted from the MSDN archive of my original blog. Thanks to the participants in this twitter thread for finding it!

A reader asked me yesterday if there was a way to detect “at compile time” (that is, before the code runs) whether a JScript program contained misspelled variables. I’m
sure we’ve all experienced the pain of

var inveigledFroboznicator = new Froboznicator();
print(inviegledFroboznicator.frabness);

The sooner bugs can be caught, the better, obviously. We catch bugs like missing braces and unterminated strings before the script even runs, so why can’t we catch use of undeclared identifiers? Doesn’t VBScript do that with Option Explicit?

Actually, no, it doesn’t. Visual Basic detects undeclared identifiers at compile time, but VBScript does not catch them until runtime. The reason is because of the way the browser name lookup rules work. Specifically:

• The window object is an expando object. You can add new properties to it at runtime.
• All globally scoped variables and methods in a script block are automatically aggregated onto the window expando.

Aside: A little-known fact is that VBScript allows you to get around the second rule, though why you’d want to is beyond me. It is legal to put the Private keyword on a global declaration; such declarations will not be subsumed onto the window object.

So far there’s nothing stopping us from finding undeclared identifiers at compile time, but then we add:

• The window object’s top-level members are implicitly visible.

And now our dream of compile time analysis vanishes. Any identifier can be added to window at any time by another script block and therefore any identifier is potentially valid in every script block. Add in the fact that the expando can be expanded with a string, and you end up with situations in which no amount of compile time analysis can possibly determine whether an identifier is legal or not. Here’s a really silly example. Suppose we have a web page with two script blogs, one JavaScript, one VBScript. The JavaScript block is:

function dothething()
{  
  window[NewVar] = 123;
}

And the VBScript block is

Option Explicit
Dim NewVar
NewVar = InputBox("Type in a word")
' type in "blah"
dothething()
window.alert blah

This creates a new property on the window object which is not known until the user decides what to type. Since the property is accessible without the window. prefix, there’s no way to know whether the blah identifier is legal until the program actually runs.

Notice that the JavaScript block uses the variable declared in VBScript — it is part of the window object because it is a global, in keeping with our rules laid out above.

Visual Basic proper does not allow access to top-level members of an object without qualification, so it knows that when it sees an undeclared variable, it really is a mistake. (Which reminds me — I wanted to tell you guys about ways to misuse the with block, but that’s another post.)

I suppose that we could have added a feature to JScript like VBScript’s Option Explicit
which catches this problem at run time, but we didn’t. As I mentioned on September 22nd, JScript already throws a runtime error when fetching an undeclared variable. There is still a potential bug due to misspelling on a variable set, so be careful out there.

Also, as Peter reminded me, the JScript .NET compiler in compatibility mode can be used to detect things like undeclared variables, provided that you’re willing to have cases like the one above show up as false positives.

One more thing before I actually go do some real work: an interesting semantic difference between Visual Basic and VBScript is caused by the fact that the declaration check is put off until run time. Because an undeclared variable causes a run time error, On Error Resume Next hides undeclared variables! So again, be careful out there! Don’t
rely on Option Explicitto save you next time you type with mittens on, because if you suppress errors, guess what? Errors are suppressed!

Commentary from 2019:

As I’ve frequently mentioned, the attitude that “the compiler is standing between me and productivity, and therefore should do what I mean” and the attitude “the compiler is my friend who is telling me about my likely mistakes” are opposites, but both are reasonable given the sort of programming you want to do.

In the 1990s we designed JavaScript and VBScript to be “do what I mean” languages, but did not anticipate that someday JavaScript would be used to write huge programs constructed by large teams. The price you pay today for the flexibility of JavaScript is the amount of testing you have to do to ensure correctness; we live in a world where even modern JS type checkers do not guarantee that they’ll find all type errors.

A while back I was discussing the differences between VBScript’s For-Each and JScript’s for-in loops.

A coworker asked me today whether there was any way to control the order in which the for-in loop enumerates the properties. They wanted to get the list in alphabetical order for some reason.

Unfortunately, we don’t support that. The specification says (ECMA 262 Revision 3 section 12.6.4):

The mechanics of enumerating the properties is implementation dependent. The order of enumeration is defined by the object. Properties of the object being enumerated may be deleted during enumeration. If a property that has not yet been visited during enumeration is deleted, then it will not be visited. If new properties are added to the object being enumerated during enumeration, the newly added properties are not guaranteed to be visited in the active enumeration.

Enumerating the properties of an object includes enumerating properties of its prototype, and the prototype of the prototype, and so on, recursively; but a property of a prototype is not enumerated if it is “shadowed” because some previous object in the prototype chain has a property with the same name.

Our implementation enumerates the properties in the order that they were added. This also implies that properties added during the enumeration will be enumerated.

If you want to sort the keys then you’ll have to do it the hard way — which, fortunately, is not that hard. Enumerate them in by-added order, add each to an array, and sort that array.

var myTable = new Object(); 
myTable["blah"] = 123; 
myTable ["abc"] = 456 
myTable [1234] = 789; 
myTable ["def"] = 346; 
myTable [345] = 566; 
var keyList = new Array(); 
for(var prop in myTable) 
      keyList.push(prop); 
keyList.sort(); 
for(var index = 0 ; index < keyList.length ; ++index)
      print(keyList[index] + " : " + myTable[keyList[index]]); 

This has the perhaps unfortunate property that it sorts numbers in alphabetical, not numeric order. If that bothers you, then you can always pass a comparator to the sort method that sorts numbers however you’d like.

Commentary from 2019:

A reader asked whether it was “tedious” to solve the problem that I mentioned in the final paragraph: that numbers are sorted as though they were strings, not numbers. It’s not too hard. The comparison will get strings; see if the operands parse as strings, and if so, come up with some rule. Say, any number is smaller than any non-number, numbers compare as numbers, and non-numbers compare as strings.

This is another example of API design decisions that have a negative impact on developers who have reasonable needs. Every platform should come with a “smart sort” algorithm that knows how to deal with strings that have numbers in them. It’s a slightly tricky algorithm, so let’s have experts write it once, test it heavily, and then line-of-business developers don’t need to try to re-invent it.

VBScript has Null, Empty and Nothing. What about JScript? Unfortunately, JScript is a little screwed up here.

Like VBScript, in JScript an uninitialized variable is a VT_EMPTY variant. To check to see if a variable is undefined, you can say

typeof(x) == "undefined"

There is a built-in Empty constant in VBScript, but not in JScript.

If you say

var x = null;
print(typeof(x));

then you get object, so you might expect that null is the JScript equivalent of Nothing — an object reference that refers to no object. From a logical perspective, you’d be right. Unfortunately, for reasons which are lost to the mists of time, JScript implements a null variable internally the same way that VBScript implements a Null variable — VT_NULL, the “data is missing” value.

This poor decision leads to two problems. First, JScript does not interoperate very well with COM objects which have methods that can legally take a Nothing but not a Null. (I believe that Peter Torr has some addon code that adds this capability to JScript.) Second, JScript processes Null variants passed into it as though they were object references, not database nulls. Thus JScript does not implement any of the rules about null propagation that VBScript implements.

We corrected these problems somewhat in JScript .NET. JScript .NET uses a null object reference to internally represent null, and can manipulate DBNull objects, so the interop problems go away. However, JScript .NET still does not implement the null propagation semantics for mathematical operations on database nulls.

Commentary from 2019:

Reader response to this article was pretty positive, with several readers noting that getting the insider perspective on the implementation decision of programming language minutia was valuable to them. As I’ve said many times before, getting that sort of information available to the public was my whole reason for starting this blog, so I’m glad to have succeeded.

One reader requested a JS linter; recall that this was still 2003 and we did not have the sort of JavaScript infrastructure available that we did today. Peter Torr noted that you could use the JScript.NET compiler as a linter, as its error detection was pretty good. And I noted that I’d be posting more about language and browser design decisions that make linting harder.

Occasionally I get questions from people who are confused over the semantics of data that are not even there. Usually they’ve written code something like

If Blah = Nothing Then

or

If Blah = Empty Then

or

If Blah = Null Then

all three of which almost certainly do not correctly express the actual intention of the
programmer. Why does VBScript have Null, Nothing and Empty, and what are the differences between them?

Let’s start with Empty. When you declare a variable in C, the variable’s value before the first assignment is undefined:

int index;
printf("%d", index); /* could print any integer */

In C, the declaration reserves space for the variable, but does not clear the contents of that space. After all, why would it need to? You’re just going to initialize it to some value yourself, right? Why should the compiler waste time by initializing it only to have that initialization overwritten?

That might seem like a sensible attitude if you are one of those people who prefers that your program be twenty or even thirty nanoseconds faster in exchange for causing any accidental use of uninitialized memory to make your program’s behaviour completely random.

The designers of VB knew that their users were not hard core bit twiddling performance wonks, but rather line-of-business developers who prefer a predictable programming environment. Thus, VB initializes variables as they are declared and eats a few processor cycles here and there. When you declare an integer in VB, it’s initialized to zero, strings are initialized to empty strings, and so on.

But what about variants? Should an uninitialized variant be initialized to zero? That seems bogus; why should an uninitialized variant automatically become a number?

Really what we need is a special of-no-particular-type “I’m an uninitialized variant” value, and that’s Empty. And since in VBScript, all variables are variants, all variables are initialized to Empty.

What if in VB you compare an uninitialized variant to an uninitialized integer? It seems sensible that the comparison would return True, and it does. Empty compares as equal to 0 and the empty string, which might cause false positives in our example above. If you need to detect whether a variable actually is an empty variant and not a string or a number, you can use IsEmpty. (Alternatively, you could use TypeName or VarType, but I prefer IsEmpty.)

Nothing is similar to Empty but subtly different. Empty says “I am an uninitialized variant”; Nothing says “I am an object reference that refers to no object”. Since the equality operator on objects checks for equality on the default property of an object, any attempt to say

If Blah = Nothing Then

is doomed to failure — Nothing does not have a default property, so this will produce a run-time error. To check to see if an object reference is invalid, use

If Blah Is Nothing Then

Null is weirder still. The semantics of Null are very poorly understood, particularly amongst people who have little experience with relational databases. Empty says “I’m an uninitialized variant”, Nothing says “I’m an invalid object” and Null says “I represent a value which is not known.”

Let me give an example. Suppose you have a database of sales reports, and you ask the database “what was the total of all sales in August?” but one of the sales staff has not reported their sales for August yet. What’s the correct answer? You could design the database to ignore the fact that data is missing and give the sum of the known sales, but that would be answering a different question. The question was not “what was the total of all known sales in August, excluding any missing data?” The question was “what was the total of all sales in August?” The answer to that question is “I don’t know — there is data missing”, so the database returns Null.

What happens when you say

If Blah = Null Then

?

Let’s try printing out the value of the comparison and see:

Sales = 123
WScript.Echo Sales = Null

You get not True, not False, but Null! Why’s that? Well, think about the semantics
of it. You’re saying “is the unknown quantity equal to 123?” The answer to that is not “yes”, it’s not “no”, it’s “I don’t know what the unknown quantity is, so, uh, maybe?”

Nulls propagate themselves. Any time you numerically manipulate a Null,
you get a Null right back. Any sum containing an unknown addend has an unknown sum, obviously! The correct way to check for Null is much as you’d do for Empty:
use IsNull (or TypeName or VarType.)

The sharp-eyed among you will have noticed that I never actually answered the question. What does happens when you say

If Blah = Null Then

does VBScript run the consequence block or the alternative (“else”) block? Obviously it has to do one of the two. When it comes right down to it, VBScript will assume falsity in this situation.

The way JScript and JScript .NET handle nulls is a little bit weird; I’ll talk about that in my next entry.

Commentary from 2019:

Most of the comments of this posting were a discussion with a reader who used to use the information in this blog post as interview questions; I pushed back on that, as I want interview questions to elicit signal on skills, not specific trivial knowledge. Sure, you can tell to what level a developer actually knows their tools by that sort of probing, but I am much more interested in a developer’s ability to analyze problems, find solutions, and write solid code. The original reader was mostly looking to interview for contract positions, where there was an expectation of expertise that could lead to immediate productivity; I tend to interview not for immediate productivity, but ability to learn.

Peter Torr pointed out that I had missed one on my list of “missing data” types. VBScript represents a call to a function with an omitted optional argument as a variant of type VT_ERROR with value DISP_E_PARAMNOTFOUND.

I was talking about reference types vs. by-reference variables a while back. Recall that both JavaScript and VBScript have reference types (“objects”) but JavaScript does not have by-reference variables.

COM supports passing variable references around, but unfortunately the intersection of early-bound COM and late-bound IDispatch is a little bit goofy. There are a few problems that you need to be aware of when you’re trying to get VBScript code to talk to COM objects that expect to be passed references to variables. (Obviously attempting to do so from JavaScript is just a non-starter, as JavaScript will not pass variable references at all.)

The most common problem happens when you are trying to call a method on a dispinterface where the vtable entry for the method looks something like this

HRESULT MyFunction([in, out] BSTR * pbstrBlah );

Calling this method via VBScript like this produces a type mismatch error:

Dim MyString
MyString = "foo"
MyObj.MyFunction MyString

And calling it like this does not produce an error but also does not actually pass the value by reference:

Dim MyString
MyString = "foo"
MyObj.MyFunction CStr(MyString)

The latter behaviour is completely expected — what you’re passing is the output of a function, not a reference to a variable. There is no reference available, so no value is filled in. But why does the former fail?

Well, VBScript does not know what the callee is expecting as far as types go. That’s what “late bound” means — the callee has to do the work of determining how to suck the relevant data out of the variants passed to Invoke, and the callee has to somehow call the underlying vtable with the correct types. So VBScript sees

MyObj.MyFunction MyString

and passes a reference to a variant. All variables are variants in VBScript.

Why does VBScript produce a type mismatch error here? VBScript doesn’t! The object produces the type mismatch error, which VBScript dutifully reports. The object’s implementation of Invoke calls the default implementation of Invoke provided for you by the type library implementation. That thing says “I’ve got a reference to a variant, and that variant is a string. I need a reference to a string. That’s a type mismatch.”

This seems like a missed trick; if I were designing such a system, I’d put some additional smarts into it that would handle this case. Clearly from a reference to a variant that contains a string one can obtain a reference to a string — just take the address of the string field of the variant! However, for reasons which are lost to the mists of time, the default implementation does not do that.

My advice to you would therefore be

• If you want to write an object that can be easily used from script, do not have any [in, out] parameters, because JScript can’t pass references.
• If you must have in/out parameters, make them variants, because VBScript can’t pass any other kind of reference.
• If you must have nonvariant in/out parameters, write some fixer-upper code for your IDispatch implementation which transforms byref variants pointing to strings into byref strings. (Or whatever byref type you require.) But if you do that, make sure you get it right.
• Do not attempt to write your own IDispatch implementation, as there are many pitfalls (which I may discuss at another time).

That’s the most common problem I see. The other common problem involves out parameters which are not in/out or out/retval parameters. Just-plain-out parameters cause memory leaks. Consider our earlier example:

Dim MyString
MyString = "foo"
MyObj.MyFunction MyString

Suppose MyFunction takes an in/out variant, and fills in the byref variant with the string “bar”. The implementor expects that something will come in and something will go out, and the rule is that the callee frees the coming-in memory before replacing it with the going-out memory. The caller is then responsible for freeing the going-out value.

But if MyFunction takes a just-plain-out variant then the callee does not free the incoming memory. It assumes that the incoming memory is garbage because it has specifically been told that nothing is coming in.

How does VBScript know whether the callee is in-out or out? VBScript doesn’t! Knowing that requires either compile time knowledge or a very expensive run-time lookup (the results of which are difficult to cache for the same reasons that dispatch ids are difficult to cache.)

The practical result is that if you pass an object or string to a callee expecting an out variant, the string or object pointer will be overwritten without first being freed, and the memory will leak. You should ensure that you always pass empty variants if you must call an object method that takes an out parameter. Yes, this is a violation of the design principle that late-bound calls must have the same semantics as early bound calls, but unfortunately, that’s just the way OLE Automation is and there’s nothing we can do about it now.

Commentary from 2019

Most of the responses to this article were of the form “I wish this had been documented earlier”. Indeed, that is the primary reason why I started this blog! I felt that a lot of this stuff was under-documented in the 1990s, and I belatedly wanted to fix that for posterity.

Now that posterity has arrived, we can see that the design of IDispatch had a number of flaws. An interface specifically designed for “late binding” does not need to provide a mechanism for callees to interrogate callables to determine the types expected, though it sure would be nice to have. But the fact that the caller needs to know the calling convention in order to avoid memory leaks, and there is no cheap way for the caller to interrogate the object to find out what it expects, is terrible! It forces the script engine developer to trade off between bad performance on the one hand, and a memory leak on the other.

This also illustrates one of the costs of runtimes that lack garbage collection; think about how much effort have we put into designing and implementing memory management protocols in COM APIs. None of that work had to be done for C# APIs, freeing the developer to concentrate on the business semantics rather than the storage mechanisms.

After the previous episode, a reader pointed out to me that FormatNumber uses yet a different rounding algorithm. FormatNumber rounds numbers ending in .5 away from zero, not towards evens. The reader also asked whether FormatNumber, FormatCurrency, and so on, actually call into the VBA Format$ code.

It does not. The VBA runtime is freely redistributable, but it is also large. Back in 1996 we did not want to force people to download the large VBA runtime. IE was on our case for every kilobyte we added to the IE download.

However, FormatNumber, and the other related functions were written by the same developer who implemented Format$ in VBA. That was the legendary Tim Paterson, who you may recall wrote a little program called QDOS that was eventually purchased by Microsoft and called MS-DOS.

And let me tell you, when I was an intern in 1993 I had to debug Format$. There are very good reasons why we decided to go with a stripped-down let’s-solve-the-90%-usage-case version for VBScript! Format$ is one of the most enormously complicated hard-core-bit-twiddling functions that I’ve ever seen. That’s what happens to these “everything but the kitchen-sink” functions that tries to be all things to all callers — they get enormously bloated and complex. Porting that thing to a new codebase and maintaining it independently would have been nightmarish.

Here’s a comment that Tim wrote into the FormatNumber code on September 16th, 1996 which confirms the reader’s observation:

// We have more digits that we're not using. Round if needed.
// This is simplistic rounding that does not use IEEE
// rules (rounding even if exactly x.5). This simple
// way is how VBA Format$ works, so it's good enough for us.

When I was first an intern who barely knew how to program real-world code, Tim Paterson was very intimidating to me, and he did not suffer foolish questions gladly. But once I got to know him, it turned out that he’s a great guy. I had many enjoyable conversations with him about the early days of Microsoft. I particularly remember walking with him to the launch event for Windows 95, reminiscing about how much operating systems had changed since the QDOS days.

Incidentally, Format$ was about 5500 lines of C++ code (counting blank lines and comments). About 1300 of those lines are the parser that turns the “format picture” into an internal representation. It’s an entire programming language of its own, practically.

I cordially hate the “format picture” approach to formatting strings for this reason; we’ve created a programming language that follows none of the standards, rules or idioms of the “host” language. Much of the work of Format$ was parsing a language that never should have existed in the first place; instead we should have had an object model of combinators that could be combined programmatically. But we’re stuck with this 1970s era idiom now.