One of the easiest bugs to write is the dreaded memory leak.  You allocate some chunk of memory and never release it.  Those of us who grew up writing application software might sometimes have a cavalier attitude towards memory leaks — after all, the memory is going to be reclaimed when the process heap is destroyed, right?  But those days are long gone.  Applications now often run for days, weeks or months on end.  Any memory that is leaked will slowly consume all the memory in the system until it dies horribly.  Web servers in particular are highly susceptible to leaks, as they run forever and move a lot of memory around with each page request.

The whole point of developing a garbage collected language is to decrease the burden on the developer.  Because the underlying infrastructure manages memory for you, you don’t have to worry about introducing leaks.  Of course, that puts the burden squarely upon the developer of the underlying infrastructure: me.  As you might imagine, I’ve been called in to debug a lot of memory leaks over the years.  The majority turned out to be in poorly written third-party components, but a few turned out to be in the script engines.

I mentioned a while back that ASP uses a technique called “thread pooling” to increase its performance.  The idea is that you maintain a pool of idle threads, and when you need work done, you grab one from the pool.  This saves on the expense of creating a new thread and destroying it when you’re done with it.  On a web server where there may be millions of page requests, the expense of creating a few million threads is non-trivial.  Also, this ensures that you can keep a lid on the number of requests handled by one server — if the server starts getting overloaded, just stop handing out threads to service requests.

I think I also mentioned a while back that JScript has a per-thread garbage collector.  That is, if you create two engines on the same thread, they actually share a garbage collector.  When one of those engines runs a GC, effectively they all get collected.

What do these things have to do with each other?

As it turns out, there is a memory leak that we have just discovered in the JScript engine.  A small data structure associated with the garbage collector is never freed when the thread goes away.  What incredible irony!  The very tool we designed to prevent your memory leaks is leaking memory.

Oh, but it gets worse.

As it turns out, this leak has been in the product for years. Why did we never notice it?  Because it is a per-thread leak, and ASP uses thread pooling! Sure, the memory leaks, but only once per thread, and ASP creates a small number of threads, so they never noticed the leak.

Why am I telling you this? Because for some reason, it never rains but it pours. We are suddenly getting a considerable number of people reporting this leak to us. Apparently, all of a sudden there are third parties developing massively multi-threaded applications that continually create and destroy threads with script engines. Bizarre, but true. They are all leaking a few dozen bytes per thread, and a few hundred thousand threads in, that adds up.

I have no idea what has led to this sudden uptick in multithreaded script hosts. But if you’re writing one, let me tell you two things:

1. We’re aware of the problem.  Top minds on the Sustaining Engineering Team are looking at it, and I hope that we can have a patched version for a future service release.
2. Use thread pooling! Not only will it effectively eliminate this leak, it will make your lives easier in the long run, believe me.

This whole thing reminds me that I want to spend some time discussing some of the pitfalls we’ve discovered in performance tuning multi-threaded applications.  But that will have to wait for another entry.

Commentary from 2019:

This post attracted a few “war stories” from readers:

Consider writing a “watch dog” process that watches your server; when it looks like it is leaking too much memory, just cleanly shut it down and restart it, problem solved.

Though indeed this is a common approach, and one taken by ASP, ideally we’d prefer to not leak resources in the first place. But as I noted, often the leaks are in third-party components that are hard to fix.

Leaks are exacerbated by the number of allocators that must be matched with their deallocators.

No kidding; it happens all the time. If you take ownership of a pointer, you have to know not just that it is valid memory, but how to deallocate it! That becomes part of the contract, but that contract is hard to describe or enforce.

Some language designer, I don’t remember who, once said that designers are torn between two competing goals: “X is important, so I’m going to make this so flexible that anyone can implement their own X that meets their needs”, and “X is important, so I’m going to implement one that does it right”. And that’s why in C we have no built-in string type, and no built-in heap memory allocator, because those things are so important that of course you want a dozen different crappy implementations of each that don’t interoperate but are confusingly similar. We had the second attitude in C#; make the runtime implement allocation and users can stop thinking about it.

In console games or other software for limited-memory embedded systems, you don’t worry about leaks because you don’t allocate memory dynamically in the first place.

I think I mentioned this in my series on writing a Z-Machine implementation. There’s no “heap allocator” per se; there’s a block of a few dozen KB of memory and you do with it as you see fit!

Thank goodness Joel wrote this article — that means that I can cross it off of my list
of potential future blog entries. Thanks Joel!

Fortunately the script engines are entirely Unicode inside.  Making sure that the script source code passed in to the engine is valid UTF-16 is the responsibility of the host, and as Joel mentions, IE certainly jumps through some hoops to try and deduce the encoding.  WSH also has heuristics which try to determine whether the file is UTF-8 or UTF-16, but nothing nearly so complex as IE.

I should mention that in JScript you can use the \u0000 syntax to put unicode codepoints into literal strings.  In VBScript it is a little trickier — you need to use the CHRW method.

Commentary from 2019:

I did not mention in this article that the original implementations of VBScript and JScript were for IE running on 16 bit Windows, and 16 bit Windows did not support Unicode. We had non-Unicode versions of the scripting toolchain for quite some time. I wrote a lot of string library code for dealing with DBCS and other odd character encoding problems when I was first at Microsoft as a full-time employee.

There were a couple of good reader questions:

What do we do with code points above FFFF ?

VBScript and JScript use UTF-16 as their string encoding, so you can represent higher codepoints with surrogate pairs.

Is that also true of JScript.NET?

Yes, JS.NET, and all the .NET languages, use UTF-16 internally also.

As I’ve discussed previously, the script engines always talk to objects on the late-bound IDispatch interface.  The point of this interface is to allow a script language to call a method or reference a property of an object by giving the name of the field and the arguments.  When the dispatch object is invoked, the object does the work of figuring out which method to call on which interface.

But what if an object supports two interfaces IFoo and IBar both of which you want to be able to call late-bound?  The most common way to solve this problem is to create two late-bound interfaces, IFooDisp and IBarDisp.  So when you ask the object for IFooDisp, you get an interface that can do late-bound invocation on IFoo, and similarly for IBarDisp.

What happens when you ask the object for an IDispatch interface?  It’s got to pick one of them!  The one it picks is called the “default dispatch”.

In JScript and VBScript, when you create an object (via new ActiveXObject in JScript or CreateObject​ in VBScript) the creation code always returns the default dispatch.  Furthermore, in JScript, when you fetch a property on an object and it returns a dispatch object, we ask the object to give us the default dispatch.  So in JScript, there is no way to script a non-default dispatch.

What about in VBScript?  There’s an irksome story here, again featuring a really bad mistake made by yours truly.  At least I meant well.

I didn’t write the variant import code, and I had always assumed that VBScript did the same thing as JScript — when an object enters the script engine from outside, we query it for its default dispatch.  As it turns out, that’s not true.  In VBScript, for whatever reason, we just pass imported dispatch objects right through and call them on whatever dispatch interface the callee gives us.

One day long ago we found a security hole in IE.  The details of the hole are not important — but what’s interesting about it was the number of things that had to go wrong to make the hole an actual vulnerability.  Basically the problem was that one of the built-in IE objects had two dispatch interfaces, one designed to be used from script and one for internal purposes.  The for-script dispatch interface was the default, and it was designed to participate in the IE security model.  The internal-only interface did not enforce the IE security model, and in fact, there was a way to use the object to extract the contents of a local disk file and send it out to the internet, which is obviously badness. Furthermore, there was a way to make another object return the non-default interface of the broken object.

JScript did not expose this vulnerability because it always uses the default dispatch even when given a non-default dispatch.  But the vulnerability was exposed by VBScript.  All these flaws had to work together to produce one vulnerability.  Now, when you find a security hole that consists of multiple small flaws, the way you fix it is not to just patch one thing and hope for the best.  You patch everything you possibly can.  Remember, secure software has defense in depth.  Make the attackers have to do twelve impossible things, not one impossible thing, because sometimes you’re wrong about what’s impossible.

Hence, when we fixed this hole, we fixed everything.  We made sure that the object’s persistence code could no longer read files off the local disk.  In case there was still a way to make it read the disk that the patch missed, we fixed the object model so that it never returned a non-default interface to a script.  In case there was still a way to do both those things that we missed, we also turned off VBScript’s ability to use non-default dispatches.

That last one turned out to be a huge mistake.  The fact that I had believed that VBScript always talked to the default dispatch does not logically imply that every VBScript user read my mind and knew that successfully using a non-default dispatch was some sort of fluke.  People naturally assume that if they write a program and it works, then it works because the language designers wanted them to be able to write that program!

As it turned out, there were plenty of programs out there in the wild that used VBScript to script non-default dispatch interfaces returned by method calls, and I broke all of them through trying to make IE safer.  We ended up shipping out a new build of VBScript with the default dispatch feature turned back on a couple of days later.  (Of course all the fixes to IE were sufficient to mitigate the vulnerability on their own, and those were not changed back.)

The morals of the story are:

1. If you want to use a non-default dispatch, you have to use VBScript.
2. There is no way to make VBScript give you a specific non-default dispatch.  In VBScript, once you have a dispatch, that’s the one you’re stuck with.
3. Defense in depth is a good idea, but it’s not such a good idea to go so deep that backwards compatibility is broken if you can avoid it!

Commentary from 2019

Here we have yet another episode where I learned from a mistake. Of course the best mistakes to learn from are other people’s, but I’ll learn from my own if I have to.

This episode produced some good technical questions in the comments:

Can VBScript call .NET managed objects without using a COM-Callable Wrapper?

Nope. It’s called a COM-Callable Wrapper because it’s what you do to call the object from COM, and VBScript is COM.

How do tearoff dispatch interfaces fit into this?

First, let me describe the technique, since a lot of readers are not familiar with it.

Normally in COM you have an object implemented as a C++ object, and when you QueryInterface it for an interface, what happens internally is the object casts this to the desired interface and hands back a ref’d pointer; that is, the C++ object actually implements all of the interfaces that the object can be QI’d for. But that is not a requirement; it is perfectly legal for QI to return a different C++ object that implements the desired interface, so long as the rules of COM are followed. (Like, QI’ing for IUnknown on what is logically one object always produces the same pointer, and so on.)

This implementation strategy of “tearoff interfaces” has performance implications in certain scenarios, like you have an object that implements a lot of interfaces but you do not want to spend time initializing the vtables for all of them every time you create an instance.

Objects are free to implement interfaces using tearoffs if they like, but there is no support in VBScript for converting a given dispatch to another dispatch, regardless of whether it is implemented as a tearoff or not.

The usual trick in this case is to implement a property on the object that hands you the torn-off interface, and if you do that, this technique will work.

A common and entirely sensible programming practice is to put commonly used utility functions in one file, and then somehow link that file in to many different programs. In traditional compiled languages you can compile a bunch of utilities into a statically linked .LIB file or a dynamically linked .DLL file.  But what do you do in script, where a program is just text?

People often ask me why there is no #include statement in VBScript or JScript.  Wouldn’t it be nice to be able to write up a page full of helpful utility functions and then in half a dozen different scripts say  #include myUtilityFunctions.vbs ?

If you think for a bit about how such a statement would be implemented, you see why we didn’t do it.  The reason is pretty simple: the script engine does not know where the current file came from, so it has no way of finding the referenced file.  Consider IE, for example:

<script language="vbscript">
#include myUtilityFunctions.vbs
' ...

How does the script engine know that this page was downloaded from any particular site? The script engine knows nothing about where the script came from — IE just hands the text to the engine.  Even assuming that it could figure it out, the script engine would then have to call IE’s code to download files off the internet. Worse, what if the path was fully qualified to a path in a different domain? Now the script engine needs to implement IE’s security rules.

Now consider: do any of those factors apply when running in ASP?  No!  ASP would have to have a completely different include mechanism where the relative path was based on facts about the current virtual root and the security semantics checked for cross-root violations.

What about WSH? There we’re worried about the current directory but have no security considerations.

What about third party hosts?  Every host could have a different rule for where the script comes from and what is a legal path.  We cannot possibly put all those semantics into the script engine.  To make an “include” feature work, we’d have to define a callback function into the host so that the host could fetch the included script for us.

It’s just too much work for too little gain, so we simply declared a rule: the host is responsible for implementing any mechanism whereby script files can be included into the script namespace.  This is why IE and WSH have the script src​ attribute.

Commentary from 2019:

Following on the heels of “Why can’t I create the WScript object?” we have another example of younger me focusing on the restrictions imposed by the implementation architecture choices rather than focusing on the user need that drives the question.

As I noted, we could have prioritized this scenario and implemented that callback, but we had many higher priorities, including improvements to performance and robustness that benefit everyone.

Commenters were quick to point out these germane critiques; my rather weak response was that the script engines tried to not be in the business of placing requirements upon the host.

That’s weak because the hosts had to solve the problem of how to find related scripts anyways, and they also had to come up with their own mechanisms for users to specify locations. Moreover, we could have made the “find a file to include” feature optional, thereby reducing the burden on all hosts.

The net result is that every script host has its own unique way of specifying “include this file”, and that created its own set of user confusions.

A stronger response is that the seemingly simple feature comes with a large set of implementation difficulties. For example, how do we detect and prevent hostile or accidental circular includes? How does the feature interact with the debugger? Are includes processed at parse time or run time? And many more; it’s not in any way a trivial feature.

From our perspective today, the whole question seems quaint. Web sites routinely download a whackload of JS modules, and there are whole framework subsystems devoted to figuring out which modules are needed, and so on.

Following up on this morning’s entry, a reader asked me why Windows Script Components don’t have access to the WScript object.  “it is running in an instance of WSH isn’t it?”

No, it isn’t.  That’s a common misperception.  Let me clear it up.

The whole point of scripting is to make it easy to write programs.  Two very common kinds of programs are executables and components.  I’m sure that you understand the difference between the two, but just let me emphasize that executables are “standalone” — they have a well defined startup, they run for a while, and then they shut down.

Components are libraries of useful functionality packaged up as objects.  These objects cannot live on their own — they have to be created by an executable or another component, and they live as long as the owner wants them to.

We provide two ways to write executables in script — Windows Script Host files (with extensions .VBS, .JS, .WSF) and HTML Applications (with extension .HTA).  When you run one of these, the host application starts up, loads the script, runs it, and then shuts down.

We also provide a way to write components in script — the aptly named Windows Script Components.

When you create a WSC, its no different from creating any other in-process COM object.  The fact that the object is implemented in script is irrelevant.

People get WSCs and WSF’s confused because of their similar names and syntaxes.  But logically, WSCs and WSH are completely separate entities.

Note though that under the covers, quite a bit of the WSF processing code is actually implemented in ScrObj.DLL, the WSC engine, which itself is consumed as a component by the WSH executable.  We saw no reason to implement two identical XML parsers, two debugger interfaces, etc, when we had one sitting right there in a DLL already.

Every now and then someone will ask me why the WSH shell object and the WSH network object are creatable from Visual Basic, but the actual root WScript object is not.

I am always completely mystified by why people ask this. Why would you want to
create the WScript object in a VB app?  What would it do for you?  It isn’t creatable as an in-process object because it represents the state of a WSH process, and a VB app is not a WSH process. The whole point of the WScript object is that it represents everything about the presently running Windows Script Host.

You can get from it all kinds of properties: the name and version of the script host, the path of the running script, the arguments which were passed in, the standard streams of the process, and whether the script was set to time out or not.  None of these properties make the slightest bit of sense to access outside of a running instance of WSH! What the heck could this possibly mean?

Set WScript = CreateObject("WSH.WScript")
Timeout = WScript.TimeOut

The timeout of what?  You’ve just got some object there, not a running instance of the script host.

What about the methods on the object?  Again, all of them are deeply tied into the underlying script hosting mechanisms.

WScript.CreateObject, GetObject, ConnectObject and DisconnectObject manipulate WSH-owned event sinks on the script dispatch.  I discussed yesterday how deeply tied WScript.Sleep is to the underlying hosting mechanism, and WScript.Echo is not particularly interesting — there are better ways to create a message box in VB already.

I just don’t get it.  Why do people keep asking me to make the WScript object creatable?  I can’t figure out what they think they’re going to do with it once they’ve created it and no one ever gives me a straight answer when I ask. It’s like asking for the ability to create the Response object without starting ASP!  What could it possibly mean to have a Response object but no web server process?  It doesn’t make any sense, so we don’t let you do it.

We do provide an object model whereby you can spawn off WSH processes (both locally and remotely) and query their status as they run.  Perhaps that is what these people are looking for?  Remote WSH is a complicated topic in of itself, so perhaps I’ll talk about that in a later blog entry.

Commentary from 2019:

This article produced a lot of reader feedback, much of it very frustrated.

I learned a lot from these responses. Microsoft deservedly got a lot of flack from people who resented being characterized as “Morts”; that was a terrible choice of name for an important and valuable classification of developers. But it really is the case that a significant fraction of VB and VBScript users view error messages not as the compiler and runtime helpfully pointing out their problems, but as the thing standing between them and their success.

This article also showed how much I still lacked the “beginner’s mind” at this point in my career. I was too close to the implementation details of the system; I was quite genuine when I said that I didn’t understand why so many people wanted this thing, which was to my expert mind both impossible and useless. Why would anyone want something that was obviously both impossible and useless? But of course what they wanted was a smart and understandable solution to their immediate line-of-business problems.

Were I attacking this problem space today, I’d more likely choose a design that was better factored into orthogonal, re-usable subsystems. The alphabet soup of scripting technologies that we gave users was confusing and under-documented.

A point which I did not make in this article was that the scripting team was somewhat constrained in that we did not actually write Windows Script Host originally; another team at Microsoft created it for their own purposes, and made a number of design and implementation decisions that were not what the script team would have done. We inherited that pile of code and needed to hammer it into a shape that solved the general problem. In retrospect, it might have been a better idea to start the design process from scratch, rather than trying to mold an existing tool into the shape we thought users wanted.

Summarizing a few of the points:

I need an “eval” method that takes a bunch of VBScript or JScript, executes it, and gives me the results. How should I do that?

That’s what the Microsoft Script Control is for.

Do Windows Script Components run inside an instance of Windows Script Host?

No; WSCs are just COM objects that happen to run script; they are not WSH instances.

There are useful services that WSH provides that I’d like to see usable outside of WSH, like argument parsing.

I wrote the argument parser, so thanks for that; I tried hard to make it useful and I agree that it probably should have been made available as a stand-alone service.

I want to be able to run WSH scripts outside of WSH.

You can’t run web client scripts outside of the browser, or web server scripts outside of the server; similarly, you can’t run WSH scripts outside of WSH. WSH scripts depend on specific implementation details of WSH in order to execute correctly.

I get “permission denied” errors when trying to use the Remote WSH Controller to run WSH scripts from another machine.

Good! We worked hard to make the security model of Remote WSH very locked down. We did not want to be a new vector for attacks. To use Remote WSH, you must explicitly turn on DCOM, and then also explicitly turn on Remote WSH, and then to use it, you must be a member of the administrator group on the target machine.

We did not want to ever be in a situation where someone turned on DCOM for some unrelated reason and suddenly exposed an invisible, unknown attack vector that allowed an attacker to run an arbitrary script on a machine remotely!

WSH scripts all run in their own process; I’d like to be able to run scripts all in one process, and use WScript.Sleep

Again we come back to my point that I made in a previous episode: to write a correct Sleep you need to understand how all the other code in the same process will react to that sleeping job. If you want to write scripts that all run in the same process, use the script control; if you want some of those scripts to sleep then you will have to write an implementation of the sleep method that has the semantics you want.

At Microsoft we have an internal email list called “Spot the Defect” — people mail around buggy code they’ve discovered and we compete to see who can find the most problems with it. It’s fun, and you learn a lot about what other people consider bugs — everything from security holes to lying comments.

I love playing Spot the Defect. Here is the code for the WScript.Sleep method with the comments removed and some serious bugs added. You’ll note that this code has all the required features I mentioned in my previous post.

We go to sleep in one-second (or less) intervals, and tell the operating system to wake us up if COM posts a message to the message queue, because there might be an event handler to dispatch. We also check to see if the host recorded a script error (either due to an event handler or due to the script timeout firing) so that we can abort the sleep. This way we never keep the script alive more than a second after it was shut down due to error.

What bugs did I add?

HRESULT CWScript::Sleep(long Time)
{
  const DWORD TimerGranularity = 1000;
  if (Time < 0)
    return E_INVALIDARG;
  DWORD StartTickCount = ::GetTickCount();
  DWORD EndTickCount = StartTickCount + Time;
  DWORD CurTickCount = StartTickCount;
  while(CurTickCount < EndTickCount)
  {
    MSG msg;
    DWORD CurWaitTime = (DWORD)(EndTickCount - CurTickCount);
    if (CurWaitTime > TimerGranularity)
      CurWaitTime = TimerGranularity;
      ::MsgWaitForMultipleObjects(
        0, NULL, TRUE, CurWaitTime, QS_ALLINPUT | QS_ALLPOSTMESSAGE);
      if (0 != ::PeekMessage(&msg, NULL, 0, 0, PM_REMOVE))
        ::DispatchMessage(&msg);
      if (m_pHost->FErrorPending())
        return S_OK;
      CurTickCount = ::GetTickCount();
  }
  return S_OK;
}

Commentary from 2019:

Readers from 2019 who are used to reading C# may need the hint that long and DWORD are both 32 bit integers.

I don’t recall why we had this strange convention of UpperCasing local variables; the WSH codebase was actually taken over by the scripting team from another team that had developed it for their own use, and there were a number of odd choices in that codebase. I suspect this was one of them; I’m a big believer in keeping new work consistent with old work, even if you wouldn’t use the conventions of the old work in brand-new green-field code.

There were a lot of good responses to this puzzle:

Is it legal to call MsgWaitForMultipleObjects with zero handles?

Yes, in regular desktop or server Windows. A user pointed out that this is not legal in the CE version of Windows.

After MsgWaitForMultipleObjects returns, you should pump all waiting messages, not just one.

Great catch! That bug was actually in the code; that’s not the bug I added for this puzzle.

I sent the bug on to the scripting sustaining engineering team.

It looks like there is an overflow bug at StartTickCount + Time​

There is; can you be more specific?

Suppose the machine has been on for just under 49.7 days, which is when GetTickCount() overflows, and adding Start overflows. The end time will be less than the start time, so it will not sleep at all.

That is a bug, but there is a worse one.

Suppose CurTickCount and EndTickCount are very close to overflowing, and the second time you set CurTickCount it is set to a small value after the overflow. The function will then sleep for 49.7 days.

That’s correct, but it’s actually worse than that; it will sleep for at least 49.7 days, and then it has a possibly very narrow “window” of tick counts that will avoid sleeping for another 49.7 days. If that window is small then we are likely to sleep for a long, long time.

The real code was written to take all of these problems into account; it does all of its timing arithmetic in doubles, which have more than enough range and precision to avoid overflow issues, and detects clock overflows correctly.

Yes, the title is grammatical. A few days ago I was discussing the semantics of data that isn’t there. Today I want to talk a little about programs that do nothing. What do you do when you want a program to pause briefly, for whatever reason?

Those of you who like me cut your programming teeth on Commodore PET Basic know the answer. A for loop will do the trick.

for(x = 0 ; x < 1000000 ; ++x) {}

Of course, that has the problem that the time spent paused is shorter on faster machines. The obvious improvement waits until a given number of milliseconds have passed:

for(var startTime = new Date(); new Date() - startTime > 1000 ; ) {}

The problem with both these techniques is that this is busy waiting.  The program may appear to be paused but it is actually executing as furiously as ever.  The processor is probably pegged to 100% but doing entirely pointless work.  In a multithreaded, multiprocess operating system busy waiting is downright rude — the processor could be tending to other tasks like redrawing the screen or background processing.

Really what you want to do is to ask the operating system to put the process asleep for a bit and then wake it up later, right?  That way the program pauses but the thread scheduler is free to run something more important. And indeed, there’s a Win32 API that does just that, Sleep.

But there is no way to call win32 APIs directly from script, so the scriptable “go to sleep” method needs to either be built in to the language, or (the horror!) in an ActiveX object. So why the heck do VBScript and JScript not have a “go to sleep” method built in?

There were good reasons why we didn’t. The principal reason is because I lied above.  You do not want to put the process to sleep and wake it up again later, because while the script is “sleeping”, you still might want event handlers to run.

The way events work in Apartment Threaded COM objects such as initialized script engines (see my earlier entries for a refresher on how the script engine threading model works) is pretty simple. When the user clicks on a button and raises a button event, what actually happens is a bunch of messages are deposited in the thread’s message queue.  When the message queue is pumped, the messages tell the COM plumbing to call the event sinks listening to the event sources.

What happens when a thread is asleep? Nothing! In particular, no message loop runs on the thread. How could it?  It’s asleep. So you put your script to sleep for ten seconds, a user presses a button, and then waits at least ten seconds for the message loop to get pumped. That seems kind of suboptimal. The sleep method needs to pump a message loop occasionally so that events get handled.

But wait, it gets worse.

What if one of those event sinks causes a script error, which is then reported to the host, and the host decides to shut down the script engine? Ten seconds later, the script wakes up and keeps running! Oh, the pain.

To properly implement a “go to sleep” method you need to know all kinds of details about the desired message processing, event handling, error handling and multi-threading semantics of the host application.  We were very worried that we’d add a method to the script engines which, when used in IE, or ASP, or WSH, or some third-party script host, completely screw up the carefully implemented host.

Hence, there is no “go to sleep” method in the languages.  If the host implementor thinks that a “go to sleep” method is necessary, they can implement one and add it to their object model, as we did for WSH.

Commentary from 2019:

A reader pointed out that you can use a combination of window.setTimeout and window.showModalDialog in the browser to sleep the UI while still allowing event handling and data binding to work.

I’m sure there are better ways to do that in modern asynchronous JavaScript, but I have not kept up with advances in JS asynchrony so I don’t know what they are.

However, the original comment points out exactly what I was talking about in the article; setTimeout knows exactly what the internal message pumping mechanisms are in the browser, and can therefore do it correctly; the sensible place to put any kind of “delay” function is in the code that knows how to manage it. We designed the script engines to be hosted in browsers, servers, games, and so on, and did not want to put anything into the language that could be badly abused in those environments.

I didn’t mention DoEvents in this article, which is often used (badly!) to make busy-loops that handle events; I believe I discussed why that is almost always wrong when I discussed adding asynchrony to C#.

I’m having my kitchen redone.

Yes, I remember that in my very first blog entry I said that I would be talking about only technical matters, not trivia about my life.  Bear with me, people.

When I say “redone” I mean that I took a crowbar, a sledgehammer and a couple of helpful housemates, pulled down all the walls, pulled up the floor, trucked the asbestos-laden vinyl flooring to the dump and then invited over a bunch of big burly guys to build a new kitchen from scratch.

Unfortunately, we found a number of “structural issues” once the walls were apart which necessitated some pretty major infrastructure improvements.  I was hoping to have all the work in the kitchen done in early October, but these various non-scheduled improvements ate up a lot of time.  I got an email from the general contractor the other day saying that the revised schedule now has us finishing up in October — October 45th to be precise.  Ha ha ha.  Very funny guys.

Update: We finished on January 16th, taking exactly twice as long as initially forecast, and confirming the old saying that a poorly planned project takes three times as long as you think, a well planned project only twice as long.

The good news is that this reminded me that when I dissed the VBScript date format a few weeks ago, I never got around to dissing the similarly goofy JavaScript date code.  October 45th?  No problem!

var d = new Date(2003, 9, 45);
print(d);

prints out

Fri Nov 14 00 :00:00 PST 2003

What the heck?  “2003, 9, 45” is the 14th of November?

Yep.  First of all, for reasons which were never adequately explained to me by my cohorts at Netscape, the date constructor numbers dates starting from zero.  So “9”, not “10”, is October.  If you want to know why that is, ask Brendan next time you see him, ’cause I sure don’t know. (UPDATE: See below.)

But what’s more interesting is that any attempt to schedule projects like my contractor made result in real dates.  November 14th is, in some sense, the best candidate for October 45th, as it is 44 days after October 1st.   You can get weirder than that, of course.

var d = new Date(2003, -1, 0);

prints out

Wed Nov 30 00:00:00 PST 2002

Which is only logical — if January was month zero of 2003 then December 2002 was the “negative first” month of 2003.  And if December 1st was the 1st day of December, then November 30th must have been the 0th day. Similarly, November 29th was the -1st day, and so on.

The underlying implementation details of JScript dates are quite a bit more sensible than VBScript’s implementation.  JScript also stores dates as double-precision 64 bit floats, but the JScript implementation stores all dates as the number of milliseconds since midnight, 1 January 1970 UTC.  Universal Time Coordinated is the proper name for what most people call “Greenwich Mean Time” — the time at the prime meridian, with no adjustment for British Summer Time.

For all practical purposes, we treat the 64 bit float as a 53 bit integer plus a sign bit.  This means that JScript has 300000 years of millisecond accuracy on either side of the epoch, as opposed to VBScript where the precision varies over the range.  Also, in this system there are no two dates with the same numeric representation, which makes comparing, subtracting, adding and rounding times straightforward.

Also, JScript, unlike VBScript, adjusts for daylight savings time and time zone differences according to the local information reported by the operating system.

There are some weirdnesses in JavaScript though.  The most bizarre is the getYear method, which returns a one-or-two-digit number for all dates from 1900 to 1999, and a four-digit number for dates before 1900 or after 2000.  Again, why JavaScript has a function that obviously causes Y2K-style errors implemented a few years before Y2K is unknown to me — ask your favourite Netscape language designer next time you meet one at a party.

Microsoft didn’t make the problem any better when we screwed up our implementation of getYear — the original IE implementation returns the year minus 1900, so 2000 was year 100.  Fortunately we corrected that incompatibility in the second version.  You should never use getYear — use getFullYear, which always returns all the necessary digits.

More generally, you might wonder if there are design principles underlying the decision to make “October 45th” a legal date.  Indeed there are (though to be perfectly frank, when it comes to the date handling, I suspect that any “justification” might be somewhat post hoc.)

Here are some thoughts that go through the minds of language designers:

• The sooner you produce an error for bogus behaviour, the sooner the developer will catch the bug.  Therefore, at the first sign of anything out of the ordinary, crash and die immediately.
• But wait a minute — the more errors we produce, the more error handling code the developer will have to write, because some of those bogosities will be produced by end user data.
• Worse, if the developer misses an error handling case then the error will be reported to the end user.
• The end user of a web page script is someone who has absolutely no ability to understand the error.  If they do understand the error, they have no ability to go to the web server and fix it.  Therefore, never produce error messages unless you absolutely have to.

The ability to start with a premise and deduce its opposite is a neat trick, you must admit!  But seriously, the existence of conflicting goals explains why there is more than one programming language. Hard-core application development languages like C# are designed to trap as many bugs as possible at compile time and run time so that you can find them, write the handling code, and ship.  C# demands that you type your variables, that you call reflection explicitly, that you mark unchecked arithmetic, and so on.

But C# is a language designed for building hundred-thousand-line application frameworks and object models, not hundred line web page scripts.  C# is designed for professional developers, not hobbyists and departmental web developers who just want to get “glue code” written as rapidly as possible.

That’s why JavaScript’s attitude towards error cases is not “die at the first sign of trouble” but rather “muddle along as best you can”.  Assign to an undeclared variable?  Declare it dynamically.  Forget a semicolon?  Insert it.  Try to create a date for the 31st of September?  Move it to October 1st.  Divide by zero?  Return a NaN .  Reference an element beyond the end of an array?  Grow the array.  All these things that would be compile time or runtime errors in other languages are just handled for you in JScript, for better or for worse.

And this is why when people come to me and say “my team of twenty developers has written a hundred-thousand-line database program in JScript and now I can’t make head nor tail of it, help!” I tell them to use the right tool for the job!  C# was designed for those tasks.  Heck, JScript .NET was designed for those tasks!  But a weakly-typed, interpreted, fail-silently, late-bound, non-generational-GC language like JScript was designed to add scripting to simple web pages.  If I give you a really great hammer then you can build a decent house but not a decent skyscraper.

Updates from 2019:

This article produced some responses from noted individuals. Raymond Chen pointed out:

Looking back over my entries in the last couple of weeks, I see that I’ve got a bit of a theme going here.  A number of entries have been about programming language features that exist in both JScript and VBScript, but have semantics just dissimilar enough to trip up the unwary.  Well, here’s another one! What’s the difference between this VBScript:

Function Rezrov(Foo)
  ' whatever
End Function
' ...
With Frobber.Blorple
  .Frotz 123, 456
  Rezrov .Cleesh
End With

and this JScript:

function Rezrov(Foo)
{
  // whatever
}
with(Frobber.Blorple)
{
  Frotz(123, 456);
  Rezrov(Cleesh);
}

?  At first glance, both look pretty straightforward.  But there is an important difference. The VBScript example is exactly equivalent to this program:

Set Temp = Frobber.Blorple
Temp.Frotz 123, 456
Rezrov Temp.Cleesh

whereas the JScript is logically equivalent to this pseudo-JScript:

var temp = Frobber.Blorple;
if (temp has property frotz)
  temp.frotz(123, 456);
else
  frotz(123, 456);
if (temp has property Rezrov)
{
  if (temp has property Cleesh)
    temp.Rezrov(temp.Cleesh);
  else
    temp.Rezrov(Cleesh);
}
else
{
  if (temp has property Cleesh)
    Rezrov(temp.Cleesh);
  else
    Rezrov(Cleesh);
}

As you can see, if Frobber.Blorple actually has a Rezrov member, these two programs do completely different things.  In the VBScript program, you know (and the compiler knows) whether you are calling a property on the With object or not.  In the JScript program, you don’t know and the compiler doesn’t know either.  The compiler has to emit code that does the namespace lookup at runtime.

This is extremely costly!  Consider for example what happens to local variables:

function foo(){
  var abc = 123;
  with(Frobber.Blorple)
  {
    Rezrov(abc);
  }
}

Is that a reference to local abc or to Frobber.Blorple.abc?  Like yesterday, because of the potential of expando objects the compiler cannot tell.  Normally in this case the compiler would generate a binding to the exact offset in the local scope table, but since the compiler cannot do that, it must instead generate a by-name lookup of the entire scope chain.

It gets worse.  Suppose you provide the Frobber.Blorple object and one day you want to add a property to it called Blah.  How can you do so safely? You can’t!  Every user of your object who put it in a with block and who has a local or global variable called Blah used in that with block is going to suddenly break.  Every time you add (or remove) a property, you run the risk of changing the variable lookup semantics of an existing program.  And the converse holds as well — every time you use a with block, you run the risk of accidentally binding to the wrong thing when someone changes the object properties.

The with block in JScript is a terrible programming construct.  It makes your programs slower, harder to understand, brittle in the face of changes and generally more bug prone.   Do not use it! 

The VBScript With block on the other hand, by the simple expedient of requiring a leading period on all references to the object, does not have these problems.

For a discussion of the pros and cons of the feature in VB, this discussion on the Fog Creek software forum has input from both me and from the designer of the feature, Joel Spolsky. https://discuss.fogcreek.com/joelonsoftware2/50125.html

Commentary from 2020:

• I stand by my opinion that “with” in JS is harmful; it’s a bad feature that should never have been added in the first place. It is deprecated in ECMAScript 6.
• The tradition of calling out bad programming language features by writing a post titled “X Considered Harmful” is a snowclone. I’m happy to participate in this tradition.
• A commenter reminisced about the pleasures of playing Enchanter, Sorceror and Spellbreaker from Infocom; many of the metasyntactic names in this blog are from those games.
• I also reinforced in the comments that the fundamental problem is not that “with” in JS makes your variable lookups hundreds of times slower — odds are good there is even slower stuff in your JS program! The problem is that it makes programs hard to read and reason about, and it makes them brittle in the face of changes; broken is much worse than slow.
• Another commenter pointed out that the combination of automatic semicolon insertion and ignoring whitespace would make it ambiguous if the VBScript-style “dot prefix” notation were adopted in JS. Excellent point!
• For reasons unknown to me, this article has been deleted from the MSDN archive of my original blog. Thanks to the participants in this twitter thread for finding it!