It’s my girlfriend Leah’s 30th birthday today! Happy birthday Leah!

Leah is a tester in the Mobility division here at Microsoft, where she works on the software that moves email between servers and cellphones. Right now she and her fellow testers are boning up on their C# skills so that they can write better testing automation scripts. Surprisingly, a lot of the sync software testing is still done “by hand”, rather than writing a program to run the same test suites over and over again every day.

The other day Leah pointed out to me something that I’ve been intending to write about — that often it really doesn’t matter what language you learn.  For many programmers, the language is just the thing that stands between you and the object model.  People writing application automation need to be able to create instances of classes, call methods, listen to events, run loops, and dump strings to log files.  Learning the syntax for those operations in JScript, VBScript, VB, C#, C++, perl (shudder), Python, etc, is trivial.  The complexity lies in understanding the object model that you’re manipulating so that you can use it well to test the product.

The same goes for the vast majority of scripting applications.  People ask me “Eric, I’m a newbie programmer — should I learn VBScript or JScript?”  I tell them that script is glue and that what matters is that you glue the right things together, not that you pick the right glue.

That’s not to say that there aren’t important differences between languages.  As I mentioned the other day, some languages are designed to support programming in the large, some are designed to facilitate rapid development of small, throwaway programs. Some are for writing device drivers, some are for research, some are for science, some are for artificial intelligence, some are for games.  If you have complex structures that you wish to model, it’s a good idea to pick a language that models them well.  Prototype classes (JScript) are quite different from inheritance classes (C#), which in turn are different from simple record classes (VBScript).  But my point is that by the time you’re ready to write programs that require these more advanced features, you’ll be able to pick up new languages quickly anyway.

And this is also not to say that automation testing is just glue code. I’ve had many long conversations with the testers on my team on the subject of writing automation tools.  When you move from the low level automation code (call this method, did we get the expected response?) to the higher-level (run tests 10, 13 and 15 against the VB and C# code generators for the Word and Excel object models on 32 and 64 bit machines but omit the 64 bit flavours of test 15, we already know that its broken) then you start to run into much deeper problems that may require their own object models to represent.  Or even their own languages!  One of the testers I work with is kicking around some ideas for a “test run definition language” that would cogently express the kinds of massive test matrices that our testers struggle with.

But these are not newbie programmer problems.  If you’re just getting into this whole scripting thing, pick a language and use it to learn the object model inside-out.  Once you know how the OM works, doing the same thing in a different language should be pretty straightforward.

It’s kind of like table tennis.  If you know the rules of table tennis, learning the rules of real tennis is pretty easy — it’s table tennis, just with a larger board.  And you stand on the board, and the ball is bigger, as are the racquets.  But, as George Carlin said, it’s basically the same game.  And if you know tennis, volleyball is pretty easy — it’s just tennis with nine people on a side and you can hit the ball three times and it can’t hit the ground.  And there are no racquets, and the ball is bigger and the net is higher, and you play to 21 points. But it’s basically the same game.

OK, maybe that’s not such a good analogy.  But you take my point, I’m sure.  Don’t stress about choice of language, but learn the object model cold.  The question shouldn’t be “what language should I learn” but rather “what object framework solves my problem, and what languages are designed to efficiently use that framework?”

Reflections from 2019:

Leah is of course no longer my girlfriend, as we’ve been married since 2005. The mobile devices division at Microsoft has been reorganized so many times that any vestige of its 2003 state is I’m sure long gone.

It was a fascinating time to be involved in mobile devices; the iPhone was still years away, and there was real competition for who was going to be a player in this market. There was always a tension between the hardware manufacturers, the operating system vendors like Microsoft, and the cell service providers; each one had an interest in differentiating their offerings from their competitors while commoditizing the others.

That is to say, AT&T wished desperately to be seen as different than, say, Sprint, but not differentiated at all on the basis of the quality of the hardware or the software; they wanted people to differentiate based on cell service, since that is what they provided. The hardware manufacturers were in the same boat; they wanted customers to differentiate on the basis of hardware features, not what networks were supported. And the same for Microsoft, which wanted customers to think about the OS provider as the differentiating factor when making a purchasing decision, not the hardware or the cell service.

This misalignment of interests led to some considerable lack of trust between the various parties — who ought to have been collaborating for each other’s mutual success.

This article produced a number of thoughtful reader responses:

Speaking of books, people keep telling me and Peter and Raymond that we should write books based on our blogs.

I probably am going to write another book this winter, but it will have pretty much nothing to do with the stuff in this blog.   The natures of a blog and a book are very different.  Let me list some differences:

Creative control

I can blog what I want, when I want, at what length I want, and can say whatever I want. In particular, I like to ramble on about the scripting technologies — which, though they are widely used, are clearly a 20th century technology.  .NET is the future.  A book has to be on a specific topic, finished by a specific time, at a specific length.  A book has to be about a current technology topic and have a clear beginning-middle-end structure. Books both allow editing and require editing.  Blogs resist editing.

Business model

Books work on the ink-on-dead-trees business model.  Weblogs work on the “bits are free” business model. If I went to a publisher and said “I want to write a short but rambling book about random, arcane, trivial details of the history and internals of a 1996 technology that is presently being made increasingly irrelevant and outmoded by .NET” then the publisher would say “thanks, but no thanks”.   People buy computer books because they have a problem that needs solving, not because they enjoy learning my opinions about proper Hungarian usage.

Books must make money to exist.  My aim for this blog isn’t to make money, it is to dump my vast collection of arcane knowledge into some searchable location.

Scope of readership

My blog is available to everyone in the world with a web browser, and given the subject matter, that’s everyone I want to reach.  Books are available to only the very small number of people who actually buy the book.  If you like my book and you want your friend in Europe to read it, you can’t just send them a link.  Again, books cost money and that limits the potential readership.

Permanence

My book is no longer available because of circumstances beyond my control.  Now, Microsoft isn’t going to go out of business, but if they did, I could just move the blog file to another machine in about five minutes and be back up and running. This blog will be archived and therefore part of the permanent searchable record of knowledge on the internet. The copies of my book in the Library of Congress (and whatever the British equivalent is) aren’t going to help a whole lot of devs.

And finally, apropos of nothing in particular, this is hilarious:  http://mama.indstate.edu/users/bones/WhyIHateWebLogs.html, mostly because it is so self-referential.  One wonders what category the author himself falls into.  Thank goodness my blog falls under one of his acceptable uses of blogs!  I don’ t know how I could continue to face myself in the mirror every day without this guy’s approval.

Commentary from 2019:

The most obvious thing I missed in this rant was the rise of electronic books as a viable business model, which mitigates many of the anti-book factors I mentioned here.

Raymond Chen did of course write a book based on his blog. Peter Torr I believe never did.

The book I mentioned that I was going to be working on was my first VSTO book.

I still edit other people’s books, but I am down to mostly my two favourites: Essential C#, and C# In Depth.

I’m still not super bullish on writing more programming books; I feel like in a world where we’re connected to the internet all the time, that writing a book about learning to program is no longer the best approach. Online interactive tutorials seem like a much better way to go; the question is, how to monetize them? It is an enormous amount of work to develop such a curriculum, and that should be compensated.

Joel and Ned are having a spirited debate over the merits of exception handling. Oddly
enough, I agree with both of them.

The only time I do programming in C++ is to write COM code, and COM and exceptions do not mix.  The only way to make COM and C++ exceptions mix is to use smart pointers, and as I’ve already discussed, that only makes the situation worse by introducing subtle, impossible-to-find-and-debug bugs in the error handling.

When I write COM code every function call returns an HRESULT and I carefully ensure that if there is an error, then everything gets cleaned up properly.  COM is fundamentally an environment which is based on error return values for error handling.  Any additional error state can be provided by stateful objects.

But the .NET CLR is a completely different environment, one with an exception model built deep into the runtime itself and into the framework.  You’d be crazy to not use the exception model in a fully garbage-collected environment like the CLR.  I write a lot of C++ and C# code that has to work together via Interop; ALL of my C++ functions returns an error code, and none of my C# functions do.  That is not to say that there is no error handling in my C# code — far from it!  My code is chock full of try-catch-finally blocks to deal with errors.

Programming languages and object frameworks are tools that afford different styles of programming — attempting to mix the style appropriate to one in another is a recipe for unmaintainable code.

Commentary from 2019:

I always meant to do a post showing my style for writing COM programs in C++ without using smart pointers. It was an extremely clear, rigid style that guaranteed that I would not create resource leaks; basically I was simulating exception handling with macros, with exactly one “catch” per method. I don’t recall if I ever wrote that article; if I did, I guess I’ll find it while I’m porting these posts.

In retrospect I no longer entirely agree with the hot take in the final paragraph. C# and F# are both perfectly usable languages that blend different coding styles. That said, I would not want to do a line-for-line rewrite of a C# program in F#, or vice-versa; that really would be unmaintainable.

There was a good question in the comments about the performance of exception handling. My response was that there are two aspects to exception handling performance:

• What is the additional over-all burden imposed by exceptions? An interesting question, but not actually relevant to making a decision. Why? Because in the .NET framework, there is no way to “turn off” exceptions. You’re stuck with this infrastructure. If you can’t change it, there’s no reason to worry about it. If you cannot turn it off then the question is not “are exceptions fast enough in the CLR?” but rather “is the CLR fast enough?”
• What’s the cost of throwing one exception? This second concern I don’t care about one bit. Exceptions are, by definition, exceptional. When one of my programs gets an exception, almost always it is either doing something incredibly expensive already (in which case the cost of the throw is irrelevant) or we are going to report the exception to the user (in which case the program is going to come to a halt.) Either way, the per-throw cost is unimportant.

However, I missed a trick here. There is an additional cost, which is: what is the cost not of throwing an exception, but catching an exception? The jitter generates more code in a method with a try-catch or try-finally, which means that it has less time available in its budget for optimization. And those optimizations get harder to perform because there is more complex control flow to consider.

Still, I wouldn’t give up exception handling and go back to return codes. There are a number of design changes I would have liked to see in the exception system though. But that is a topic for another day.

A while back a reader asked for a rundown on some of the design decisions we made when designing JScript .NET.  That’s a huge topic, but fortunately I started writing a book on the subject a few years ago that never found a publisher. Tell you what — whenever I can’t think of something more interesting to post, I’ll put snippets of it up on my blog.

There were four main design goals for JScript .NET:

1. JScript .NET should be an extension of JScript. This means that whenever possible a legal JScript program should be a legal JScript .NET program. JScript .NET must continue to be usable as a dynamic scripting language. Further, it should be easy to take an existing JScript Active Server Page and transform it into a JScript .NET page for ASP.NET.
2. JScript .NET should afford the creation of highly performant programs. In particular, JScript .NET should work well with ASP.NET because performance is often critical in server-side applications. Also, JScript .NET should warn programmers when they use a language feature which could adversely impact performance.
3.    JScript .NET should work well with the .NET Common Language Runtime (CLR). The CLR provides interoperability with other languages. It also ensures that JScript .NET works well with the .NET Frameworks, a huge library of useful packages.
4. JScript .NET should make programming in the large easier. It should support programming styles that make it easy to reason about large programs. It should also provide programming constructs which make it easy to divide up large projects among team members.

Today I’ll talk a bit about the second point — how can we make JScript .NET faster than JScript? (I know, I said a while back that I’d give you my rant about performance and script languages.  Maybe if I have time tomorrow.)

JScript .NET takes a three-pronged approach towards improving performance. Individually each would help somewhat, but as is often the case, the whole is greater than the sum of its parts. These three prongs work together to afford the creation of more performant programs:

1. JScript .NET has a type annotation system.
2. JScript .NET discourages and/or restricts certain programming idioms which cause egregious performance problems.
3. JScript .NET uses the Common Language Runtime.

The type annotation system can be complex when you look at the details — I’ll probably talk more about it in detail later.  But essentially the type annotation system is quite simple: the programmer attaches annotations to variable declarations, function argument lists and function return values. These annotations restrict the kinds of data which may be stored, passed or returned respectively.

Consider the simple example of adding two and two:

var Alpha = 2, Beta = 2;
// [ many lines of code omitted ]
var Gamma = Alpha + Beta;

Though Alpha and Beta start off their lives as numbers who knows what they are by the time Gamma is declared? Some code may have stored a string or a date or any other kind of data into Alpha or Beta.

Ironically, adding two and two thereby becomes a complicated mess. The JScript engine must determine at run time whether a string has been assigned to either Alpha or Beta (or both). If so then they must be both converted to strings and concatenated. If neither is a string then both must be converted to numbers and added.

This sounds simple enough but glosses over a number of details. For instance, the conversions are themselves possibly extremely complicated depending on what data have been stored in Alpha and Beta. Suffice to say that a seemingly simple operation like addition can take thousands of machine cycles, entire microseconds in some cases.

This is unfortunate. If there is one thing that computers are good at it is adding numbers blindingly fast. If the JScript .NET compiler could somehow know that only numbers could possibly be stored in Alpha and Beta then there would be no need to determine the types at run time. The compiler could simply emit instructions optimized for the case of adding two numbers. This operation could take nanoseconds instead of microseconds.

In JScript .NET you annotate the type of a variable by appending a colon and the type after the declaration.

var Alpha : Number = 2, Beta : Number = 2;
// [many lines of code omitted ]
var Gamma : Number = Alpha + Beta;

The code generated for this addition should now be much more efficient.

Type annotation also allows function calls to be much more efficient. Consider a simple function call on a string:

var Delta = 123;
// [many lines of code omitted ]
var Zeta = Delta.toString();

Again, though Delta starts off as a number it could be anything by the time toString is called. This is what we mean by a late bound scenario. The type of Delta is unknown, so at run time the object must be searched to see if it has a method named toString. This search is potentially expensive. It could be far more efficient if the compiler knew ahead of time that Delta should be converted to a string the same way that all numbers are converted to strings.

var Delta : Number = 123;
// [many lines of code omitted ]
var Zeta : String = Delta.toString();

Note that the JScript .NET compiler has an automatic type inference engine. If the compiler sees that a local variable in a function only ever has, say, strings assigned to it then the compiler will treat the variable as though it was annotated with the String type. However, it is a poor programming practice to rely upon the type inference engine. Certain programming practices (especially the use of the eval function) prevent the inferrer from making valid inferences. Also the inferrer can only make inferences on local variables; the types of unannotated global variables are not inferred.

There are also other good reasons to add annotations rather than relying on the inferencer to do it for you, but those will have to wait for another entry.

Commentary from 2019:

We’re coming up on a long stretch in this blog that is going to make me a little sad to revisit. We worked so hard on JScript .NET because we strongly believed in both the value proposition of the .NET framework, and that Microsoft needed to make a major investment in JavaScript. I feel like we were doing the right work at the wrong time, because the company was not amenable to making the sorts of investments that were needed.

Of course, Microsoft ended up developing the Chakra engine for the browser many years later at great expense. Eventually the value of a type-annotated JavaScript became apparent and a lot of effort was put into designing and implementing TypeScript. (Which is awesome; it is everything I dreamed of doing with JScript, just ten years later than I wanted to do it.)

It is still frustrating to me, over fifteen years later. Microsoft could have saved so much time and effort in the long run, and provided so much customer value, and really taken leadership in this tools space; it was obvious to me that it was going to be an important space and that we needed to be involved, but for reasons which were never made clear to me, all our efforts in this area were defunded. Microsoft was then left to play catch-up for the next decade, in a position of weakness of its own making. We had an optionally-typed, high-performance (for its time) JS engine that interoperated with C#; why would you defund that? In retrospect, that was obviously a strategic asset.

Anyways, expect more rants as I port over these posts.

I mentioned earlier that I wrote one of the last books published by the now-bankrupt Wrox Press. A sharp-eyed coworker who happened to be in China the other day sent me a copy of this:

Holy cow, I had no idea that Wrox translated my book into Chinese! No one ever told me that such a thing was even in the works. Just when I thought that the Wrox saga couldn’t get more bizarre, this turns up. Thanks Zhanbo! I’ll treasure it always.

A number of people have expressed surprise to me that Wrox is out of business.  Here’s the scoop.

Wrox was owned by a company called Peer, which also owned Friends Of Ed, Glasshaus, and a number of other small businesses in the “developer education” field.

Peer made a big mistake:  they failed to time the end of the dot com bubble economy.  They ended up with massive overproduction of educational materials for developers that suddenly were not sellable. Returns from bookstores were large, and they ended up in big financial trouble.  They managed to keep the company afloat for a few more months, but were eventually unable to get new loans to service the interest on their old loans, which is by definition bankruptcy.

All of this happened very suddenly.  I was actually in the process of writing content for their web site and tech reviewing books up to the day before they announced their insolvency.  As it turned out, I was pretty lucky.  At least my book was published and I was actually paid my advance, though I never received any additional royalties.  All in all, Wrox only owed me a few thousand dollars when they went under.  There were plenty of people who had spent months writing books that were ready for the presses — those people never saw a dime for their work.

Wrox had two main creditors. They owed the bank of Scotland a lot of cash, and also had a large loan from AOL-Time-Warner-Netscape against all existing stocks of their books as collateral.   Under British law (similar to American law), all creditors get their collateral when the default happens, and everyone else (employees owed wages, authors owed royalties, and so on) gets in a line as the remaining assets are auctioned off.  Unsecured creditors like yours truly go to the end of the creditor line, and shareholders get in line after them.

Of course, since Peer had millions of pounds in debt and no assets save the existing stocks, all of which went to AOL-Time-Warner-Netscape, I didn’t even bother getting in on the action, as there was no action to get in on.

Since AOL-Time-Warner-Netscape owned the book stocks, one might wonder what happened to the royalties when they sold them.  Well, news flash: royalties are a share of profits, and no one made a profit on the sale of those books.  They were certainly all sold at a loss.

Wiley bought the rights to the dozen or so best sellers and the Wrox web site.  APress bought the rights to everything else, and may bring out APress editions of former Wrox books.

If you want to buy a copy of my book, you are probably out of luck.  I may ask Gary at APress if I can just reformat it into HTML and put it up on the web, if they don’t want to bring out an APress edition.  After all, I didn’t write the thing to make money.

Commentary from 2019:

I found a web site that had two dozen copies of my Wrox book and bought all of them for cheap. Amazon apparently also has some copies left. I gradually gave them away as joke gifts over the next few years and I think I only have one copy left. My mother probably also has a copy somewhere.

Wiley owns the Wrox trademark and continues to publish books under it.

Of course AOL-Time-Warner-Netscape is also long dead. Younger readers may have no clear memory of the bizarre time that was the pre-widespread-broadband internet economy. AOL, a dial-up internet company, managed to get big enough to acquire both Time-Warner and Netscape before flaming out spectacularly when broadband became a thing. AOL was spun back off into its own company in 2009, and was recently acquired by Verizon for reasons which surpass my understanding.

One of the easiest bugs to write is the dreaded memory leak.  You allocate some chunk of memory and never release it.  Those of us who grew up writing application software might sometimes have a cavalier attitude towards memory leaks — after all, the memory is going to be reclaimed when the process heap is destroyed, right?  But those days are long gone.  Applications now often run for days, weeks or months on end.  Any memory that is leaked will slowly consume all the memory in the system until it dies horribly.  Web servers in particular are highly susceptible to leaks, as they run forever and move a lot of memory around with each page request.

The whole point of developing a garbage collected language is to decrease the burden on the developer.  Because the underlying infrastructure manages memory for you, you don’t have to worry about introducing leaks.  Of course, that puts the burden squarely upon the developer of the underlying infrastructure: me.  As you might imagine, I’ve been called in to debug a lot of memory leaks over the years.  The majority turned out to be in poorly written third-party components, but a few turned out to be in the script engines.

I mentioned a while back that ASP uses a technique called “thread pooling” to increase its performance.  The idea is that you maintain a pool of idle threads, and when you need work done, you grab one from the pool.  This saves on the expense of creating a new thread and destroying it when you’re done with it.  On a web server where there may be millions of page requests, the expense of creating a few million threads is non-trivial.  Also, this ensures that you can keep a lid on the number of requests handled by one server — if the server starts getting overloaded, just stop handing out threads to service requests.

I think I also mentioned a while back that JScript has a per-thread garbage collector.  That is, if you create two engines on the same thread, they actually share a garbage collector.  When one of those engines runs a GC, effectively they all get collected.

What do these things have to do with each other?

As it turns out, there is a memory leak that we have just discovered in the JScript engine.  A small data structure associated with the garbage collector is never freed when the thread goes away.  What incredible irony!  The very tool we designed to prevent your memory leaks is leaking memory.

Oh, but it gets worse.

As it turns out, this leak has been in the product for years. Why did we never notice it?  Because it is a per-thread leak, and ASP uses thread pooling! Sure, the memory leaks, but only once per thread, and ASP creates a small number of threads, so they never noticed the leak.

Why am I telling you this? Because for some reason, it never rains but it pours. We are suddenly getting a considerable number of people reporting this leak to us. Apparently, all of a sudden there are third parties developing massively multi-threaded applications that continually create and destroy threads with script engines. Bizarre, but true. They are all leaking a few dozen bytes per thread, and a few hundred thousand threads in, that adds up.

I have no idea what has led to this sudden uptick in multithreaded script hosts. But if you’re writing one, let me tell you two things:

1. We’re aware of the problem.  Top minds on the Sustaining Engineering Team are looking at it, and I hope that we can have a patched version for a future service release.
2. Use thread pooling! Not only will it effectively eliminate this leak, it will make your lives easier in the long run, believe me.

This whole thing reminds me that I want to spend some time discussing some of the pitfalls we’ve discovered in performance tuning multi-threaded applications.  But that will have to wait for another entry.

Commentary from 2019:

This post attracted a few “war stories” from readers:

Consider writing a “watch dog” process that watches your server; when it looks like it is leaking too much memory, just cleanly shut it down and restart it, problem solved.

Though indeed this is a common approach, and one taken by ASP, ideally we’d prefer to not leak resources in the first place. But as I noted, often the leaks are in third-party components that are hard to fix.

Leaks are exacerbated by the number of allocators that must be matched with their deallocators.

No kidding; it happens all the time. If you take ownership of a pointer, you have to know not just that it is valid memory, but how to deallocate it! That becomes part of the contract, but that contract is hard to describe or enforce.

Some language designer, I don’t remember who, once said that designers are torn between two competing goals: “X is important, so I’m going to make this so flexible that anyone can implement their own X that meets their needs”, and “X is important, so I’m going to implement one that does it right”. And that’s why in C we have no built-in string type, and no built-in heap memory allocator, because those things are so important that of course you want a dozen different crappy implementations of each that don’t interoperate but are confusingly similar. We had the second attitude in C#; make the runtime implement allocation and users can stop thinking about it.

In console games or other software for limited-memory embedded systems, you don’t worry about leaks because you don’t allocate memory dynamically in the first place.

I think I mentioned this in my series on writing a Z-Machine implementation. There’s no “heap allocator” per se; there’s a block of a few dozen KB of memory and you do with it as you see fit!

Thank goodness Joel wrote this article — that means that I can cross it off of my list
of potential future blog entries. Thanks Joel!

Fortunately the script engines are entirely Unicode inside.  Making sure that the script source code passed in to the engine is valid UTF-16 is the responsibility of the host, and as Joel mentions, IE certainly jumps through some hoops to try and deduce the encoding.  WSH also has heuristics which try to determine whether the file is UTF-8 or UTF-16, but nothing nearly so complex as IE.

I should mention that in JScript you can use the \u0000 syntax to put unicode codepoints into literal strings.  In VBScript it is a little trickier — you need to use the CHRW method.

Commentary from 2019:

I did not mention in this article that the original implementations of VBScript and JScript were for IE running on 16 bit Windows, and 16 bit Windows did not support Unicode. We had non-Unicode versions of the scripting toolchain for quite some time. I wrote a lot of string library code for dealing with DBCS and other odd character encoding problems when I was first at Microsoft as a full-time employee.

There were a couple of good reader questions:

What do we do with code points above FFFF ?

VBScript and JScript use UTF-16 as their string encoding, so you can represent higher codepoints with surrogate pairs.

Is that also true of JScript.NET?

Yes, JS.NET, and all the .NET languages, use UTF-16 internally also.

As I’ve discussed previously, the script engines always talk to objects on the late-bound IDispatch interface.  The point of this interface is to allow a script language to call a method or reference a property of an object by giving the name of the field and the arguments.  When the dispatch object is invoked, the object does the work of figuring out which method to call on which interface.

But what if an object supports two interfaces IFoo and IBar both of which you want to be able to call late-bound?  The most common way to solve this problem is to create two late-bound interfaces, IFooDisp and IBarDisp.  So when you ask the object for IFooDisp, you get an interface that can do late-bound invocation on IFoo, and similarly for IBarDisp.

What happens when you ask the object for an IDispatch interface?  It’s got to pick one of them!  The one it picks is called the “default dispatch”.

In JScript and VBScript, when you create an object (via new ActiveXObject in JScript or CreateObject​ in VBScript) the creation code always returns the default dispatch.  Furthermore, in JScript, when you fetch a property on an object and it returns a dispatch object, we ask the object to give us the default dispatch.  So in JScript, there is no way to script a non-default dispatch.

What about in VBScript?  There’s an irksome story here, again featuring a really bad mistake made by yours truly.  At least I meant well.

I didn’t write the variant import code, and I had always assumed that VBScript did the same thing as JScript — when an object enters the script engine from outside, we query it for its default dispatch.  As it turns out, that’s not true.  In VBScript, for whatever reason, we just pass imported dispatch objects right through and call them on whatever dispatch interface the callee gives us.

One day long ago we found a security hole in IE.  The details of the hole are not important — but what’s interesting about it was the number of things that had to go wrong to make the hole an actual vulnerability.  Basically the problem was that one of the built-in IE objects had two dispatch interfaces, one designed to be used from script and one for internal purposes.  The for-script dispatch interface was the default, and it was designed to participate in the IE security model.  The internal-only interface did not enforce the IE security model, and in fact, there was a way to use the object to extract the contents of a local disk file and send it out to the internet, which is obviously badness. Furthermore, there was a way to make another object return the non-default interface of the broken object.

JScript did not expose this vulnerability because it always uses the default dispatch even when given a non-default dispatch.  But the vulnerability was exposed by VBScript.  All these flaws had to work together to produce one vulnerability.  Now, when you find a security hole that consists of multiple small flaws, the way you fix it is not to just patch one thing and hope for the best.  You patch everything you possibly can.  Remember, secure software has defense in depth.  Make the attackers have to do twelve impossible things, not one impossible thing, because sometimes you’re wrong about what’s impossible.

Hence, when we fixed this hole, we fixed everything.  We made sure that the object’s persistence code could no longer read files off the local disk.  In case there was still a way to make it read the disk that the patch missed, we fixed the object model so that it never returned a non-default interface to a script.  In case there was still a way to do both those things that we missed, we also turned off VBScript’s ability to use non-default dispatches.

That last one turned out to be a huge mistake.  The fact that I had believed that VBScript always talked to the default dispatch does not logically imply that every VBScript user read my mind and knew that successfully using a non-default dispatch was some sort of fluke.  People naturally assume that if they write a program and it works, then it works because the language designers wanted them to be able to write that program!

As it turned out, there were plenty of programs out there in the wild that used VBScript to script non-default dispatch interfaces returned by method calls, and I broke all of them through trying to make IE safer.  We ended up shipping out a new build of VBScript with the default dispatch feature turned back on a couple of days later.  (Of course all the fixes to IE were sufficient to mitigate the vulnerability on their own, and those were not changed back.)

The morals of the story are:

1. If you want to use a non-default dispatch, you have to use VBScript.
2. There is no way to make VBScript give you a specific non-default dispatch.  In VBScript, once you have a dispatch, that’s the one you’re stuck with.
3. Defense in depth is a good idea, but it’s not such a good idea to go so deep that backwards compatibility is broken if you can avoid it!

Commentary from 2019

Here we have yet another episode where I learned from a mistake. Of course the best mistakes to learn from are other people’s, but I’ll learn from my own if I have to.

This episode produced some good technical questions in the comments:

Can VBScript call .NET managed objects without using a COM-Callable Wrapper?

Nope. It’s called a COM-Callable Wrapper because it’s what you do to call the object from COM, and VBScript is COM.

How do tearoff dispatch interfaces fit into this?

First, let me describe the technique, since a lot of readers are not familiar with it.

Normally in COM you have an object implemented as a C++ object, and when you QueryInterface it for an interface, what happens internally is the object casts this to the desired interface and hands back a ref’d pointer; that is, the C++ object actually implements all of the interfaces that the object can be QI’d for. But that is not a requirement; it is perfectly legal for QI to return a different C++ object that implements the desired interface, so long as the rules of COM are followed. (Like, QI’ing for IUnknown on what is logically one object always produces the same pointer, and so on.)

This implementation strategy of “tearoff interfaces” has performance implications in certain scenarios, like you have an object that implements a lot of interfaces but you do not want to spend time initializing the vtables for all of them every time you create an instance.

Objects are free to implement interfaces using tearoffs if they like, but there is no support in VBScript for converting a given dispatch to another dispatch, regardless of whether it is implemented as a tearoff or not.

The usual trick in this case is to implement a property on the object that hands you the torn-off interface, and if you do that, this technique will work.

A common and entirely sensible programming practice is to put commonly used utility functions in one file, and then somehow link that file in to many different programs. In traditional compiled languages you can compile a bunch of utilities into a statically linked .LIB file or a dynamically linked .DLL file.  But what do you do in script, where a program is just text?

People often ask me why there is no #include statement in VBScript or JScript.  Wouldn’t it be nice to be able to write up a page full of helpful utility functions and then in half a dozen different scripts say  #include myUtilityFunctions.vbs ?

If you think for a bit about how such a statement would be implemented, you see why we didn’t do it.  The reason is pretty simple: the script engine does not know where the current file came from, so it has no way of finding the referenced file.  Consider IE, for example:

<script language="vbscript">
#include myUtilityFunctions.vbs
' ...

How does the script engine know that this page was downloaded from any particular site? The script engine knows nothing about where the script came from — IE just hands the text to the engine.  Even assuming that it could figure it out, the script engine would then have to call IE’s code to download files off the internet. Worse, what if the path was fully qualified to a path in a different domain? Now the script engine needs to implement IE’s security rules.

Now consider: do any of those factors apply when running in ASP?  No!  ASP would have to have a completely different include mechanism where the relative path was based on facts about the current virtual root and the security semantics checked for cross-root violations.

What about WSH? There we’re worried about the current directory but have no security considerations.

What about third party hosts?  Every host could have a different rule for where the script comes from and what is a legal path.  We cannot possibly put all those semantics into the script engine.  To make an “include” feature work, we’d have to define a callback function into the host so that the host could fetch the included script for us.

It’s just too much work for too little gain, so we simply declared a rule: the host is responsible for implementing any mechanism whereby script files can be included into the script namespace.  This is why IE and WSH have the script src​ attribute.

Commentary from 2019:

Following on the heels of “Why can’t I create the WScript object?” we have another example of younger me focusing on the restrictions imposed by the implementation architecture choices rather than focusing on the user need that drives the question.

As I noted, we could have prioritized this scenario and implemented that callback, but we had many higher priorities, including improvements to performance and robustness that benefit everyone.

Commenters were quick to point out these germane critiques; my rather weak response was that the script engines tried to not be in the business of placing requirements upon the host.

That’s weak because the hosts had to solve the problem of how to find related scripts anyways, and they also had to come up with their own mechanisms for users to specify locations. Moreover, we could have made the “find a file to include” feature optional, thereby reducing the burden on all hosts.

The net result is that every script host has its own unique way of specifying “include this file”, and that created its own set of user confusions.

A stronger response is that the seemingly simple feature comes with a large set of implementation difficulties. For example, how do we detect and prevent hostile or accidental circular includes? How does the feature interact with the debugger? Are includes processed at parse time or run time? And many more; it’s not in any way a trivial feature.

From our perspective today, the whole question seems quaint. Web sites routinely download a whackload of JS modules, and there are whole framework subsystems devoted to figuring out which modules are needed, and so on.