Unknown's avatar

About ericlippert

http://ericlippert.com

I Take Exception To That

Joel and Ned are having a spirited debate over the merits of exception handling. Oddly
enough, I agree with both of them.

The only time I do programming in C++ is to write COM code, and COM and exceptions do not mix.  The only way to make COM and C++ exceptions mix is to use smart pointers, and as I’ve already discussed, that only makes the situation worse by introducing subtle, impossible-to-find-and-debug bugs in the error handling.

When I write COM code every function call returns an HRESULT and I carefully ensure that if there is an error, then everything gets cleaned up properly.  COM is fundamentally an environment which is based on error return values for error handling.  Any additional error state can be provided by stateful objects.

But the .NET CLR is a completely different environment, one with an exception model built deep into the runtime itself and into the framework.  You’d be crazy to not use the exception model in a fully garbage-collected environment like the CLR.  I write a lot of C++ and C# code that has to work together via Interop; ALL of my C++ functions returns an error code, and none of my C# functions do.  That is not to say that there is no error handling in my C# code — far from it!  My code is chock full of try-catch-finally blocks to deal with errors.

Programming languages and object frameworks are tools that afford different styles of programming — attempting to mix the style appropriate to one in another is a recipe for unmaintainable code.


Commentary from 2019:

I always meant to do a post showing my style for writing COM programs in C++ without using smart pointers. It was an extremely clear, rigid style that guaranteed that I would not create resource leaks; basically I was simulating exception handling with macros, with exactly one “catch” per method. I don’t recall if I ever wrote that article; if I did, I guess I’ll find it while I’m porting these posts.

In retrospect I no longer entirely agree with the hot take in the final paragraph. C# and F# are both perfectly usable languages that blend different coding styles. That said, I would not want to do a line-for-line rewrite of a C# program in F#, or vice-versa; that really would be unmaintainable.

There was a good question in the comments about the performance of exception handling. My response was that there are two aspects to exception handling performance:

  • What is the additional over-all burden imposed by exceptions? An interesting question, but not actually relevant to making a decision. Why? Because in the .NET framework, there is no way to “turn off” exceptions. You’re stuck with this infrastructure. If you can’t change it, there’s no reason to worry about it. If you cannot turn it off then the question is not “are exceptions fast enough in the CLR?” but rather “is the CLR fast enough?”
  • What’s the cost of throwing one exception? This second concern I don’t care about one bit. Exceptions are, by definition, exceptional. When one of my programs gets an exception, almost always it is either doing something incredibly expensive already (in which case the cost of the throw is irrelevant) or we are going to report the exception to the user (in which case the program is going to come to a halt.) Either way, the per-throw cost is unimportant.

However, I missed a trick here. There is an additional cost, which is: what is the cost not of throwing an exception, but catching an exception? The jitter generates more code in a method with a try-catch or try-finally, which means that it has less time available in its budget for optimization. And those optimizations get harder to perform because there is more complex control flow to consider.

Still, I wouldn’t give up exception handling and go back to return codes. There are a number of design changes I would have liked to see in the exception system though. But that is a topic for another day.

Designing JScript .NET

A while back a reader asked for a rundown on some of the design decisions we made when designing JScript .NET.  That’s a huge topic, but fortunately I started writing a book on the subject a few years ago that never found a publisher. Tell you what — whenever I can’t think of something more interesting to post, I’ll put snippets of it up on my blog.

There were four main design goals for JScript .NET:

  1. JScript .NET should be an extension of JScript. This means that whenever possible a legal JScript program should be a legal JScript .NET program. JScript .NET must continue to be usable as a dynamic scripting language. Further, it should be easy to take an existing JScript Active Server Page and transform it into a JScript .NET page for ASP.NET.
  2. JScript .NET should afford the creation of highly performant programs. In particular, JScript .NET should work well with ASP.NET because performance is often critical in server-side applications. Also, JScript .NET should warn programmers when they use a language feature which could adversely impact performance.
  3.    JScript .NET should work well with the .NET Common Language Runtime (CLR). The CLR provides interoperability with other languages. It also ensures that JScript .NET works well with the .NET Frameworks, a huge library of useful packages.
  4. JScript .NET should make programming in the large easier. It should support programming styles that make it easy to reason about large programs. It should also provide programming constructs which make it easy to divide up large projects among team members.

Today I’ll talk a bit about the second point — how can we make JScript .NET faster than JScript? (I know, I said a while back that I’d give you my rant about performance and script languages.  Maybe if I have time tomorrow.)

JScript .NET takes a three-pronged approach towards improving performance. Individually each would help somewhat, but as is often the case, the whole is greater than the sum of its parts. These three prongs work together to afford the creation of more performant programs:

  1. JScript .NET has a type annotation system.
  2. JScript .NET discourages and/or restricts certain programming idioms which cause egregious performance problems.
  3. JScript .NET uses the Common Language Runtime.

The type annotation system can be complex when you look at the details — I’ll probably talk more about it in detail later.  But essentially the type annotation system is quite simple: the programmer attaches annotations to variable declarations, function argument lists and function return values. These annotations restrict the kinds of data which may be stored, passed or returned respectively.

Consider the simple example of adding two and two:

var Alpha = 2, Beta = 2;
// [ many lines of code omitted ]
var Gamma = Alpha + Beta;

Though Alpha and Beta start off their lives as numbers who knows what they are by the time Gamma is declared? Some code may have stored a string or a date or any other kind of data into Alpha or Beta.

Ironically, adding two and two thereby becomes a complicated mess. The JScript engine must determine at run time whether a string has been assigned to either Alpha or Beta (or both). If so then they must be both converted to strings and concatenated. If neither is a string then both must be converted to numbers and added.

This sounds simple enough but glosses over a number of details. For instance, the conversions are themselves possibly extremely complicated depending on what data have been stored in Alpha and Beta. Suffice to say that a seemingly simple operation like addition can take thousands of machine cycles, entire microseconds in some cases.

This is unfortunate. If there is one thing that computers are good at it is adding numbers blindingly fast. If the JScript .NET compiler could somehow know that only numbers could possibly be stored in Alpha and Beta then there would be no need to determine the types at run time. The compiler could simply emit instructions optimized for the case of adding two numbers. This operation could take nanoseconds instead of microseconds.

In JScript .NET you annotate the type of a variable by appending a colon and the type after the declaration.

var Alpha : Number = 2, Beta : Number = 2;
// [many lines of code omitted ]
var Gamma : Number = Alpha + Beta;

The code generated for this addition should now be much more efficient.

Type annotation also allows function calls to be much more efficient. Consider a simple function call on a string:

var Delta = 123;
// [many lines of code omitted ]
var Zeta = Delta.toString();

Again, though Delta starts off as a number it could be anything by the time toString is called. This is what we mean by a late bound scenario. The type of Delta is unknown, so at run time the object must be searched to see if it has a method named toString. This search is potentially expensive. It could be far more efficient if the compiler knew ahead of time that Delta should be converted to a string the same way that all numbers are converted to strings.

var Delta : Number = 123;
// [many lines of code omitted ]
var Zeta : String = Delta.toString();

Note that the JScript .NET compiler has an automatic type inference engineIf the compiler sees that a local variable in a function only ever has, say, strings assigned to it then the compiler will treat the variable as though it was annotated with the String type. However, it is a poor programming practice to rely upon the type inference engine. Certain programming practices (especially the use of the eval function) prevent the inferrer from making valid inferences. Also the inferrer can only make inferences on local variables; the types of unannotated global variables are not inferred.

There are also other good reasons to add annotations rather than relying on the inferencer to do it for you, but those will have to wait for another entry.


Commentary from 2019:

We’re coming up on a long stretch in this blog that is going to make me a little sad to revisit. We worked so hard on JScript .NET because we strongly believed in both the value proposition of the .NET framework, and that Microsoft needed to make a major investment in JavaScript. I feel like we were doing the right work at the wrong time, because the company was not amenable to making the sorts of investments that were needed.

Of course, Microsoft ended up developing the Chakra engine for the browser many years later at great expense. Eventually the value of a type-annotated JavaScript became apparent and a lot of effort was put into designing and implementing TypeScript. (Which is awesome; it is everything I dreamed of doing with JScript, just ten years later than I wanted to do it.)

It is still frustrating to me, over fifteen years later. Microsoft could have saved so much time and effort in the long run, and provided so much customer value, and really taken leadership in this tools space; it was obvious to me that it was going to be an important space and that we needed to be involved, but for reasons which were never made clear to me, all our efforts in this area were defunded. Microsoft was then left to play catch-up for the next decade, in a position of weakness of its own making. We had an optionally-typed, high-performance (for its time) JS engine that interoperated with C#; why would you defund that? In retrospect, that was obviously a strategic asset.

Anyways, expect more rants as I port over these posts.

Digging A Security Hole All The Way To China

I mentioned earlier that I wrote one of the last books published by the now-bankrupt Wrox Press. A sharp-eyed coworker who happened to be in China the other day sent me a copy of this:

CodeSecurity_2.jpg

Holy cow, I had no idea that Wrox translated my book into Chinese! No one ever told me that such a thing was even in the works. Just when I thought that the Wrox saga couldn’t get more bizarre, this turns up. Thanks Zhanbo! I’ll treasure it always.

Wrox is dead, long live Wrox

A number of people have expressed surprise to me that Wrox is out of business.  Here’s the scoop.

Wrox was owned by a company called Peer, which also owned Friends Of Ed, Glasshaus, and a number of other small businesses in the “developer education” field.

Peer made a big mistake:  they failed to time the end of the dot com bubble economy.  They ended up with massive overproduction of educational materials for developers that suddenly were not sellable. Returns from bookstores were large, and they ended up in big financial trouble.  They managed to keep the company afloat for a few more months, but were eventually unable to get new loans to service the interest on their old loans, which is by definition bankruptcy.

All of this happened very suddenly.  I was actually in the process of writing content for their web site and tech reviewing books up to the day before they announced their insolvency.  As it turned out, I was pretty lucky.  At least my book was published and I was actually paid my advance, though I never received any additional royalties.  All in all, Wrox only owed me a few thousand dollars when they went under.  There were plenty of people who had spent months writing books that were ready for the presses — those people never saw a dime for their work.

Wrox had two main creditors. They owed the bank of Scotland a lot of cash, and also had a large loan from AOL-Time-Warner-Netscape against all existing stocks of their books as collateral.   Under British law (similar to American law), all creditors get their collateral when the default happens, and everyone else (employees owed wages, authors owed royalties, and so on) gets in a line as the remaining assets are auctioned off.  Unsecured creditors like yours truly go to the end of the creditor line, and shareholders get in line after them.

Of course, since Peer had millions of pounds in debt and no assets save the existing stocks, all of which went to AOL-Time-Warner-Netscape, I didn’t even bother getting in on the action, as there was no action to get in on.

Since AOL-Time-Warner-Netscape owned the book stocks, one might wonder what happened to the royalties when they sold them.  Well, news flash: royalties are a share of profits, and no one made a profit on the sale of those books.  They were certainly all sold at a loss.

Wiley bought the rights to the dozen or so best sellers and the Wrox web site.  APress bought the rights to everything else, and may bring out APress editions of former Wrox books.

If you want to buy a copy of my book, you are probably out of luck.  I may ask Gary at APress if I can just reformat it into HTML and put it up on the web, if they don’t want to bring out an APress edition.  After all, I didn’t write the thing to make money.


Commentary from 2019:

I found a web site that had two dozen copies of my Wrox book and bought all of them for cheap. Amazon apparently also has some copies left. I gradually gave them away as joke gifts over the next few years and I think I only have one copy left. My mother probably also has a copy somewhere.

Wiley owns the Wrox trademark and continues to publish books under it.

Of course AOL-Time-Warner-Netscape is also long dead. Younger readers may have no clear memory of the bizarre time that was the pre-widespread-broadband internet economy. AOL, a dial-up internet company, managed to get big enough to acquire both Time-Warner and Netscape before flaming out spectacularly when broadband became a thing. AOL was spun back off into its own company in 2009, and was recently acquired by Verizon for reasons which surpass my understanding.

 

It Never Leaks But It Pours

One of the easiest bugs to write is the dreaded memory leak.  You allocate some chunk of memory and never release it.  Those of us who grew up writing application software might sometimes have a cavalier attitude towards memory leaks — after all, the memory is going to be reclaimed when the process heap is destroyed, right?  But those days are long gone.  Applications now often run for days, weeks or months on end.  Any memory that is leaked will slowly consume all the memory in the system until it dies horribly.  Web servers in particular are highly susceptible to leaks, as they run forever and move a lot of memory around with each page request.

The whole point of developing a garbage collected language is to decrease the burden on the developer.  Because the underlying infrastructure manages memory for you, you don’t have to worry about introducing leaks.  Of course, that puts the burden squarely upon the developer of the underlying infrastructure: me.  As you might imagine, I’ve been called in to debug a lot of memory leaks over the years.  The majority turned out to be in poorly written third-party components, but a few turned out to be in the script engines.

I mentioned a while back that ASP uses a technique called “thread pooling” to increase its performance.  The idea is that you maintain a pool of idle threads, and when you need work done, you grab one from the pool.  This saves on the expense of creating a new thread and destroying it when you’re done with it.  On a web server where there may be millions of page requests, the expense of creating a few million threads is non-trivial.  Also, this ensures that you can keep a lid on the number of requests handled by one server — if the server starts getting overloaded, just stop handing out threads to service requests.

I think I also mentioned a while back that JScript has a per-thread garbage collector.  That is, if you create two engines on the same thread, they actually share a garbage collector.  When one of those engines runs a GC, effectively they all get collected.

What do these things have to do with each other?

As it turns out, there is a memory leak that we have just discovered in the JScript engine.  A small data structure associated with the garbage collector is never freed when the thread goes away.  What incredible irony!  The very tool we designed to prevent your memory leaks is leaking memory.

Oh, but it gets worse.

As it turns out, this leak has been in the product for years. Why did we never notice it?  Because it is a per-thread leak, and ASP uses thread pooling! Sure, the memory leaks, but only once per thread, and ASP creates a small number of threads, so they never noticed the leak.

Why am I telling you this? Because for some reason, it never rains but it pours. We are suddenly getting a considerable number of people reporting this leak to us. Apparently, all of a sudden there are third parties developing massively multi-threaded applications that continually create and destroy threads with script engines. Bizarre, but true. They are all leaking a few dozen bytes per thread, and a few hundred thousand threads in, that adds up.

I have no idea what has led to this sudden uptick in multithreaded script hosts. But if you’re writing one, let me tell you two things:

  1. We’re aware of the problem.  Top minds on the Sustaining Engineering Team are looking at it, and I hope that we can have a patched version for a future service release.
  2. Use thread pooling! Not only will it effectively eliminate this leak, it will make your lives easier in the long run, believe me.

This whole thing reminds me that I want to spend some time discussing some of the pitfalls we’ve discovered in performance tuning multi-threaded applications.  But that will have to wait for another entry.


Commentary from 2019:

This post attracted a few “war stories” from readers:

Consider writing a “watch dog” process that watches your server; when it looks like it is leaking too much memory, just cleanly shut it down and restart it, problem solved.

Though indeed this is a common approach, and one taken by ASP, ideally we’d prefer to not leak resources in the first place. But as I noted, often the leaks are in third-party components that are hard to fix.

Leaks are exacerbated by the number of allocators that must be matched with their deallocators.

No kidding; it happens all the time. If you take ownership of a pointer, you have to know not just that it is valid memory, but how to deallocate it! That becomes part of the contract, but that contract is hard to describe or enforce.

Some language designer, I don’t remember who, once said that designers are torn between two competing goals: “X is important, so I’m going to make this so flexible that anyone can implement their own X that meets their needs”, and “X is important, so I’m going to implement one that does it right”. And that’s why in C we have no built-in string type, and no built-in heap memory allocator, because those things are so important that of course you want a dozen different crappy implementations of each that don’t interoperate but are confusingly similar. We had the second attitude in C#; make the runtime implement allocation and users can stop thinking about it.

In console games or other software for limited-memory embedded systems, you don’t worry about leaks because you don’t allocate memory dynamically in the first place.

I think I mentioned this in my series on writing a Z-Machine implementation. There’s no “heap allocator” per se; there’s a block of a few dozen KB of memory and you do with it as you see fit!

What Everyone Should Know About Character Encoding

Thank goodness Joel wrote this article — that means that I can cross it off of my list
of potential future blog entries. Thanks Joel!

Fortunately the script engines are entirely Unicode inside.  Making sure that the script source code passed in to the engine is valid UTF-16 is the responsibility of the host, and as Joel mentions, IE certainly jumps through some hoops to try and deduce the encoding.  WSH also has heuristics which try to determine whether the file is UTF-8 or UTF-16, but nothing nearly so complex as IE.

I should mention that in JScript you can use the \u0000 syntax to put unicode codepoints into literal strings.  In VBScript it is a little trickier — you need to use the CHRW method.


Commentary from 2019:

I did not mention in this article that the original implementations of VBScript and JScript were for IE running on 16 bit Windows, and 16 bit Windows did not support Unicode. We had non-Unicode versions of the scripting toolchain for quite some time. I wrote a lot of string library code for dealing with DBCS and other odd character encoding problems when I was first at Microsoft as a full-time employee.

There were a couple of good reader questions:

What do we do with code points above FFFF ?

VBScript and JScript use UTF-16 as their string encoding, so you can represent higher codepoints with surrogate pairs.

Is that also true of JScript.NET?

Yes, JS.NET, and all the .NET languages, use UTF-16 internally also.

How Do I Script A Non-Default Dispatch?

As I’ve discussed previously, the script engines always talk to objects on the late-bound IDispatch interface.  The point of this interface is to allow a script language to call a method or reference a property of an object by giving the name of the field and the arguments.  When the dispatch object is invoked, the object does the work of figuring out which method to call on which interface.

But what if an object supports two interfaces IFoo and IBar both of which you want to be able to call late-bound?  The most common way to solve this problem is to create two late-bound interfaces, IFooDisp and IBarDisp.  So when you ask the object for IFooDisp, you get an interface that can do late-bound invocation on IFoo, and similarly for IBarDisp.

What happens when you ask the object for an IDispatch interface?  It’s got to pick one of them!  The one it picks is called the “default dispatch”.

In JScript and VBScript, when you create an object (via new ActiveXObject in JScript or CreateObject​ in VBScript) the creation code always returns the default dispatch.  Furthermore, in JScript, when you fetch a property on an object and it returns a dispatch object, we ask the object to give us the default dispatch.  So in JScript, there is no way to script a non-default dispatch.

What about in VBScript?  There’s an irksome story here, again featuring a really bad mistake made by yours truly.  At least I meant well.

I didn’t write the variant import code, and I had always assumed that VBScript did the same thing as JScript — when an object enters the script engine from outside, we query it for its default dispatch.  As it turns out, that’s not true.  In VBScript, for whatever reason, we just pass imported dispatch objects right through and call them on whatever dispatch interface the callee gives us.

One day long ago we found a security hole in IE.  The details of the hole are not important — but what’s interesting about it was the number of things that had to go wrong to make the hole an actual vulnerability.  Basically the problem was that one of the built-in IE objects had two dispatch interfaces, one designed to be used from script and one for internal purposes.  The for-script dispatch interface was the default, and it was designed to participate in the IE security model.  The internal-only interface did not enforce the IE security model, and in fact, there was a way to use the object to extract the contents of a local disk file and send it out to the internet, which is obviously badness. Furthermore, there was a way to make another object return the non-default interface of the broken object.

JScript did not expose this vulnerability because it always uses the default dispatch even when given a non-default dispatch.  But the vulnerability was exposed by VBScript.  All these flaws had to work together to produce one vulnerability.  Now, when you find a security hole that consists of multiple small flaws, the way you fix it is not to just patch one thing and hope for the best.  You patch everything you possibly can.  Remember, secure software has defense in depth.  Make the attackers have to do twelve impossible things, not one impossible thing, because sometimes you’re wrong about what’s impossible.

Hence, when we fixed this hole, we fixed everything.  We made sure that the object’s persistence code could no longer read files off the local disk.  In case there was still a way to make it read the disk that the patch missed, we fixed the object model so that it never returned a non-default interface to a script.  In case there was still a way to do both those things that we missed, we also turned off VBScript’s ability to use non-default dispatches.

That last one turned out to be a huge mistake.  The fact that I had believed that VBScript always talked to the default dispatch does not logically imply that every VBScript user read my mind and knew that successfully using a non-default dispatch was some sort of fluke.  People naturally assume that if they write a program and it works, then it works because the language designers wanted them to be able to write that program!

As it turned out, there were plenty of programs out there in the wild that used VBScript to script non-default dispatch interfaces returned by method calls, and I broke all of them through trying to make IE safer.  We ended up shipping out a new build of VBScript with the default dispatch feature turned back on a couple of days later.  (Of course all the fixes to IE were sufficient to mitigate the vulnerability on their own, and those were not changed back.)

The morals of the story are:

  1. If you want to use a non-default dispatch, you have to use VBScript.
  2. There is no way to make VBScript give you a specific non-default dispatch.  In VBScript, once you have a dispatch, that’s the one you’re stuck with.
  3. Defense in depth is a good idea, but it’s not such a good idea to go so deep that backwards compatibility is broken if you can avoid it!

Commentary from 2019

Here we have yet another episode where I learned from a mistake. Of course the best mistakes to learn from are other people’s, but I’ll learn from my own if I have to.

This episode produced some good technical questions in the comments:

Can VBScript call .NET managed objects without using a COM-Callable Wrapper?

Nope. It’s called a COM-Callable Wrapper because it’s what you do to call the object from COM, and VBScript is COM.

How do tearoff dispatch interfaces fit into this?

First, let me describe the technique, since a lot of readers are not familiar with it.

Normally in COM you have an object implemented as a C++ object, and when you QueryInterface it for an interface, what happens internally is the object casts this to the desired interface and hands back a ref’d pointer; that is, the C++ object actually implements all of the interfaces that the object can be QI’d for. But that is not a requirement; it is perfectly legal for QI to return a different C++ object that implements the desired interface, so long as the rules of COM are followed. (Like, QI’ing for IUnknown on what is logically one object always produces the same pointer, and so on.)

This implementation strategy of “tearoff interfaces” has performance implications in certain scenarios, like you have an object that implements a lot of interfaces but you do not want to spend time initializing the vtables for all of them every time you create an instance.

Objects are free to implement interfaces using tearoffs if they like, but there is no support in VBScript for converting a given dispatch to another dispatch, regardless of whether it is implemented as a tearoff or not.

The usual trick in this case is to implement a property on the object that hands you the torn-off interface, and if you do that, this technique will work.

Why is there no #include?

A common and entirely sensible programming practice is to put commonly used utility functions in one file, and then somehow link that file in to many different programs. In traditional compiled languages you can compile a bunch of utilities into a statically linked .LIB file or a dynamically linked .DLL file.  But what do you do in script, where a program is just text?

People often ask me why there is no #include statement in VBScript or JScript.  Wouldn’t it be nice to be able to write up a page full of helpful utility functions and then in half a dozen different scripts say  #include myUtilityFunctions.vbs ?

If you think for a bit about how such a statement would be implemented, you see why we didn’t do it.  The reason is pretty simple: the script engine does not know where the current file came from, so it has no way of finding the referenced file.  Consider IE, for example:

<script language="vbscript">
#include myUtilityFunctions.vbs
' ...

 

How does the script engine know that this page was downloaded from any particular site? The script engine knows nothing about where the script came from — IE just hands the text to the engine.  Even assuming that it could figure it out, the script engine would then have to call IE’s code to download files off the internet. Worse, what if the path was fully qualified to a path in a different domain? Now the script engine needs to implement IE’s security rules.

Now consider: do any of those factors apply when running in ASP?  No!  ASP would have to have a completely different include mechanism where the relative path was based on facts about the current virtual root and the security semantics checked for cross-root violations.

What about WSH? There we’re worried about the current directory but have no security considerations.

What about third party hosts?  Every host could have a different rule for where the script comes from and what is a legal path.  We cannot possibly put all those semantics into the script engine.  To make an “include” feature work, we’d have to define a callback function into the host so that the host could fetch the included script for us.

It’s just too much work for too little gain, so we simply declared a rule: the host is responsible for implementing any mechanism whereby script files can be included into the script namespace.  This is why IE and WSH have the script src​ attribute.


Commentary from 2019:

Following on the heels of “Why can’t I create the WScript object?” we have another example of younger me focusing on the restrictions imposed by the implementation architecture choices rather than focusing on the user need that drives the question.

As I noted, we could have prioritized this scenario and implemented that callback, but we had many higher priorities, including improvements to performance and robustness that benefit everyone.

Commenters were quick to point out these germane critiques; my rather weak response was that the script engines tried to not be in the business of placing requirements upon the host.

That’s weak because the hosts had to solve the problem of how to find related scripts anyways, and they also had to come up with their own mechanisms for users to specify locations. Moreover, we could have made the “find a file to include” feature optional, thereby reducing the burden on all hosts.

The net result is that every script host has its own unique way of specifying “include this file”, and that created its own set of user confusions.

A stronger response is that the seemingly simple feature comes with a large set of implementation difficulties. For example, how do we detect and prevent hostile or accidental circular includes? How does the feature interact with the debugger? Are includes processed at parse time or run time? And many more; it’s not in any way a trivial feature.

From our perspective today, the whole question seems quaint. Web sites routinely download a whackload of JS modules, and there are whole framework subsystems devoted to figuring out which modules are needed, and so on.

 

WSC vs WSH

Following up on this morning’s entry, a reader asked me why Windows Script Components don’t have access to the WScript object.  “it is running in an instance of WSH isn’t it?”

No, it isn’t.  That’s a common misperception.  Let me clear it up.

The whole point of scripting is to make it easy to write programs.  Two very common kinds of programs are executables and components.  I’m sure that you understand the difference between the two, but just let me emphasize that executables are “standalone” — they have a well defined startup, they run for a while, and then they shut down.

Components are libraries of useful functionality packaged up as objects.  These objects cannot live on their own — they have to be created by an executable or another component, and they live as long as the owner wants them to.

We provide two ways to write executables in script — Windows Script Host files (with extensions .VBS, .JS, .WSF) and HTML Applications (with extension .HTA).  When you run one of these, the host application starts up, loads the script, runs it, and then shuts down.

We also provide a way to write components in script — the aptly named Windows Script Components.

When you create a WSC, its no different from creating any other in-process COM object.  The fact that the object is implemented in script is irrelevant.

People get WSCs and WSF’s confused because of their similar names and syntaxes.  But logically, WSCs and WSH are completely separate entities.

Note though that under the covers, quite a bit of the WSF processing code is actually implemented in ScrObj.DLL, the WSC engine, which itself is consumed as a component by the WSH executable.  We saw no reason to implement two identical XML parsers, two debugger interfaces, etc, when we had one sitting right there in a DLL already.

 

Why Can’t I Create The WScript Object?

Every now and then someone will ask me why the WSH shell object and the WSH network object are creatable from Visual Basic, but the actual root WScript object is not.

I am always completely mystified by why people ask this. Why would you want to
create the WScript object in a VB app?  What would it do for you?  It isn’t creatable as an in-process object because it represents the state of a WSH process, and a VB app is not a WSH process. The whole point of the WScript object is that it represents everything about the presently running Windows Script Host.

You can get from it all kinds of properties: the name and version of the script host, the path of the running script, the arguments which were passed in, the standard streams of the process, and whether the script was set to time out or not.  None of these properties make the slightest bit of sense to access outside of a running instance of WSH! What the heck could this possibly mean?

Set WScript = CreateObject("WSH.WScript")
Timeout = WScript.TimeOut

The timeout of what?  You’ve just got some object there, not a running instance of the script host.

What about the methods on the object?  Again, all of them are deeply tied into the underlying script hosting mechanisms.

WScript.CreateObject, GetObject, ConnectObject and DisconnectObject manipulate WSH-owned event sinks on the script dispatch.  I discussed yesterday how deeply tied WScript.Sleep is to the underlying hosting mechanism, and WScript.Echo is not particularly interesting — there are better ways to create a message box in VB already.

I just don’t get it.  Why do people keep asking me to make the WScript object creatable?  I can’t figure out what they think they’re going to do with it once they’ve created it and no one ever gives me a straight answer when I askIt’s like asking for the ability to create the Response object without starting ASP!  What could it possibly mean to have a Response object but no web server process?  It doesn’t make any sense, so we don’t let you do it.

We do provide an object model whereby you can spawn off WSH processes (both locally and remotely) and query their status as they run.  Perhaps that is what these people are looking for?  Remote WSH is a complicated topic in of itself, so perhaps I’ll talk about that in a later blog entry.


Commentary from 2019:

This article produced a lot of reader feedback, much of it very frustrated.

I learned a lot from these responses. Microsoft deservedly got a lot of flack from people who resented being characterized as “Morts”; that was a terrible choice of name for an important and valuable classification of developers. But it really is the case that a significant fraction of VB and VBScript users view error messages not as the compiler and runtime helpfully pointing out their problems, but as the thing standing between them and their success.

This article also showed how much I still lacked the “beginner’s mind” at this point in my career. I was too close to the implementation details of the system; I was quite genuine when I said that I didn’t understand why so many people wanted this thing, which was to my expert mind both impossible and useless. Why would anyone want something that was obviously both impossible and useless? But of course what they wanted was a smart and understandable solution to their immediate line-of-business problems.

Were I attacking this problem space today, I’d more likely choose a design that was better factored into orthogonal, re-usable subsystems. The alphabet soup of scripting technologies that we gave users was confusing and under-documented.

A point which I did not make in this article was that the scripting team was somewhat constrained in that we did not actually write Windows Script Host originally; another team at Microsoft created it for their own purposes, and made a number of design and implementation decisions that were not what the script team would have done. We inherited that pile of code and needed to hammer it into a shape that solved the general problem. In retrospect, it might have been a better idea to start the design process from scratch, rather than trying to mold an existing tool into the shape we thought users wanted.

Summarizing a few of the points:

I need an “eval” method that takes a bunch of VBScript or JScript, executes it, and gives me the results. How should I do that?

That’s what the Microsoft Script Control is for.

Do Windows Script Components run inside an instance of Windows Script Host?

No; WSCs are just COM objects that happen to run script; they are not WSH instances.

There are useful services that WSH provides that I’d like to see usable outside of WSH, like argument parsing.

I wrote the argument parser, so thanks for that; I tried hard to make it useful and I agree that it probably should have been made available as a stand-alone service.

I want to be able to run WSH scripts outside of WSH.

You can’t run web client scripts outside of the browser, or web server scripts outside of the server; similarly, you can’t run WSH scripts outside of WSH. WSH scripts depend on specific implementation details of WSH in order to execute correctly.

I get “permission denied” errors when trying to use the Remote WSH Controller to run WSH scripts from another machine.

Good! We worked hard to make the security model of Remote WSH very locked down. We did not want to be a new vector for attacks. To use Remote WSH, you must explicitly turn on DCOM, and then also explicitly turn on Remote WSH, and then to use it, you must be a member of the administrator group on the target machine.

We did not want to ever be in a situation where someone turned on DCOM for some unrelated reason and suddenly exposed an invisible, unknown attack vector that allowed an attacker to run an arbitrary script on a machine remotely!

WSH scripts all run in their own process; I’d like to be able to run scripts all in one process, and use WScript.Sleep

Again we come back to my point that I made in a previous episode: to write a correct Sleep you need to understand how all the other code in the same process will react to that sleeping job. If you want to write scripts that all run in the same process, use the script control; if you want some of those scripts to sleep then you will have to write an implementation of the sleep method that has the semantics you want.