One of the easiest bugs to write is the dreaded memory leak.  You allocate some chunk of memory and never release it.  Those of us who grew up writing application software might sometimes have a cavalier attitude towards memory leaks — after all, the memory is going to be reclaimed when the process heap is destroyed, right?  But those days are long gone.  Applications now often run for days, weeks or months on end.  Any memory that is leaked will slowly consume all the memory in the system until it dies horribly.  Web servers in particular are highly susceptible to leaks, as they run forever and move a lot of memory around with each page request.

The whole point of developing a garbage collected language is to decrease the burden on the developer.  Because the underlying infrastructure manages memory for you, you don’t have to worry about introducing leaks.  Of course, that puts the burden squarely upon the developer of the underlying infrastructure: me.  As you might imagine, I’ve been called in to debug a lot of memory leaks over the years.  The majority turned out to be in poorly written third-party components, but a few turned out to be in the script engines.

I mentioned a while back that ASP uses a technique called “thread pooling” to increase its performance.  The idea is that you maintain a pool of idle threads, and when you need work done, you grab one from the pool.  This saves on the expense of creating a new thread and destroying it when you’re done with it.  On a web server where there may be millions of page requests, the expense of creating a few million threads is non-trivial.  Also, this ensures that you can keep a lid on the number of requests handled by one server — if the server starts getting overloaded, just stop handing out threads to service requests.

I think I also mentioned a while back that JScript has a per-thread garbage collector.  That is, if you create two engines on the same thread, they actually share a garbage collector.  When one of those engines runs a GC, effectively they all get collected.

What do these things have to do with each other?

As it turns out, there is a memory leak that we have just discovered in the JScript engine.  A small data structure associated with the garbage collector is never freed when the thread goes away.  What incredible irony!  The very tool we designed to prevent your memory leaks is leaking memory.

Oh, but it gets worse.

As it turns out, this leak has been in the product for years. Why did we never notice it?  Because it is a per-thread leak, and ASP uses thread pooling! Sure, the memory leaks, but only once per thread, and ASP creates a small number of threads, so they never noticed the leak.

Why am I telling you this? Because for some reason, it never rains but it pours. We are suddenly getting a considerable number of people reporting this leak to us. Apparently, all of a sudden there are third parties developing massively multi-threaded applications that continually create and destroy threads with script engines. Bizarre, but true. They are all leaking a few dozen bytes per thread, and a few hundred thousand threads in, that adds up.

I have no idea what has led to this sudden uptick in multithreaded script hosts. But if you’re writing one, let me tell you two things:

1. We’re aware of the problem.  Top minds on the Sustaining Engineering Team are looking at it, and I hope that we can have a patched version for a future service release.
2. Use thread pooling! Not only will it effectively eliminate this leak, it will make your lives easier in the long run, believe me.

This whole thing reminds me that I want to spend some time discussing some of the pitfalls we’ve discovered in performance tuning multi-threaded applications.  But that will have to wait for another entry.

Commentary from 2019:

This post attracted a few “war stories” from readers:

Consider writing a “watch dog” process that watches your server; when it looks like it is leaking too much memory, just cleanly shut it down and restart it, problem solved.

Though indeed this is a common approach, and one taken by ASP, ideally we’d prefer to not leak resources in the first place. But as I noted, often the leaks are in third-party components that are hard to fix.

Leaks are exacerbated by the number of allocators that must be matched with their deallocators.

No kidding; it happens all the time. If you take ownership of a pointer, you have to know not just that it is valid memory, but how to deallocate it! That becomes part of the contract, but that contract is hard to describe or enforce.

Some language designer, I don’t remember who, once said that designers are torn between two competing goals: “X is important, so I’m going to make this so flexible that anyone can implement their own X that meets their needs”, and “X is important, so I’m going to implement one that does it right”. And that’s why in C we have no built-in string type, and no built-in heap memory allocator, because those things are so important that of course you want a dozen different crappy implementations of each that don’t interoperate but are confusingly similar. We had the second attitude in C#; make the runtime implement allocation and users can stop thinking about it.

In console games or other software for limited-memory embedded systems, you don’t worry about leaks because you don’t allocate memory dynamically in the first place.

I think I mentioned this in my series on writing a Z-Machine implementation. There’s no “heap allocator” per se; there’s a block of a few dozen KB of memory and you do with it as you see fit!