About ericlippert

http://ericlippert.com

Short questions

Finishing off my series of questions people asked during my recent webcast that I didn’t have time to answer, some short Q&A:

Does the analyzer require that we share our code assets with you?

No; the analyzer and defect database is local to your network. The considerable majority of customers have strict policies about who gets to see code assets. There are some scenarios in which you might want to ship your code to us for analysis: open source projects, for example, where there is no concern about privacy of the code. I’ll discuss these scenarios in more detail in future blog posts.

Can code analysis be used as a gate on checkins? That is, can we say that if analysis finds a new defect then the checkin is rejected?

Yes. The traditional model is to run the analyzer nightly and discover new defects checked in that day, but some customers want stricter gates on their checkins. We have seen that pretty much every such customer has their own custom-built system for gating checkins, so it is a bit tricky to do this “out of the box”. However we have a lot of experience helping customers integrate the analyzer into their systems.

Does the analyzer work well with “inversion of control”, “dependency injection”, “event driven programming” and similar patterns?

It works reasonably well. Many of these design patterns intend to facilitate decoupled systems, where two pieces of code communicate over an abstract but well-defined boundary that hides the implementation details. Living in a decoupled world means that you have some uncertainty about what code you’re actually calling; if you can’t figure out what code you’re calling then the static analyzer is going to have difficulty as well. That said, the analyzer has excellent heuristics for making guesses about what code is going to actually run when you invoke a method on an interface, virtual method or delegate.

Do statistical checkers consider statistics deduced from libraries?

I mentioned in my talk that some of the “checkers” — the algorithms which attempt to find defects in source code — use statistical rather than logical methods to deduce a potential defect. For example, if we see that 19 times out of 20, a method override void M() calls base.M(), then the 20th time is likely to be a mistake. If we see that 19 times out of 20, a field is accessed under a particular lock, then the 20th time is likely to be a mistake. And so on.

Yes, statistics are deduced from library code where you only have the assembly, not the source code. However, defects are not reported in library code, because how would you fix them? You don’t have the source code.

Is there more than just finding defects?

In my webcast I talked almost entirely about the analyzer, because that’s what I work on. Finding great defects is definately the heart of the product, but finding the defects is just the beginning. We’ve got sophisticated tools for managing new and legacy defects, deducing what code is not adequately tested, deducing what tests need to be run on a given change, reporting on progress against quality goals, detecting violations of quality metrics policies, and so on.

About these ads

Analyzing test code

Continuing with my series of answers to questions that were asked during my webcast last week…

Do the “checkers” (algorithms that find specific defect patterns) find defects in unit testing code?

If you want them to, yes.

First off, as I described in my talk, we deduce which code you want analyzed by monitoring the process that makes a clean build: by watching every invocation of the C# compiler during a clean build we know precisely which source code is being compiled, which assemblies are being referenced, and so on. If you don’t want your test code analyzed, don’t build it while the build capture system is running, and it won’t be analyzed. If you do want your test code analyzed, do build it.

Second, we have a feature called “Components” which allows you to separate defects sources into different logical groupings. This way in our defect presentation tool you can say “I only want to see defects found in the testing component”, and so on.

There are definitely some subtleties to consider when looking for defects in test code. For example, if you have a test which contains:

while(i++ < 10);
{ 
  Test.Assert(Bar(foo));
}

then that test has a genuine “stray semicolon” defect. (Of course the C# compiler will note this one as well, but that’s not germane to my point. The point is that this is clearly a case where the test is not behaving as desired and an analysis tool can determine that.)

By contrast, I’ve seen test code like this:

  Test.ExpectException<ArgumentNullException>(Bar(null));

The static analyzer will likely report that Bar is being called improperly, but that’s the whole point. It can be tricky to detect that this is an “intentional defect” and suppress it. Testing code tends to have a high concentration of intentional defects.

Copy-paste defects

Continuing with my series of answers to questions that were asked during my webcast on Tuesday:

The copy-paste checker example you showed was interesting. I’ve heard that NASA disallows copy-pasting in code because it is so error prone; is this true?

For readers who did not attend the talk: my favourite Coverity checker looks for code where you cut some code from one place, pasted it in another, and then made a series of almost but not quite consistent edits. An example taken from real world code is: Continue reading

Analysis vs code review

Thanks to everyone who came out to my “webinar” talk today; we had an excellent turnout. Apologies for the problems with the slides; there is some performance issue in the system where it works fine when it is not under load, but when there are lots of people using it, the slides do not advance as fast as they should. Hopefully the hosting service will get it sorted out.

As I mentioned last time, the recording will be edited and posted on the Coverity blog; I’ll post a link when I have one.

We got far, far more questions from users than we could possibly answer in the few minutes we had left at the end, and far too many to fit into one reasonably-sized blog post, so I’m going to split them up over the next few episodes. Today:

What percentage of defects does the Coverity analyzer find that should have been caught by code review? Continue reading

Avoiding C# defects talk Tuesday

Hello all, I have been crazy busy these last few weeks either traveling for work or actually programming with Roslyn — woo hoo! — and have not had time to blog. I’ve been meaning to do a short tour of the Roslyn codebase, now that it is open-sourced, but that will have to wait for later this summer.

Today I just want to mention that tomorrow, July 15th, at 8:30 AM Pacific Daylight Time, I’ll be doing a live talk broadcast on the internet where I’ll describe how the Coverity static analyzer works and what some of the most common defect patterns we find are. In particular I’m very excited by a new concurrency issue checker that looks for incorrect implementations of double-checked locking, and other “I avoided a lock when I should not have” defects. My colleague Kristen will also be talking about the new “desktop” mode of the analyzer.

If you’re interested, please register beforehand at this link. Thanks to Visual Studio Magazine for sponsoring this event.

If you missed it: the webcast will be recorded and the recording will be posted on the Coverity blog in a couple of days. The recording will also be posted on the Visual Studio Magazine site link above for 90 days.

Real world async/await defects

Today I’m once again asking for your help.

The headliner feature for C# 5 was of course the await operator for asynchronous methods. The intention of the feature was to make it easier to write asynchronous programs where the program text is not a mass of inside-out code that emphasizes the mechanisms of asynchrony, but rather straightforward, easy-to-read code; the compiler can deal with the mess of turning the code into a form of continuation passing style.

However, asynchrony is still hard to understand, and making methods that allow other code to run before their postconditions are met makes for all manner of interesting possible bugs. There are a lot of great articles out there giving good advice on how to avoid some of the pitfalls, such as:

These are all great advice, but it is not always clear which of these potential defects are merely theoretical, and which you have seen and fixed in actual production code. That’s what I am very interested to learn from you all: what mistakes were made by real people, how were they discovered, and what was the fix?

Here’s an example of what I mean. This defect was found in real-world code; obviously the extraneous details have been removed:

Frob GetFrob()
{
  Frob result = null;
  var networkDevice = new NetworkDevice();
  networkDevice.OnDownload += 
    async (state) => { result = await Frob.DeserializeStateAsync(state); };
  networkDevice.GetSerializedState(); // Synchronous
  return result; 
}

The network device synchronously downloads the serialized state of a Frob. When it is done, the delegate stored in OnDownload runs synchronously, and is passed the state that was just downloaded. But since it is itself asynchronous, the event handler starts deserializing the state asynchronously, and returns immediately. We now have a race between GetFrob returning null, and the mutation of closed-over local result, a race almost certainly won by returning null.

If you’d rather not leave comments here — and frankly, the comment system isn’t much good for code snippets anyways — feel free to email me at eric@lippert.com. If I get some good examples I’ll follow up with a post describing the defects.

London and Budapest

I recently spent a week visiting customers and giving talks in Europe and holy goodness, never have I been in so many countries in so little time. I flew from Seattle to London, changed planes, flew to Amsterdam, visited customers in the Netherlands and Belgium, then took the Eurostar from Lille back to London, and that was just the first two days! Continue reading

ATBG: Why do enumerators avoid a bounds check?

I am back from a week in which I visited England, Holland, Belgium, France and Hungary; this is by far the most countries I’ve been to in so little time. It was great to meet so many customers; more on bicycling around Budapest in a future episode. For today though, I’ve posted on the Coverity Development Testing Blog’s continuing series Ask The Bug Guys a follow up to the previous episode. I’ll discuss why the struct-based list enumerator does not do bounds checking in its implementation of the Current property. Check it out! Thanks to reader Xi Chen for the question. Continue reading

Funniest Hungarian joke ever

No computer stuff today; since I am actually in Hungary today on business and it is the the anniversary of the happy event mentioned below, today’s FAIC is a rerun. Enjoy!


I’m back from my fabulous adventures in Austria, Romania and Canada and I had a fabulous time, as you might imagine. We were in Romania for a wedding of some close personal friends who live here in Seattle; much of the groom’s family escaped from Romania during the Communist period and settled in Austria, so we spent some time in Vienna and then headed to Bucharest, and then crossed the Carpathian mountains by bus into Transylvania for the wedding. Some of the highlights included:

Continue reading