Unknown's avatar

About ericlippert

http://ericlippert.com

Simple names are not so simple, part one

C# has many rules that are designed to prevent some common sources of bugs and encourage good programming practices. So many, in fact, that it is often quite confusing to sort out exactly which rule has been violated. I thought I might spend some time talking about what the different rules are. We’ll finish up with a puzzle.

To begin with, it will be vital to understand the difference between scope and declaration space. To refresh your memory of my earlier article: the scope of an entity is the region of text in which that entity may be referred to by its unqualified name. A declaration space is a region of text in which no two things may have the same name (with an exception for methods which differ by signature.) A “local variable declaration space” is a particular kind of declaration space used for declaring local variables; local variable declaration spaces have special rules for determining when they overlap.

The next thing that you have to understand to make any sense o this is what a “simple name” is. A simple name is always either just a plain identifier, like x, or, in some cases, a plain identifier followed by a type argument list, like Frob<int, string>.

Lots of things are treated as “simple names” by the compiler: local variable declarations, lambda parameters, and so on, always have the first form of simple name in their declarations. When you say Console.WriteLine(x); the Console and x are simple names but the WriteLine is not. Confusingly, there are some textual entities which have the form of simple names, but are not treated as simple names by the compiler. We might talk about some of those situations in later fabulous adventures.

So, without further ado, here are some relevant rules which are frequently confused. It’s rules 3 and 4 that people find particularly confusing.

  1. It is illegal to refer to a local variable before its declaration. (This seems reasonable I hope.)
  2. It is illegal to have two local variables of the same name in the same local variable declaration space or nested local variable declaration spaces.
  3. Local variables are in scope throughout the entire block in which the declaration occurs. This is in contrast with C++, in which local variables are in scope in their block only at points after the declaration.
  4. For every occurrence of a simple name, whether in a declaration or as part of an expression, all uses of that simple name within the immediately enclosing local variable declaration space must refer to the same entity.

The purpose of all of these rules is to prevent the class of bugs in which the reader/maintainer of the code is tricked into believing they are referring to one entity with a simple name, but are in fact accidentally referring to another entity entirely. These rules are in particular designed to prevent nasty surprises when performing what ought to be safe refactorings.

Consider a world in which we did not have rules 3 and 4. In that world, this code would be legal:

class C
{
  int x;
  void M()
  {
    // 100 lines of code
    x = 20; // means "this.x";
    Console.WriteLine(x); // means "this.x"
    // 100 lines of code
    int x = 10;
    Console.WriteLine(x); // means "local x"
  }
}

This is hard on the person reading the code, who has a reasonable expectation that the two Console.WriteLine(x) lines do in fact both print out the contents of the same variable. But it is particularly nasty for the maintenance programmer who wishes to impose a reasonable coding standard upon this body of code. “Local variables are declared at the top of the block where they’re used” is a reasonable coding standard in a lot of shops. But changing the code to:

class C
{
  int x;
  void M()
  {
    int x;
    // 100 lines of code
    x = 20; // no longer means "this.x";
    Console.WriteLine(x); // no longer means "this.x"
    // 100 lines of code
    x = 10;
    Console.WriteLine(x); // means "local x"
  }
}

changes the meaning of the code! We wish to discourage authoring of multi-hundred-line methods, but making it harder and more error-prone to refactor them into something cleaner is not a good way to achieve that goal.

Notice that the original version of this program, rule 3 means that the program violates rule 1 — the first usage of x is treated as a reference to the local before it is declared. The fact that it violates rule 1 because of rule 3 is precisely what prevents it from being a violation of rule 4! The meaning of x is consistent throughout the block; it always means the local, and therefore is sometimes used before it is declared. If we scrapped rule 3 then this would be a violation of rule 4, because we would then have two inconsistent meanings for the simple name x within one block.

Now, these rules do not mean that you can refactor willy-nilly. We can still construct situations in which similar refactorings fail. For example:

class C
{
  int x;
  void M()
  {
    {
      // 100 lines of code
      x = 20; // means "this.x";
      Console.WriteLine(x); // means "this.x"
    }
    {
      // 100 lines of code
      int x = 10;
      Console.WriteLine(x); // means "local x"
    }
  }
}

This is perfectly legal. We have the same simple name being used two different ways in two different blocks, but the immediately enclosing block of each usage does not overlap that of any other usage. The local variable is in scope throughout its immediately enclosing block, but that block does not overlap the block above. In this case, it is safe to refactor the declaration of the local to the top of its block, but not safe to refactor the declaration to the top of the outermost block; that would change the meaning of x in the first block. Moving a declaration up is almost always a safe thing to do; moving it out is not necessarily safe.

Now that you know all that, here’s a puzzle for you, a puzzle that I got completely wrong the first time I saw it:

using System.Linq;
class Program
{
  static void Main()
  {
    int[] data = { 1, 2, 3, 1, 2, 1 };
    foreach (var m in from m in data orderby m select m)
      System.Console.Write(m);
  }
}

It certainly looks like simple name m is being used multiple times to mean different things. Is this program legal? If yes, why do the rules for not re-using simple names not apply? If no, precisely what rule has been violated?

Next time on FAIC: The solution to this puzzle.

Begging the question

Michelle Pfeiffer 2007 In my last post I described the syllogism “Photogenic people look good in photographs; Michelle Pfeiffer is photogenic; therefore, Michelle Pfeiffer looks good in photographs” as “begging the question”. A few people commented on that, so I thought I’d address this point of English usage.

In modern usage, “begging the question” has come to mean nothing more than “the situation suggests that an obvious question to raise at this time is blah blah blah.” For example, “The global financial meltdown begs the question: was there insufficient federal oversight of the American mortgage industry?” Though this usage is certainly common in civic discourse and the media, it is entirely a modern departure from the historic usage of the phrase. I try to eschew this modern usage when I say “begs the question”.

Continue reading

What’s the difference between “as” and cast operators?

Most people will tell you that the difference between (Alpha) bravo and bravo as Alpha is that the former throws an exception if the conversion fails, whereas the latter produces null. Though this is correct, and this is the most obvious difference, it’s not the only difference. There are pitfalls to watch out for here.

First off, since the result of the as operator can be null, the resulting type has to be one that includes a null value: either a reference type or a nullable value type. You cannot do x as int, that doesn’t make any sense. If the operand isn’t an int, then what should the expression’s value be? The type of the as expression is always the type named in the expression, so it needs to be a type that can take a null.

Second, the cast operator, as I’ve discussed before, is a strange beast. It means two contradictory things: “check to see if this object really is of this type, throw if it is not” and “this object is not of the given type; find me an equivalent value that belongs to the given type”. The latter meaning of the cast operator is not shared by the as operator. If you say

short s = (short)123;
int? i = s as int?;

then you’re out of luck. The as operator will not make the representation-changing conversions from short to nullable int like the cast operator would. Similarly, if you have class Alpha and unrelated class Bravo, with a user-defined conversion from Bravo to Alpha, then (Alpha) bravo will run the user-defined conversion, but bravo as Alpha will not. The as operator only considers reference, boxing and unboxing conversions.

And finally, of course the use cases of the two operators are superficially similar, but semantically quite different. A cast communicates to the reader “I am certain that this conversion is legal and I am willing to take a runtime exception if I’m wrong”. The as operator communicates “I don’t know if this conversion is legal or not; we’re going to give it a try and see how it goes”.


Notes from 2020

There were a number of pragmatic comments to the original post:

  • There is pressure to use as over casting for legibility reasons; (x as C).M() is perceived as easier to read than ((C)x).M(). Indeed, the cast operator has other syntactic problems than just being ugly; is (C)-1 casting -1 to C, or subtracting 1 from (C)?
  • It seems like there is a missed opportunity for a syntactic sugar that both tests the type and gives you a value of that type in one step; those comments and similar comments from many other users eventually led to if (x as C c) being added to the language.
  • One commenter pointed out that use of type checking operators in generic code is a bad smell; I thoroughly agree.
  • Another pointed out that any sort of type checking at runtime is an indication that the programmer was unable to produce a proof of type safety that could be checked by the compiler, and is therefore suspect. It would be nice if those points were easier to find quickly in a program because they need deeper code review. This is an often neglected aspect of language design; I should write more about that!
  • There was also a long conversation about the use of various operators for different kinds of dynamic dispatch. I discussed these in my Wizards and Warriors series in more detail.

String interning and String.Empty

Here’s a curious program fragment:

object obj = "Int32";
string str1 = "Int32";
string str2 = typeof(int).Name;
Console.WriteLine(obj == str1); // true
Console.WriteLine(str1 == str2); // true
Console.WriteLine(obj == str2); // false !?

Surely if A equals B, and B equals C, then A equals C; that’s the transitive property of equality. It appears to have been thoroughly violated here.
Continue reading

What’s the difference between conditional compilation and the Conditional attribute?

User: Why does this program not compile correctly in the release build?

class Program 
{ 
#if DEBUG 
    static int testCounter = 0; 
#endif 
    static void Main(string[] args) 
    { 
        SomeTestMethod(testCounter++); 
    } 
    [Conditional("DEBUG")] 
    static void SomeTestMethod(int t) { } 
}

Eric: This fails to compile in the release build because testCounter cannot be found in the call to SomeTestMethod.

User: But that call site is going to be omitted anyway, so why does it matter? Clearly there’s some difference here between removing code with the conditional compilation directive versus using the conditional attribute, but what’s the difference?

Eric: You already know the answer to your question, you just don’t know it yet. Let’s get Socratic; let me turn this around and ask you how this works. How does the compiler know to remove the method call site?

User: Because the method called has the Conditional attribute on it.

Eric: You know that. But how does the compiler know that the method called has the Conditional attribute on it?

User: Because overload resolution chose that method. If this were a method from an assembly, the metadata associated with that method has the attribute. If it is a method in source code, the compiler knows that the attribute is there because the compiler can analyze the source code and figure out the meaning of the attribute.

Eric: I see. So fundamentally, overload resolution does the heavy lifting. How does overload resolution know to choose that method? Suppose hypothetically there were another method of the same name with different parameters.

User: Overload resolution works by examining the arguments to the call and comparing them to the parameter types of each candidate method and then choosing the unique best match of all the candidates.

Eric: And there you go. Therefore the arguments must be well-defined at the point of the call, even if the call is going to be removed. In fact, the call cannot be removed unless the arguments are extant! But in the release build, the type of the argument cannot be determined because its declaration has been removed.

So now you see that the real difference between these two techniques for removing unwanted code is what the compiler is doing when the removal happens. At a high level, the compiler processes a text file like this. First it “lexes” the file. That is, it breaks the string down into “tokens” — sequences of letters, numbers and symbols that are meaningful to the compiler. Then those tokens are “parsed” to make sure that the program conforms to the grammar of C#. Then the parsed state is analyzed to determine semantic information about it; what all the types are of all the expressions and so on. And finally, the compiler spits out code that implements those semantics.

The effect of a conditional compilation directive happens at lex time; anything that is inside a removed #if block is treated by the lexer as a comment. It’s like you simply deleted the whole contents of the block and replaced it with whitespace. But removal of call sites depending on conditional attributes happens at semantic analysis time; everything necessary to perform that semantic analysis must be present. 

User: Fascinating. Which parts of the C# specification define this behavior?

Eric: The specification begins with a handy “table of contents”, which is very useful for answering such questions. The table of contents states that section 2.5.1 describes “Conditional compilation symbols” and section 17.4.2 describes “The Conditional attribute”.

User: Awesome!

What’s the difference? fixed versus fixed

I got an email the other day that began:

I have a question about fixed sized buffers in C#:

unsafe struct FixedBuffer 
{ 
  public fixed int buffer[100];

Now by declaring buffer as fixed it is not movable…

And my heart sank. This is one of those deeply unfortunate times when subtle choices made in the details of language design encourage misunderstandings.

When doing pointer arithmetic in unsafe code on a managed object, you need to make sure that the garbage collector does not move the memory you’re looking at. If a collection on another thread happens while you’re doing pointer arithmetic on an object, the pointer can get all messed up. Therefore, C# classifies all variables as “fixed” or “movable”. If you want to do pointer arithmetic on a movable object, you can use the fixed keyword to say “this local variable contains data which the garbage collector should not move.” When a collection happens, the garbage collector needs to look at all the local variables for in-flight calls (because of course, stuff that is in local variables needs to stay alive); if it sees a “fixed” local variable then it makes a note to itself to not move that memory, even if that fragments the managed heap. (This is why it is important to keep stuff fixed for as little time as possible.) So typically, we use “fixed” to mean “fixed in place”.

But that’s not what “fixed” means in this context; this means “the buffer in question is fixed in size to be one hundred ints” — basically, it’s the same as generating one hundred int fields in this structure.

Obviously we often use the same keyword to mean conceptually the same thing. For example, we use the keyword internal in many ways in C#, but all of them are conceptually the same. It is only ever used to mean “accessibility to some entity is restricted to only code in this assembly”.

Sometimes we use the same keyword to mean two completely different things, and rely upon context for the user to figure out which meaning is intended. For example:

var results = from c in customers where c.City == "London" select c;

versus

class C<T> where T : IComparable<T>

It should be clear that where is being used in two completely different ways: to build a filter clause in a query, and to declare a type constraint on a generic type parameter.

We cause people to run into trouble when one keyword is used in two different ways but the difference is subtle, like our example above. The user’s email went on to ask a whole bunch of questions which were predicated on the incorrect assumption that a fixed-in-size buffer is automatically fixed in place in memory.

Now, one could say that this is just an unfortunate confluence of terms; that “fixed in size” and “fixed in place” just happen to both use the word “fixed” in two different ways, how vexing. But the connection is deeper than that: you cannot safely access the data stored in a fixed-in-size buffer unless the container of the buffer has been fixed in place. The two concepts are actually quite strongly related in this case, but not at allthe same.

On the one hand it might have been less confusing to use two keywords, say pinned and fixed. But on the other hand, both usages of fixed are only valid in unsafe code. A key assumption of all unsafe code features is that if you are willing to use unsafe code in C#, then you are already an expert programmer who fully understands memory management in the CLR. That’s why we make you write unsafe on the code; it indicates that you’re turning off the safety system and you know what you’re doing.

A considerable fraction of the keywords of C# are used in two or more ways: fixed, into, partial, out, in, new, delegate, where, using, class, struct, true, false, base, this, event, return and void all have at least two different meanings. Most of those are clear from the context, but at least the first three — fixed, into and partial — have caused enough confusion that I’ve gotten questions about the differences from perplexed users. I’ll take a look at into and partial next.

Arrays of arrays

Most C# developers understand that there’s a difference between a “rectangular” and a “ragged” two-dimensional array.

int[,] rectangle = { 
  {10, 20}, 
  {30, 40}, 
  {50, 60} }; 
int[][] ragged = { 
  new[] {10},
  new[] {20, 30},
  new[] {40, 50, 60} };

Here we have a two-dimensional array with six elements, arranged in three rows of two elements each. And we have a one-dimensional array with three elements, where each element is itself a one-dimensional array with one, two or three elements.

That’s all very straightforward. Where things get brain-destroying is when you try to make a ragged array of two-dimensional arrays.  Continue reading

Color Color

The original post of this article is archived here.

IMPORTANT NOTE FROM 2023: the C# rules regarding overload resolution of static members have changed since this article was written in 2009; it is accurate for C# 1 through 5, but this rule was changed in a more recent version; I do not offhand recall which. This article should be considered to be of historical interest; see more recent documentation for the current behaviour.


Pop quiz: What does the following code do when compiled and run?

class C
{
  public static void M(string x)
  {
    System.Console.WriteLine("static M(string)");
  }
  public void M(object s)
  {
    System.Console.WriteLine("M(object)");
  }
}
class Program
{
  static void Main()
  {
    C c = new C();
    c.M("hello");
  }
}

(1) writes static M(string)
(2) writes M(object)
(3) uh, dude, this code doesn’t even compile much less run
(4) something else

Think about that for a bit and then try it and see if you were right. Scroll down when you’re ready.

.

.

.

.

.

.

.

In option (1), the compiler could decide that the best match is the static one and let you call it through the instance, ignoring the instance value. In option (2), the compiler could decide that the object version is the best match and ignore the static one, even though the argument match is better. But neither of those actually happens; the rules of C# say that the compiler should pick the best match based on the arguments, and then disallow static calls that are through an instance! The actual result here is therefore (3):

error CS0176: Member 'C.M(string)' cannot be accessed with an instance reference; qualify it with a type name instead

What is up with this craziness? If you believe that the rule “static methods should never be accessed through instances” is a good rule – and it seems reasonable – then why doesn’t overload resolution remove the static methods from the candidate set when called through an instance? Why does it even allow the “string” version to be an applicable candidate in the first place, if the compiler knows that it can never possibly work?

I agree that this seems really goofy at first.

To explain why this is not quite as dumb as it seems, consider the following variation on the problem. Class C stays the same.

class B
{
  public C C = new C();
  public void N()
  {
    C.M("hello");
  }
}

What does a call to N on an instance of B do?

(1) writes static M(string)
(2) writes M(object)
(3) compilation error
(4) something else

Again, scroll down when you’ve decided.

.

.

.

.

.

.

A bit trickier now, isn’t it? Does C.M mean “call instance method M on the instance stored in this.C?” or does it mean “call static method M on type C”? Both are applicable!

Because of our goofy rule we do the right thing in this case. We first resolve the call based solely on the arguments and determine that the static method is the best match. Then we check to see if the “receiver” at the call site can be interpreted as being through the type. It can. So we make the static call and write “static M(string)”. If the instance version had been the best match then we would successfully call the instance method through the property.

The reason that the compiler does not remove static methods when calling through an instance is because the compiler does not necessarily know that you are calling through an instance. Because there are situations where it is ambiguous whether you’re calling through an instance or a type, we defer deciding which you meant until the best method has been selected.

Now, one could make the argument that in our first case, the receiver cannot possibly be a type. We could have further complicated the language semantics by having three overload resolution rules, one for when the receiver is known to be a type, one for when the receiver is known to not be a type, and one for when the receiver might or might not be a type. But rather than complicate the language further, we made the pragmatic choice of simply deferring the staticness check until after the bestness check. That’s not perfect but it is good enough for most cases and it solves the common problem.

The common problem is the situation that arises when you have a property of a type with the same name, and then try to call either a static method on the type, or an instance method on the property. (This can also happen with locals and fields, but local and field names typically differ in case from the names of their types.) The canonical motivating example is a public property named Color of type Color, so we call this “the Color Color problem”.

In real situations the instance and static methods typically have different names, so in the typical scenario you never end up calling an instance method when you expect to call a static method, or vice versa. The case I presented today with a name collision between a static and an instance method is relatively rare.

If you’re interested in reading the exact rules for how the compiler deals with the Color Color problem, see section 7.5.4.1 of the C# 3.0 specification.