I'll remind you all again that in C# the term "scope" has a very carefully defined meaning: the scope of a named entity is the region of program text in which the unqualified name can be used to refer to the entity.¹

For example, the scope of a local variable is the text of the block which declares it. The scope of a private method is the text of the body of the class or struct which declares it. And so on; the C# specification has careful definitions which define the scope of everything that has a name.

The word "lexical" means, in a broad sense "relating to text", and clearly we have defined "scope" as being a relationship involving text, so is this kind of scoping also called "lexical scoping"?

Sort of, but not exactly. Let me explain.

Programming languages can be broadly divided into two categories: the lexically scoped languages and the dynamically scoped languages. The difference between the two is: in a lexically scoped language, the meaning of an unqualified name can be completely determined by looking at the program text; the analysis can be done "statically". In a dynamically scoped language the meaning of an unqualified name can change at runtime; the name analysis can only be done "dynamically".

Let me give you an example; it is easiest to show this with lambdas.

class C
{
  public static Func<int> M()
  {
    int x = 123;
    return () => x;
  }
}
class P
{
  static void Main()
  {
    int x = 456;
    Func<int> f = C.M();
    System.Console.WriteLine(f());
  }
}

The question is: what gets printed out? C# is a lexically scoped language, so the meaning of x in the lambda is determined at compile time, by analyzing the text where the lambda was written. C# prints out 123. If C# were a dynamically scoped language then the meaning of x would be determined by analyzing the location where the delegate was executed at runtime, so it would print out 456.

Dynamically scoped languages essentially make the C# definition of scope useless; any method that executes a lambda in a dynamically scoped language makes the region of program text in which its locals can be referred to by their names arbitrarily large.

JavaScript, though a very dynamic language, is actually for the most part lexically scoped:

function M()
{
  var x = 123;
  return function () { return x; };
}
function N()
{
  var x = 456;
  var f = M();
  print(f()); // 123
}

However, JavaScript does have one feature which makes it dynamically scoped:

function Q(y)
{
  var x = 123;
  with(y)
    return x;
}
print(Q({ x : 456 })); // 456
print(Q(789));         // 123

Here the meaning of unqualified name x changes at runtime depending on whether y has a member x or not. For this reason I recommend avoiding the with block in JavaScript; it makes it hard for the reader to understand the meaning of the program.

Most modern languages are lexically scoped; experience has shown that lexical scoping is easier on all concerned; developers and maintenance programmers have an easier time understanding lexically scoped languages, and compiler developers have an easier time writing efficient compilers. Some variants of Lisp use dynamic scoping, though Scheme requires lexical scoping. There are still a few dynamically scoped languages in common usage though; PostScript, the programming language which runs on printers, is perhaps the most commonly used of them.

How bad does your quality assurance have to be to ship to customers a power supply that cannot possibly fit into a power socket?

Math.Floor(value + 0.5)

The value which does not round correctly is the double 0.49999999999999994, which is the largest double that is smaller than 0.5. With the given algorithm this rounds up to 1.0, even though clearly 0.49999999999999994 is less than one half, and therefore should round down.

What the heck is going on here?

As you probably know, doubles are represented internally as binary fractions. The original double in binary¹ is

1.1111111111111111111111111111111111111111111111111111 x 2-2

Or

0.011111111111111111111111111111111111111111111111111111

which in decimal is

0.499999999999999944488848768742172978818416595458984375

As you can see it is extremely close to the stated value. Again, just so we are clear: when in source code you say 0.49999999999999994, that decimal fraction is rounded to the nearest binary fraction that has no more than 52 bits after the decimal place.

Already we have one rounding, and as you can see, we rounded up -- but we're still less than half.

Now suppose we took this value and added exactly 0.5 to it. We'd get

0.999999999999999944488848768742172978818416595458984375

which is still smaller than one. Therefore we'd expect this to go to zero when passed to Math.Floor, right?

Wrong; that fraction isn't a legal double because it would require 53 bits of precision to represent exactly. A double only has 52 bits of precision, and therefore this must be rounded off when it becomes a double.

But rounded to what? Clearly it is extremely close to 1.0. What is the largest double value that is smaller than 1.0? In binary that would be

1.1111111111111111111111111111111111111111111111111111 x 2-1

Or

0.11111111111111111111111111111111111111111111111111111

which is

0.99999999999999988897769753748434595763683319091796875

in decimal.

So we now have three relevant values:

0.999999999999999888977697537484345957636833190917968750 (largest double smaller than 1.0)
0.999999999999999944488848768742172978818416595458984375 (exact)
1.000000000000000000000000000000000000000000000000000000 (1.0)

Which one should we round the middle one to? Well, which is closer?

It's OK, I'll wait while you do the math by hand.

.

.

.

Neither is closer. The value is exactly in the middle of the two possibilities. The difference is

0.000000000000000055511151231257827021181583404541015625

in both directions.

Of course you could have worked that out a lot more easily in binary than in decimal! The three numbers are in binary:

0.111111111111111111111111111111111111111111111111111110
0.111111111111111111111111111111111111111111111111111111
1.000000000000000000000000000000000000000000000000000000

From which you can read off that difference between both pairs is 2^-54.

When faced with this situation the floating point chip has got to make a decision and it decides to once again round up; it has no reason to believe that you really want it to round down. So the sum rounds to the double 1.0, which is then "floored" to 1.0.

Once again we see that floating point math does not behave at all like pen-and-paper math. Just because an algorithm would always work in exact arithmetic does not mean that it always works in inexact floating point. Multiple roundings of extremely small values can add up to a large difference.

I've posted the code I used to make all these calculations below; long-time readers will recognize it from my previous article on how to decompose a double into its constituent parts.

using System;
using System.Collections.Generic;
using System.Text;
using Microsoft.SolverFoundation.Common;

class Program
{
    static void Main()
    {
        MyDouble closeToHalf_Double = 0.49999999999999994;
        Rational closeToHalf_Rational = DoubleToRational(closeToHalf_Double);

        Console.WriteLine("0.49999999999999994");
        Console.WriteLine("Actual value of double in binary:");
        Console.WriteLine("1.{0} x 2 to the {1}", 
            closeToHalf_Double.MantissaBits.Join(), 
            closeToHalf_Double.Exponent - 1023);
        Console.WriteLine("Actual value of double as fraction:");
        Console.WriteLine(closeToHalf_Rational.ToString());
        Console.WriteLine("Actual value of double in decimal:");
        Console.WriteLine(closeToHalf_Rational.ToDecimalString());
        Console.WriteLine();

        MyDouble closeToOne_Double = 0.99999999999999994;
        Rational closeToOne_Rational = DoubleToRational(closeToOne_Double);

        Console.WriteLine("0.99999999999999994");
        Console.WriteLine("Actual value of double in binary:");
        Console.WriteLine("1.{0} x 2 to the {1}",
            closeToOne_Double.MantissaBits.Join(),
            closeToOne_Double.Exponent - 1023);
        Console.WriteLine("Actual value of double as fraction:");
        Console.WriteLine(closeToOne_Rational.ToString());
        Console.WriteLine("Actual value of double in decimal:");
        Console.WriteLine(closeToOne_Rational.ToDecimalString());
        Console.WriteLine();

        // Now let's do the arithmetic in "infinite precision":

        Rational sum = closeToHalf_Rational + (Rational)0.5;

        Console.WriteLine("Sum in exact arithmetic as fraction:");
        Console.WriteLine(closeToOne_Rational.ToString());
        Console.WriteLine("Sum in exact arithmetic in decimal:");
        Console.WriteLine(closeToOne_Rational.ToDecimalString());

        // But that has more precision than will fit into a double, 
        // so we have to round off. We have two possible values:

        Rational d1 = Rational.One - sum;
        Rational d2 = sum - closeToOne_Rational;

        Console.WriteLine("Difference from one:");
        Console.WriteLine(d1.ToDecimalString());
        Console.WriteLine("Difference from closeToOne:");
        Console.WriteLine(d2.ToDecimalString());
    }

    static Rational DoubleToRational(MyDouble d)
    {
        bool subnormal = d.Exponent == 0;
        var two = (Rational)2;
        var fraction = subnormal ? Rational.Zero : Rational.One;
        var adjust = subnormal ? 1 : 0;
        for (int bit = 51; bit >= 0; --bit)
            fraction += d.Mantissa.Bit(bit) * two.Exp(bit - 52 + adjust);
        fraction = fraction * two.Exp(d.Exponent - 1023);
        if (d.Sign == 1)
            fraction = -fraction;
        return fraction;
    }
}

struct MyDouble
{
    private ulong bits;
    public MyDouble(double d)
    {
        this.bits = (ulong)BitConverter.DoubleToInt64Bits(d);
    }

    public int Sign
    {
        get
        {
            return this.bits.Bit(63);
        }
    }

    public int Exponent
    {
        get
        {
            return (int)this.bits.Bits(62, 52);
        }
    }

    public IEnumerable<int> ExponentBits
    {
        get
        {
            return this.bits.BitSeq(62, 52);
        }
    }

    public ulong Mantissa
    {
        get
        {
            return this.bits.Bits(51, 0);
        }
    }

    public IEnumerable MantissaBits
    {
        get
        {
            return this.bits.BitSeq(51, 0);
        }
    }

    public static implicit operator MyDouble(double d)
    {
        return new MyDouble(d);
    }
}

static class Extensions
{
    public static int Bit(this ulong x, int bit)
    {
        return (int)((x >> bit) & 0x01);
    }

    public static ulong Bits(this ulong x, int high, int low)
    {
        x <<= (63 - high);
        x >>= (low + 63 - high);
        return x;
    }

    public static IEnumerable<int> BitSeq(this ulong x, int high, int low)
    {
        for (int bit = high; bit >= low; --bit)
            yield return x.Bit(bit);
    }

    public static Rational Exp(this Rational x, int y)
    {
        Rational result;
        Rational.Power(x, y, out result);
        return result;
    }

    public static string ToDecimalString(this Rational x)
    {
        var sb = new StringBuilder();
        x.AppendDecimalString(sb, 50000);
        return sb.ToString();
    }

    public static string Join<T>(this IEnumerable seq)
    {
        return string.Concat(seq);
    }
}

You can check out part one here. Enjoy!

static double MyRound(double d)
{
  return Math.Floor(d + 0.5);
}

Is it correct? Can you find a value for which it does not give the mathematically correct value?²

UPDATE: The answer is in the comments, so if you don't want spoilers, don't read the comments.

Next time on FAIC: The answer, of course.

However, apparently some of them are just a wee bit disorganized. I just had the following conversations:

[Ring ring]

Me: Hi, this is Eric.

Her: Hi, this is Barbara¹ at XYZ Recruiters. How are you today?

Me: I am extremely well! How are you feeling today, Barbara?

[This seemed to completely flummox Barbara. Perhaps people who are interrupted at work by cold callers do not usually inquire as to her health?]

Her: Uh. Um. Me? Uh, I'm fine I guess! Thanks for asking!

[Brief pause -- OK, I guess keeping this conversation going is up to me.]

Me: So what's up, Barbara?

Her, back to the script: There's a small company in downtown Seattle that is 60% ex-Microsoft people and they're looking to hire C++ devs. They've just landed a big contract with Foo corp.

Me: Well thanks for thinking of me Barbara. I'm happy to speak with you but to be fair I first must warn you that I am intensely loyal.

Her: Oh. Well, thanks for your time. Bye.

Me: Bye!

Wow, she didn't put up a fight at all. Maybe the economy isn't picking up so much. Four years ago recruiters - who were for some reason invariably female - would flirt with me and then try to get me interested in crappy database admin jobs in the Cayman Islands, of all places.

Ah well, back to work. I resolve an old bug that got fixed a while back but was never updated in the bug database. I start looking at another bug and researching the history of a particular code change. We've made a minor change to the formatting of an XML file and Mario wants to know whether that was by design or an accident, when...

[Ring Ring -- hey, the caller id looks familiar...]

Me: Hi, this is Eric.

Her: Hi this is...

Me: Barbara at XYZ recruiters?

Her: Uh, yes. How...

Me: What's new?

[Like Hobbes, I love the moment of dawning comprehension. Barbara hits it.]

Her: Wait... did I call you already?

Me: Yes, about ten minutes ago.

Her, trying to place me: Uh… are you a C++ developer?

Me: Why yes I am as a matter of fact!

Her, paging in at last: You're the one who's "intensely loyal", right?

Me: Indeed. And I still am. Cold-calling recruiting really is kind of about looking for disloyal people, isn't it? People who will just pick up when something better comes along, right?

Her: Hey, some people are looking for a change! Wanting change in your life doesn't make you disloyal, does it?

Me: Well, you're the expert. I'll take your word for it.²

This brought back fond memories of my teenage days. Unlike my crazy friends, I never prank called people, but I prank answered them all the time.³ Having two voice lines in the house (one for my 300 baud modem on the Commodore 64, of course) led to ample opportunities for consecutive calls from clueless telemarketers. On the second call I'd just answer with "Don't say anything! I have a mysterious fifth sense! My psychic powers tell me that your name is Helen, and you want to sell me... magazine subscriptions! Yes?"

Freaked them out every time.

The problem with my algorithm, as David points out in the comments, is that there are 52! possible orderings of a deck, which is approximately 2²²⁶, but there are only 2³² possible "seeds" to the pseudo-random number generate that is Math.Random, and therefore only a tiny fraction of the decks can possibly be generated. In fact, the situation is far worse than that; when no seed is given, the class defaults to using the absolute value of Environment.TickCount which measures the number of milliseconds since the machine was turned on. However, this value is only updated approximately 60 times a second. The upshot of this is:

• Because the absolute value is taken, the actual seed is only drawn from 2³¹ possible values.
• Since a lot of people boot their machines when they need them rather than keeping them on all the time, the seed is typically much less than the number of milliseconds in a day.
• If Random is seeded twice and you manage to work out one of the seeds, you might be able to cut down the problem of figuring out the other by a factor of 60 or so -- not that it matters; as we've seen, brute-forcing is fast.

There are 52 x 51 x 50 x 49 x 48 = about 2²⁸ possible first five cards of a deck, and 2³¹ possible seeds. That means that on average there will be only 2³ decks that can possibly be generated by this algorithm that start with those five cards, out of the actual 2¹⁹⁷ decks that could start with those five cards! And when given a sixth card, it is almost always going to be possible to find the unique deck that can be generated using this algorithm.

If you're going to be shuffling a deck of cards by choosing a random permutation and there is a need for actual unpredictability then you should use a random or pseudo-random number generator that has at least 226 bits of "entropy" in the seed. The RandomNumberGenerator class is suitable for this purpose, Math.Random is not.

Next time on FAIC: I've been asked to write a series of articles for another site and they'll take me a bit to work out, so I'll probably skip the next couple of episodes of FAIC. More bulletins as events warrant.

39: Ace of Clubs
11: Queen of Spades
29: Four of Diamonds
5: Six of Spades
20: Eight of Hearts
15: Trey of Hearts
3: Four of Spades
16: Four of Hearts
33: Eight of Diamonds
43: Five of Clubs
49: Jack of Clubs
1: Deuce of Spades
22: Ten of Hearts
27: Deuce of Diamonds
25: King of Hearts
2: Trey of Spades
44: Six of Clubs
6: Seven of Spades
21: Nine of Hearts
17: Five of Hearts
35: Ten of Diamonds
12: King of Spades
48: Ten of Clubs
4: Five of Spades
41: Trey of Clubs
42: Four of Clubs
51: King of Clubs
36: Jack of Diamonds
7: Eight of Spades
28: Trey of Diamonds
18: Six of Hearts
0: Ace of Spades
13: Ace of Hearts
14: Deuce of Hearts
10: Jack of Spades
9: Ten of Spades
24: Queen of Hearts
50: Queen of Clubs
19: Seven of Hearts
47: Nine of Clubs
37: Queen of Diamonds
34: Nine of Diamonds
40: Deuce of Clubs
45: Seven of Clubs
23: Jack of Hearts
31: Six of Diamonds
32: Seven of Diamonds
30: Five of Diamonds
46: Eight of Clubs
8: Nine of Spades
38: King of Diamonds
26: Ace of Diamonds

static Permutation RandomPermutation(int size, Random random)
{
  return Permutation.NthPermutation(size, RandomFactoradic(size, random));
}

Is this actually a correct algorithm for generating a random permutation? Give it some thought.

Turns out, surprisingly, no, it is not correct. Now, there's no flaw that I'm aware of in the logic or the mathematics that underlies it. Rather, the problem is that the Random type is a big fat lie. We'd like to have the property that a random permutation generator produces every possible permutation with equal likelihood; that is, it should be unbiased. This algorithm is deeply biased because Random is not a strong enough source of random numbers. Some permutations are less likely than others.

So what's the big deal? If there were just a small bias, that would be unfortunate but not fatal. It's actually worse than that. With this bad source of randomness, given a portion of a permutation, you can work out what the rest of the permutation likely is.

And that's my challenge to you. I've posted some code below, putting together everything we've seen so far in this series. The first five card dealt when I ran the program were:

39: Ace of Clubs
11: Queen of Spades
29: Four of Diamonds
5: Six of Spades
20: Eight of Hearts

My challenge to you is: what are the next five cards that I generated? And.... go!

using System;
using System.Collections;
using System.Collections.Generic;
using System.Linq;
using System.Numerics;

public static class Extensions
{
    public static IEnumerable<T> InsertAt<T>(
      this IEnumerable<T> items, int position, T newItem)
    {
        if (items == null)
            throw new ArgumentNullException("items");
        if (position < 0)
            throw new ArgumentOutOfRangeException("position");
        return InsertAtIterator<T>(items, position, newItem);
    }

    private static IEnumerable<T> InsertAtIterator<T>(
        this IEnumerable<T> items, int position, T newItem)
    {
        int index = 0;
        bool yieldedNew = false;
        foreach (T item in items)
        {
            if (index == position)
            {
                yield return newItem;
                yieldedNew = true;
            }
            yield return item;
            index += 1;
        }
        if (index == position)
        {
            yield return newItem;
            yieldedNew = true;
        }
        if (!yieldedNew)
            throw new ArgumentOutOfRangeException("position");
    }
}

struct Permutation : IEnumerable<int>
{
    public static Permutation Empty { get { return empty; } }
    private static Permutation empty = new Permutation(new int[] { });
    private int[] permutation;
    private Permutation(int[] permutation)
    {
      this.permutation = permutation;
    }

    private Permutation(IEnumerable<int> permutation) 
      : this(permutation.ToArray()) { }

    private static BigInteger Factorial(int x)
    {
      BigInteger result = 1;
      for (int i = 2 ; i <= x ; ++i)
        result *= i;
      return result;       
    }

    public static Permutation NthPermutation(int size, BigInteger index)
    {
      if (index < 0 || index >= Factorial(size))
        throw new ArgumentOutOfRangeException("index");
      if (size == 0)
        return Empty;
      BigInteger group = index / size;
      bool forwards = group % 2 != 0;
      Permutation permutation = NthPermutation(size - 1, group);
      int i = (int) (index % size);                      
      return new Permutation(
        permutation.InsertAt(forwards ? i : size - i - 1, size - 1));
    }

    // Produce a random number between 0 and n!-1
    static BigInteger RandomFactoradic(int n, Random random)
    {
      BigInteger result = 0;
      BigInteger radix = 1;
      for (int i = 1; i < n; ++i)
      {
        // We need a digit between 0 and i.
        result += radix * random.Next(i+1);
        radix *= (i+1);
      }
      return result;
    }
    public static Permutation RandomPermutation(int size, Random random)
    {
      return NthPermutation(size, RandomFactoradic(size, random));
    }
    public int this[int index]
    {
      get { return permutation[index]; }
    }
    public IEnumerator<int> GetEnumerator()
    {
      foreach (int item in permutation)
        yield return item;
    }
    System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator()
    {
      return this.GetEnumerator();
    }
    public int Count { get { return this.permutation.Length; } }
    public override string ToString()
    {
      return string.Join<int>(",", permutation);
    }
}

class Program
{
    static void Main()
    {
        string[] suits = { "Spades", "Hearts", "Diamonds", "Clubs" };
        string[] values = {"Ace", "Deuce", "Trey", "Four", "Five",
                       "Six", "Seven", "Eight", "Nine", "Ten",
                       "Jack", "Queen", "King" };
        Permutation p = Permutation.RandomPermutation(52, new Random());
        foreach (int i in p)
            Console.WriteLine("{0}: {1} of {2}", 
                i, values[i % 13], suits[i / 13]);
    }
}

It's free, it's online, but you have to "subscribe" to read it. Check it out!

Next time on FAIC: More on random permutations.

Photo by my colleague Bob.