tips for writing numerical code in .NET

Bayes Server has an advanced library (API) for Bayesian networks which can be called by .NET languages such as C#, F#, VB.NET, managed C++, and many others. In this article we list some tips for writing numerical code in .NET languages. The tips are not specific to the Bayes Server API, but apply to writing numerical .NET code in general. In fact, many of the tips apply to other languages also. Many tips are accompanied by C# source code.

We do not consider performance issues in this article, but rather things to watch out for.

contents

• divide by zero
• signed zero
• infinity
• rounding errors
• numerical equality
• not a number (NaN)
• epsilon
• big numbers
• checked arithmetic
• single vs. double vs. decimal data types

divide by zero

Calculations involving floating point numbers do not raise exceptions, but those involving integer data types can.

To illustrate this, see the code below, where we divide a number by zero, first using floating point numbers and then integers. With floating point numbers the answer is +Infinity if the numerator is positive, -Infinity if the numerator is negative and NaN if the numerator is zero, while with integers a DivideByZero exception is raised.

using System;
using System.Text;
using System.Collections.Generic;
using System.Linq;
using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace NumericalCodeTips
{
    /// <summary>
    /// Divide by zero tests.
    /// </summary>
    [TestClass]
    public class DivideByZeroTest
    {
        /// <summary>
        /// Test that dividing a positive floating point number
        /// by zero equals +Infinity, and does not raise an exception.
        /// </summary>
        [TestMethod]
        public void PositiveFloatingPointDivideByZero()
        {
            double numerator = 12.0;
            double denominator = 0.0;

            double result = numerator / denominator;    // no exception raised

            Assert.IsTrue(double.IsPositiveInfinity(result));
        }

        /// <summary>
        /// Test that dividing a negative floating point number
        /// by zero equals -Infinity, and does not raise an exception.
        /// </summary>
        [TestMethod]
        public void NegativeFloatingPointDivideByZero()
        {
            double numerator = -12.0;
            double denominator = 0.0;

            double result = numerator / denominator;    // no exception raised

            Assert.IsTrue(double.IsNegativeInfinity(result));
        }

        /// <summary>
        /// Test that dividing zero by zero using floating point
        /// numbers equals NaN, and does not raise an exception.
        /// </summary>
        [TestMethod]
        public void ZeroFloatingPointDivideByZero()
        {
            double numerator = 0.0;
            double denominator = 0.0;

            double result = numerator / denominator;    // no exception raised

            Assert.IsTrue(double.IsNaN(result));
        }

        /// <summary>
        /// Test that dividing a positive integer by zero raises
        /// a <see cref="DivideByZeroException"/>.
        /// </summary>
        [TestMethod]
        [ExpectedException(typeof(DivideByZeroException))]
        public void PositiveIntegerDivideByZero()
        {
            int numerator = 12;
            int denominator = 0;

            int result = numerator / denominator;    // DivideByZeroException raised
        }

        /// <summary>
        /// Test that dividing a negative integer by zero raises
        /// a <see cref="DivideByZeroException"/>.
        /// </summary>
        [TestMethod]
        [ExpectedException(typeof(DivideByZeroException))]
        public void NegativeIntegerDivideByZero()
        {
            int numerator = -12;
            int denominator = 0;

            int result = numerator / denominator;    // DivideByZeroException raised
        }

        /// <summary>
        /// Test that dividing zero by zero using integers raises
        /// a <see cref="DivideByZeroException"/>.
        /// </summary>
        [TestMethod]
        [ExpectedException(typeof(DivideByZeroException))]
        public void ZeroIntegerDivideByZero()
        {
            int numerator = 0;
            int denominator = 0;

            int result = numerator / denominator;    // DivideByZeroException raised
        }
    }
}

signed zero

Since floating point numbers in .NET conform to IEEE 754, they allow the number zero to be positive (+0) or negative (-0). Although a test for equality on +0 and -0 will return true, they can lead to very different results when used in subsequent calculations, as demonstrated in the code sample below.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace NumericalCodeTips
{
    /// <summary>
    /// Demonstrates the concept of negative zero.
    /// </summary>
    [TestClass]
    public class SignedZeroTest
    {
        /// <summary>
        /// Demonstrates signed zero.
        /// </summary>
        [TestMethod]
        public void SignedZero()
        {
            double positiveZero = +0.0;
            double negativeZero = -0.0; // or 0.0 / -1.0;

            Assert.AreEqual(0, Math.Sign(negativeZero));

            Assert.IsTrue(positiveZero == negativeZero);    // a
                test for equality will return true

            // however they behave differently
            Assert.IsTrue(double.IsPositiveInfinity(1.0 / positiveZero));
            Assert.IsTrue(double.IsNegativeInfinity(1.0 / negativeZero));
        }
    }
}

infinity

Infinity is used to signal an overflow condition, i.e. when a number is too big (positive or negative) to be represented by the data type you are using. Infinite values are common in numerical programming, and cannot simply be ignored. It is also important to be aware of some subtleties when manipulating infinite values. The code below shows how infinite values can be created and some important properties, such as the fact that double.PositiveInfinity - double.PositiveInfinity does not equal zero.

Note that when testing for Infinity either Double.IsPositiveInfinity(), Double.IsNegativeInfinity() or Double.IsInfinity() should be used.

using System;
using System.Text;
using System.Collections.Generic;
using System.Linq;
using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace NumericalCodeTips
{
    /// <summary>
    /// Demonstrates some properties of infinite values.
    /// </summary>
    [TestClass]
    public class InfinityTest
    {
        /// <summary>
        /// Demonstrates how to generate infinite values.
        /// </summary>
        [TestMethod]
        public void Infinity()
        {
            Assert.IsTrue(double.IsPositiveInfinity(double.PositiveInfinity));
            Assert.IsTrue(double.IsNegativeInfinity(double.NegativeInfinity));
            Assert.IsTrue(double.IsPositiveInfinity(1.0 / 0.0));
            Assert.IsTrue(double.IsNegativeInfinity(-1.0 / 0.0));

            Assert.IsTrue(double.IsPositiveInfinity(double.MaxValue * 2.0));
        }

        /// <summary>
        /// Demonstrates some properties of infinite values
        /// to be aware of.
        /// </summary>
        [TestMethod]
        public void InfiniteProperties()
        {
            Assert.IsFalse(0.0 == (double.PositiveInfinity - double.PositiveInfinity));
            Assert.IsFalse(0.0 == (double.PositiveInfinity + double.NegativeInfinity));
            Assert.IsTrue(double.IsNaN(double.PositiveInfinity / double.PositiveInfinity));
        }
    }
}

rounding errors

The limited size of a floating point data type, means that numbers and arithmetic using floating point representations can only be performed with limited accuracy. For example 0.5 - 0.4 - 0.1 will not exactly equal 0.0, as shown in the code below. The difference between the approximate value and the expected mathematical quantity is call a rounding error (or round off error). This is standard behavior in programming languages, and cannot be ignored when writing scientific code.

There are a number of important side effects of rounding errors, which software engineers should be aware of. As described above, since 0.5-0.4-0.1 does not exactly equal zero when using floating point arithmetic, we must take care when testing for equality. Another problem can be caused when round off errors accumulate, often leading to disastrous consequences (e.g. the European Space Agency Ariane rocket launched in 1996).

Rounding errors can also be caused when converting from one data type to another. Thankfully .NET makes these issues less common due to the concept of implicit and explicit casts.

using System;
using System.Text;
using System.Collections.Generic;
using System.Linq;
using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace NumericalCodeTips
{
    /// <summary>
    /// Demonstrates rounding error.
    /// </summary>
    [TestClass]
    public class RoundingErrorTest
    {
        /// <summary>
        /// Demonstrates rounding errors due to the precision
        /// of floating point numbers and arithmetic.
        /// </summary>
        [TestMethod]
        public void RoundingError()
        {
            double x = 0.5 - 0.4 - 0.1;
            Assert.IsFalse(0.0 == x);
        }
    }
}

numerical equality

As explained in the section on rounding errors floating point arithmetic has limited accuracy, meaning that 0.5 - 0.4 - 0.1 will not exactly equal zero. Therefore care must be taken, testing for equality when numerical programming.

• If possible, remove the test for equality altogether.
• If possible change the test to use either <= or >=.
• Test that the value lies within a certain tolerance, in either absolute or relative (percentage difference) terms.

Note however that testing whether a value lies within certain bounds, is not always straight forward, as the choice of the tolerance, and whether to test the absolute or relative difference will depend on how many calculations have been performed, and the magnitude of the numbers involved. In some cases (e.g. unit tests) a simple small absolute tolerance can be used, but in more complex situations Knuth's algorithm for testing for equality, or a thorough numerical analysis may be required.

not a number (NaN)

System.Double.NaN represents the result of an invalid or undefined calculation. The code below demonstrates a number of ways to generate NaN, and how it is important to determine if a number is NaN using the function System.Double.IsNaN() rather than checking for equality.

When writing numerical code, it is important to check all eventualities of every calculation. For example, every time a division is performed, is it possible that the code might divide 0.0 by 0.0? An exception would not be raised, but the result would be invalid.

Although out of scope of this article, it should be mentioned that there are actually two types of NaN. The first is called a signalling NaN (SNaN or 1.#INF) denoting that the operation was invalid, and the second is a quiet NaN (QNaN or –1.#IND), denoting that the operation is undefined.

using System;
using System.Text;
using System.Collections.Generic;
using System.Linq;
using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace NumericalCodeTips
{
    /// <summary>
    /// Demonstrates Not a Number (NaN)
    /// </summary>
    [TestClass]
    public class NaNTest
    {
        /// <summary>
        /// Demonstrate some different ways of generating Not
        /// a Number (NaN)
        /// </summary>
        [TestMethod]
        public void NaN()
        {
            Assert.IsTrue(double.IsNaN(double.NaN));
            Assert.IsTrue(double.IsNaN(0.0 / 0.0));
            Assert.IsTrue(double.IsNaN(Math.Sqrt(-1.0)));
            Assert.IsTrue(double.IsNaN(double.PositiveInfinity / double.PositiveInfinity));
            Assert.IsTrue(double.IsNaN(double.PositiveInfinity - double.PositiveInfinity));
        }

        /// <summary>
        /// Demonstrate NaN equality and testing for NaN
        /// </summary>
        [TestMethod]
        public void NaNEquality()
        {
            double x = (0.0 / 0.0);

            Assert.IsFalse(x == Double.NaN);

            Assert.IsTrue(double.IsNaN(x));
        }
    }
}

epsilon

Often numerical code uses a constant known as machine epsilon, or the square root of machine epsilon, to compensate for the effects of rounding errors. Machine epsilon equals the smallest number such that (1.0 + machine epsilon) != 1.0, which equals Math.Pow(2.0, -53.0) for double precision numbers. However, machine epsilon is not the same as the constant System.Double.Epsilon (double.Epsilon in C#) which represents the smallest positive Double value that is greater than zero.

These two constants have very different values, but are often confused, especially when converting code from one language to another.

big numbers

If you require numbers that are larger than the basic data types, note that as of .NET 4.0 the System.Numerics namespace contains an implementation of a big integer, appropriately called BigInteger. This allows arbitrarily large integral values to be represented, albeit with the expected performance and storage penalties.

checked arithmetic

By default, exceptions are raised in .NET programs if an arithmetic operation leads to overflow or underflow using integral data types. There are two ways this can be changed.

• Using the \checked compiler option (in Visual Studio this can be changed via the project property page, build tab, by clicking the 'Advanced' button and changing the 'Check for arithmetic overflow/underflow' checkbox.)
• using the checked and unchecked C# keywords in a block of code.

It is important to note that floating point arithmetic does not raise exceptions, but instead uses the values double.PositiveInfinity, double.NegativeInfinity and double.NaN.

using System;
using System.Text;
using System.Collections.Generic;
using System.Linq;
using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace NumericalCodeTips
{
    /// <summary>
    /// Demonstrates checked and unchecked arithmetic.
    /// </summary>
    [TestClass]
    public class CheckedTest
    {
        /// <summary>
        /// Demonstrates that an exception will be thrown if
        /// using a checked code block.
        /// </summary>
        [ExpectedException(typeof(OverflowException))]
        [TestMethod]
        public void CheckedInteger()
        {
            checked
            {
                int x = int.MaxValue;
                x *= 2; // will
                    raise an OverflowException
            }
        }

        /// <summary>
        /// Demonstrates that an exception will NOT be thrown
        /// if using an unchecked code block.
        /// </summary>
        [TestMethod]
        public void UncheckedInteger()
        {
            unchecked
            {
                int x = int.MaxValue;
                x *= 2; // will
                    not raise an exception
            }
        }

        /// <summary>
        /// Demonstrates that an exception will NOT be thrown
        /// using floating point values in a checked code block.
        /// </summary>
        [TestMethod]
        public void CheckedDouble()
        {
            checked
            {
                double x = double.MaxValue;
                x *= 2;  // will
                    not raise an exception
            }
        }
    }
}

single vs. double vs. decimal data types

When writing numerical code in .NET, System.Double (double in C#) is usually the appropriate data type to use for numerical calculations.

In some circumstances, an additional single precision floating point version may be provided, in order to reduce memory and/or increase performance. Just switching to single precision for performance reasons, when memory requirements are not an issue, might not increase the performance as much as you expect (if at all), so profiling should always be employed.

Software that manipulates financial quantities, can use the System.Decimal data type if additional precision is required. However analytical applications will still tend to use double precision data types, even if they are financial in nature, as long as the additional precision is not required.

feedback

send feedback