We figured we could just as well keep the core count up and since we weren't using all of them we'd only be billed for the CPU cores used.

Apparently we were wrong! But we couldn't figure out why the bills weren't going down so we started digging deeper.

We were seeing utilization for a 40 vCore instance in the low 10-20% ranges so we expected to be billed for some 4 - 8 cores used but we were being billed constantly for all 40 cores. How could that be? That's when I started really looking into the billing for HyperScale and my eyes stopped at this line:

Amount billed: vCore unit price * maximum (minimum vCores, vCores used, minimum memory GB * 1/3, memory GB used * 1/3)

After my brain processed that single line a bit i dawned on us...

Well ain't that sneaky! For a service like ours where data is always beeing queried and since SQL Server by default uses all the memory it has to cache data, the maximum of that is ALWAYS going to be "(memory GB used * 1/3)". This was confirmed when we looked at the Memory % Used graphs which pretty much constantly look like this:

Since our memory utilization is pretty much always maxed, we are always billed for the max vCores regardless of our actual vCore usage.

Again...

If your HyperScale server is querying a lot of data from your DB, you will be billed for the max vCore it is configured for regardless of your actual vCore usage, because you will be using all the available memory in SQL Server due to how the SQL Server cache works.

Ok, now that we made that discovery, how can we manage the max vCore configuration? Azure SQL does not have anything like Azure App Service autoscaling to scale core counts up and down based on load. That's when I found out that you can scale Azure SQL HyperScale with a SQL command!

-- Set the HyperScale DB to 8 vCores max
ALTER DATABASE <database_name> MODIFY (SERVICE_OBJECTIVE = 'HS_S_Gen5_8');

Ok, but how do we know when it's safe for us to scale things up and down then? Easy! You can also query performance counters for the SQL instance! So, we decided that we'd write a few procs to query this data and depending on the state of the DB we could determine if we should scale the database up/down or not do anything. So, we came up with this script!

CREATE PROCEDURE [dbo].[ScaleDatabase]
  @CoreCount              INT   = NULL, @CpuScaleUpThreshold FLOAT = 60, @CpuScaleDownThreshold FLOAT = 20,
  @WorkerScaleUpThreshold FLOAT = 10, @WorkerScaleDownThreshold FLOAT = 5, @MinutesToEvaluate INT = 5
AS
BEGIN
  -- Let's first check if we are allowed to scale
  DECLARE @IsSet BIT
  EXEC dbo.GetHibpFlag 'ScaleLocked', @IsSet OUTPUT
  IF @IsSet = 1
  BEGIN
    EXEC dbo.LogScalingEvent @Message = 'Scaling is locked. Not doing anything this time.'
    RETURN
  END

  -- Get the current workload
  DECLARE @currentWorkload NVARCHAR(20) = CAST(DATABASEPROPERTYEX(DB_NAME(), 'ServiceObjective') AS NVARCHAR),
    @nextWorkload          NVARCHAR(20) = NULL, @logMessage NVARCHAR(1000) = NULL

  PRINT 'Current workload: ' + @currentWorkload
  DECLARE @ScaleCommand NVARCHAR(1000)

  -- If the CoreCount is not provided, we will use automatic scaling
  IF @CoreCount IS NULL
  BEGIN
    -- Automatic scaling
    DECLARE @StartDate DATETIME = DATEADD(MINUTE, -@MinutesToEvaluate, GETUTCDATE())
    DECLARE @StartTime DATETIME, @EndTime DATETIME, @AvgCpuPercent FLOAT, @MaxWorkerPercent FLOAT, @ScaleUp BIT,
      @ScaleDown       BIT, @DataPoints INT;

    WITH DbStats (TimeStamp, AvgCpuPercent, MaxWorkerPercent, ScaleUp, ScaleDown)
    AS
    (
      SELECT [TimeStamp] = [end_time], AvgCpuPercent = [avg_cpu_percent], MaxWorkerPercent = [max_worker_percent],
        -- If the CPU OR Worker percentage exceeds the scale up threshold, increment the ScaleUp counter
        ScaleUp = IIF([avg_cpu_percent] > @CpuScaleUpThreshold OR [max_worker_percent] > @WorkerScaleUpThreshold,
                  1.0,
                  0.0),
        -- If the CPU AND Worker percentage are below the scale down threshold, increment the ScaleDown counter
        ScaleDown = IIF([avg_cpu_percent] < @CpuScaleDownThreshold AND [max_worker_percent] < @WorkerScaleDownThreshold,
                    1.0,
                    0.0)
        FROM sys.dm_db_resource_stats s
    )
    SELECT @StartTime = MIN(TimeStamp), @EndTime = MAX(TimeStamp), @AvgCpuPercent = AVG(AvgCpuPercent),
      @MaxWorkerPercent = MAX(MaxWorkerPercent), @ScaleUp = CONVERT(BIT, ROUND(MAX(ScaleUp), 0)),
      @ScaleDown = CONVERT(BIT, ROUND(MIN(ScaleDown), 0)), @DataPoints = COUNT(*)
      FROM DbStats
     WHERE TimeStamp BETWEEN @StartDate AND GETUTCDATE()

    DECLARE @EndDate DATETIME = DATEADD(MINUTE, 10, GETUTCDATE())
    IF @ScaleUp = 1
    BEGIN
      EXEC dbo.GetNextScaleWorkload 1, @nextWorkload OUTPUT
      IF @nextWorkload IS NULL
      BEGIN
        SELECT @logMessage = N'Already at highest scale. Not doing anything this time.'
        EXEC dbo.LogScalingEvent @Message = @logMessage
      END
      ELSE
      BEGIN
        SELECT @logMessage = CONCAT(
                               'Scaling up from ',
                               @currentWorkload,
                               ' to ',
                               @nextWorkload,
                               ' due to high CPU or worker utilization (CPU: ',
                               FORMAT(@AvgCpuPercent, 'N2'),
                               '%, Workers: ',
                               FORMAT(@MaxWorkerPercent, 'N2'),
                               '%)')
        SELECT @ScaleCommand = CONCAT(
                                 'ALTER DATABASE ', DB_NAME(), ' MODIFY (SERVICE_OBJECTIVE = ''', @nextWorkload, ''')')
        EXEC dbo.SetHibpFlag 'ScaleLocked', 1, @EndDate
        EXEC sp_executesql @ScaleCommand
        EXEC dbo.LogScalingEvent @Message = @logMessage
      END
    END
    ELSE IF @ScaleDown = 1
    BEGIN
      EXEC dbo.GetNextScaleWorkload 0, @nextWorkload OUTPUT
      IF @nextWorkload IS NULL
      BEGIN
        SELECT @logMessage = N'Already at lowest scale. Not doing anything this time.'
        EXEC dbo.LogScalingEvent @Message = @logMessage
      END
      ELSE
      BEGIN
        SELECT @logMessage = CONCAT(
                               'Scaling down from ',
                               @currentWorkload,
                               ' to ',
                               @nextWorkload,
                               ' due to low utilization (CPU: ',
                               FORMAT(@AvgCpuPercent, 'N2'),
                               '%, Workers: ',
                               FORMAT(@MaxWorkerPercent, 'N2'),
                               '%)')
        SELECT @ScaleCommand = CONCAT(
                                 'ALTER DATABASE ', DB_NAME(), ' MODIFY (SERVICE_OBJECTIVE = ''', @nextWorkload, ''')')
        EXEC dbo.SetHibpFlag 'ScaleLocked', 1, @EndDate
        EXEC sp_executesql @ScaleCommand
        EXEC dbo.LogScalingEvent @Message = @logMessage
      END
    END
    ELSE
    BEGIN
      SELECT @logMessage = CONCAT(
                             'No scaling action taken. CPU: ',
                             FORMAT(@AvgCpuPercent, 'N2'),
                             '%, Worker: ',
                             FORMAT(@MaxWorkerPercent, 'N2'),
                             '%. Current workload: ',
                             @currentWorkload)
      EXEC dbo.LogScalingEvent @Message = @logMessage
    END
  END
  ELSE
  BEGIN
    -- Manual scaling
    SELECT @nextWorkload = Workload
      FROM dbo.ScaleWorkloads
     WHERE Cores = @CoreCount

    IF @nextWorkload IS NULL
    BEGIN
      EXEC dbo.LogScalingEvent @Message = 'Invalid core count provided. No action taken.'
      RETURN
    END
    ELSE
    BEGIN
      SELECT @logMessage = CONCAT('Manually scaling from ', @currentWorkload, ' to ', @nextWorkload)
      SELECT @ScaleCommand = CONCAT(
                               'ALTER DATABASE ', DB_NAME(), ' MODIFY (SERVICE_OBJECTIVE = ''', @nextWorkload, ''')')
      EXEC sp_executesql @ScaleCommand
      EXEC dbo.LogScalingEvent @Message = @logMessage
    END
  END
END
GO

What this script basically does, is look at the performance data we have for a time period, and depending on the provided thresholds, will scale the HyperScale database up or down. It'll also give the DB time to stabilize after scaling operations by locking the scale for about 10 minutes after scaling events. We can configure all of this by sending different parameters as we iterate on this to find our optimal configuration.

However, there was one more problem. Azure SQL HyperScale does not support SQL Agent jobs so there is no built-in way to run this automatically or on a schedule. However, Azure does provide a service called Elastic Jobs which can run SQL statements against SQL databases on a schedule. Perfect!

So we created an elastic job to run the stored procedure every minute. Here is an example of what the logging output looks like:

Here we see the procedure decide to automatically scale the max vCores down from 20 to 18 due to low CPU and worker utilization. Success!

This has made our lives a lot easier, although I do wish that Azure SQL HyperScale had a way to do this automatically.

Hope this helps!

public class ManualResetEventSlimChecks
{
    private ManualResetEventSlim _set = new ManualResetEventSlim(true);

    [Benchmark]
    public void CheckSetBit()
    {
        if (!_set.IsSet)
        {
            _set.Wait();
        }
    }

    [Benchmark]
    public void WaitZero()
    {
        if (!_set.Wait(0))
        {
            _set.Wait();
        }
    }

    [Benchmark(Baseline = true)]
    public void Wait()
    {
        _set.Wait();
    }
}

This yielded the following results:

BenchmarkDotNet=v0.12.1, OS=Windows 10.0.21313
Intel Core i7-10700 CPU 2.90GHz, 1 CPU, 16 logical and 8 physical cores
.NET Core SDK=5.0.103
  [Host]     : .NET Core 3.1.12 (CoreCLR 4.700.21.6504, CoreFX 4.700.21.6905), X64 RyuJIT
  Job-HJNSVS : .NET Framework 4.8 (4.8.4311.0), X64 RyuJIT
  Job-FBCCVP : .NET Core 3.1.12 (CoreCLR 4.700.21.6504, CoreFX 4.700.21.6905), X64 RyuJIT

The lesson here was, that if you have a ManualResetEventSlim that will most of the time be set and it is being checked in a high-throughput scenario you should first check the IsSet property before deciding to call Wait, because even though Wait does do the IsSet check itself, it still requires a method call to Wait which is more expensive than simply checking the IsSet bit first.

Hope this helps!

Brief introduction to the AMQP 0.9.1 Protocol

RabbitMQ uses the AMQP 0.9.1 protocol by default. In overly broad terms, every AMQP operation is defined as a command. Before transmission, commands are split into frames of a specified maximum size (often dictated by the server), and these frames are then serialized to bytes for transmission on the wire when the client and server communicate. AMQP uses network byte order representation/big-endian for its data types so you have to be careful when serializing them on little-endian systems like Windows, so you don't corrupt data and the client can be interoperable with other operating systems. Now there are certain nuances to the protocol, and I am oversimplifying things here, but this is the gist of how the AMQP protocol works.

Binary Serialization

To serialize data to/from RabbitMQ we need to be able to translate data into a binary format. This is called serialization and as I mentioned above, RabbitMQ uses network-byte-order.

Previously the library used a class called NetworkBinaryReader which inherited from BinaryReader and overrode certain methods. BinaryReader is a very old class in the .NET BCL and does not take endianness into account, and the library also assumed that it was running on little-endian systems so quite a lot of overhead went into serializing doubles and floats.

Here is an example of how NetworkBinaryReader previously implemented the ReadDouble method:

public override double ReadDouble()
{
    byte[] bytes = ReadBytes(8);
    byte temp = bytes[0];
    bytes[0] = bytes[7];
    bytes[7] = temp;
    temp = bytes[1];
    bytes[1] = bytes[6];
    bytes[6] = temp;
    temp = bytes[2];
    bytes[2] = bytes[5];
    bytes[5] = temp;
    temp = bytes[3];
    bytes[3] = bytes[4];
    bytes[4] = temp;
    return TemporaryBinaryReader(bytes).ReadDouble();
}

This was changed to get rid of the temporary MemoryStreams (used in the TemporaryBinaryReader method above) and byte arrays to reduce allocations.

Here is an example of how we are now reading Double from a ReadOnlySpan<byte>:

[MethodImpl(MethodImplOptions.AggressiveInlining)]
internal static double ReadDouble(ReadOnlySpan<byte> span)
{
    if (span.Length < 8)
    {
        throw new ArgumentOutOfRangeException(nameof(span), "Insufficient length to decode Double from memory.");
    }

    uint num1 = (uint)((span[0] << 24) | (span[1] << 16) | (span[2] << 8) | span[3]);
    uint num2 = (uint)((span[4] << 24) | (span[5] << 16) | (span[6] << 8) | span[7]);
    ulong val = ((ulong)num1 << 32) | num2;
    return Unsafe.As<ulong, double>(ref val);
}

This resulted in a dramatic decrease in allocations as well as CPU savings. Here is an example of BenchmarkDotNet benchmarks I created for verification.

Double

Single

Serializing data and reusing buffers

Serializing data types is one thing. Serializing the AMQP data structures composed of that data is another thing. Messages are serialized into what's called Frames. Frames consist of a frame header, channel number, frame size and then the eventual frame payload (our data) and ends with a frame-end marker byte.

This brings up an interesting problem, which stems from the frame size. The frame size appears in the structure before the actual payload does, which makes sense because we need the consumers of the protocol to know how many bytes are in the payload they are about to read. But, since our payload can contain all sorts of data as well as header properties, and data types are serialized differently, it's hard for us to know the size of the serialized payload before we have actually serialized all the data in the message.

The obvious solution: MemoryStream

Previously, this was solved in a way many of us would think of. The client library created a MemoryStream, which was as the output stream for the NetworkBinaryWriter class used to serialize the data. After all the data had been written, it was easy to keep track of how many bytes had been written to the MemoryStream, seek back to the correct header position and write the payload size. Job done!

This method did its job, and most importantly did it correctly, but it had the drawback that the entire frame had to be buffered as it was being serialized, and as the MemoryStream grew bigger, new backing arrays needed to be allocated, memory copied etc. until the entire frame was correctly serialized and fit into the buffer, and eventually the resulting array was output to the network.

A better solution: MemoryPool<byte>

This was a noticeably big part of where allocations were being made so it became one of the focus points. When I looked at the code I figured, since I already had all the data, that I didn't need to serialize all of it to know how big it would be. I could simply walk through the data structure and calculate the required buffer size. We know a byte is one byte, a float is four bytes, int is four bytes etc. The biggest problem was strings. Given an arbitrary string, how do I know how many bytes it'll occupy when it's encoded using UTF8? Luckily for us, such a method already exists (Encoding.GetMaxByteCount), so that problem was solved as well.

So, what I decided to do, was to simply calculate how big each frame would need be at minimum when serialized, utilize System.Buffers.MemoryPool<T> from the System.Memory NuGet packages to fetch a pooled byte array of the minimum size, and then serialize the data directly to that pooled array, write it to the network socket and eventually return the array to the pool to be reused later. This would allow the client to eliminate any memory churn when serializing the data since data would be serialized to reusable byte arrays.

Now, you may wonder why I didn't use one of the many libraries that provide implementations of MemoryStream that reuse pooled arrays, like Microsoft.IO.RecyclableMemoryStream? The answer is future proofing! Moving to System.Memory gives the library access to Memory<T> and Span<T>, which are a mainstay in .NET Core, and will make the migration to System.IO.Pipelines a lot easier. So, stay tuned because there is even more good stuff coming in later versions :)

Armed with this knowledge and idea, let's take a closer look at some of the improvements made and the impact it had!

Converting strings to UTF8 bytes

One of the first things I noticed was that when strings were being serialized, they were converted directly to a UTF8 byte array. Obviously, this will create a new array to contain the UTF8 bytes. However, strings can also be serialized to an existing array and given an offset to where to start to write the bytes and we'll also know how many bytes are eventually written. This of course fit in extremely well with the pooled array idea.

This is how strings were serialized to a stream before:

public static void WriteShortstr(NetworkBinaryWriter writer, string value)
{
    byte[] bytes = Encoding.UTF8.GetBytes(value);
    writer.Write((ushort) bytes.Length);
    writer.Write(bytes);            
}

And this is how it's now serialized to a Memory<byte>

public static int WriteShortstr(Memory<byte> memory, string val)
{
    int stringBytesNeeded = Encoding.UTF8.GetByteCount(val);
    if (MemoryMarshal.TryGetArray(memory.Slice(1, stringBytesNeeded), out ArraySegment<byte> segment))
    {
        memory.Span[0] = (byte)stringBytesNeeded;
        Encoding.UTF8.GetBytes(val, 0, val.Length, segment.Array, segment.Offset);
        return stringBytesNeeded + 1;
    }

    throw new WireFormattingException("Unable to get array segment from memory.");
}

Now, granted, the new code is a little more verbose, but it is MUCH more efficient. Here is an example benchmark converting the first "Lorem Ipsum" paragraph to UTF8 bytes.

Arrays

As we have seen, the benefits of pooling and reusing arrays on code hot paths can be quite dramatic. This is mostly the result of freeing up our precious CPU cycles from being used by the Garbage Collector, cleaning up all our temporary objects. Having a garbage collector is one of the benefits of a managed language such as C#, but not having to use the Garbage Collector is even better.

I was able to take advantage of pooled arrays in the RabbitMQ client in several places. One of them was in the serialization as I mentioned above, where we could estimate the minimum amount of memory we'd need to serialize a frame, "rent" that amount of memory from our array pool, serialize the data directly into the array instead of using a MemoryStream + BinaryWriter object, and then send that array directly to the network Socket where it was sent to the server. Then we could return the array to the pool, rinse, and repeat. ‌

But what about the messages received from the server? Could I take advantage of them there.

YES!

After thinking about it for a while, I realized the the message payloads were being sent and received as byte arrays (byte[]). After talking it over the the RabbitMQ client maintainers, we decided that we'd make the change to have the message payloads represented as Memory<byte> instead. That allows us to use pooled arrays for messages received, copy the payload into another pooled array before being sent to the message event handlers, and then we could return them to the pool once the consumers had processed the messages!

Thankfully this was easy to solve using the MemoryPool<T> type. It allows us to easily rent and return pooled arrays using the IDisposable pattern. To give you an example of how to use it, let's imagine that we need calculate the SHA1 hash for a UTF8 string. As an example we'll use the same first "Lorem Ipsum" paragraph as an example.

Using the easy and convenient way:

private SHA1 hasher = SHA1.Create();

public byte[] GetByteArray()
{
    return hasher.ComputeHash(Encoding.UTF8.GetBytes(LoremIpsum));
}

Using MemoryPool<byte>:

private SHA1 hasher = SHA1.Create();
// Emulating a reusable pooled array to put the calculated hashes into
private Memory<byte> hashedBytes = new Memory<byte>(new byte[20]);

public Memory<byte> WriteToMemoryWithGetByteCount()
{
    int numCharactersNeeded = Encoding.UTF8.GetByteCount(LoremIpsum);
    // Let's use pooled memory to put the converted UTF8 bytes into. This is
    // wrapped in a using statement so once we have finished calculating the hash,
    // we will return the array to the pool, since it implements IDisposable.
    using (IMemoryOwner<byte> memory = MemoryPool<byte>.Shared.Rent(numCharactersNeeded))
    {
        if (MemoryMarshal.TryGetArray(memory.Memory.Slice(0, numCharactersNeeded), out ArraySegment<byte> segment))
        {
            Encoding.UTF8.GetBytes(LoremIpsum, 0, LoremIpsum.Length, segment.Array, segment.Offset);
            hasher.TryComputeHash(segment, hashedBytes.Span, out int _);
            return hashedBytes;
        }

        throw new InvalidOperationException("Failed to get memory");
    }
}

Now again, the code is more verbose, but it is much more memory friendly.

Results!

After going a few rounds on this, finding more places where I could put this to practice, and submitting several PRs, this work was finally released in the 6.0.0 release of the RabbitMQ .NET Client NuGet package. I decided to write a small program to see the result and run it through the JetBrains dotMemory profiler to see the result. ‌

The program did the following:

• Set up two connections to RabbitMQ, one to publish messages and another to receive them.
• Used publisher confirms
• Sent 50.000 messages, in bathes of five hundred messages at a time.
• Used 512 byte and 16kb message payloads to test memory usage.
• Ran until it had received those 50.000 messages back.

Here are the results:

As you can see, the memory usage when sending and receiving the events is now pretty much constant, regardless of the payload size. This is a massive improvement and frees up a lot of valuable CPU cycles and memory churn for other tasks.

What really made me happy with this release, was the fact that the API change only very slightly, and in most cases required little or no changes at all for the consumers of the package.

This is however not the end of this performance work by any means. There are more improvements in the pipeline (hint hint) that are sure to bring even more gains, for the benefit of everyone, so stay tuned :)

The problem is, by default, the JWT authentication handler in ASP.NET Core tries to communicate with the issuer defined in the JWT token to download the appropriate metadata needed to validate the tokens, but in our case we didn't want to be dependent on that when running through the tests, but we still wanted to make sure our authentication policies worked as intended.

I found out that by setting the Configuration property in the JwtBearerOptions we were able to short-circuit this behavior and make the JWT authentication handler skip the metadata download step.

Here's how!

First of all, we created a static helper class in our integration test project that initializes the required security keys and algorithms to create and sign our mocked JWT tokens. We then, through the standard service configuration, initialize the JWT authentication handler with that same information, making it possible for it to validate our generated JWT tokens.

It's quite simple really!

First, let's create our static helper class:

public static class MockJwtTokens
{
    public static string Issuer { get; } = Guid.NewGuid().ToString();
    public static SecurityKey SecurityKey { get; }
    public static SigningCredentials SigningCredentials { get; }

    private static readonly JwtSecurityTokenHandler s_tokenHandler = new JwtSecurityTokenHandler();
    private static readonly RandomNumberGenerator s_rng = RandomNumberGenerator.Create();
    private static readonly byte[] s_key = new byte[32];

    static MockJwtTokens()
    {
        s_rng.GetBytes(s_key);
        SecurityKey = new SymmetricSecurityKey(s_key) { KeyId = Guid.NewGuid().ToString() };
        SigningCredentials = new SigningCredentials(SecurityKey, SecurityAlgorithms.HmacSha256);
    }

    public static string GenerateJwtToken(IEnumerable<Claim> claims)
    {
        return s_tokenHandler.WriteToken(new JwtSecurityToken(Issuer, null, claims, null, DateTime.UtcNow.AddMinutes(20), SigningCredentials));
    }
}

Here we define our mock JWT issuer, it's security key, and create a simple helper method that accept claims and generates a JWT token with those claims. We just set our JWT tokens to be valid for 20 minutes.

Now all we need to do is hook this up to the JWT authentication handler and that was as simple as overriding a few settings when the tests are being initialized.

public class BaseIntegrationTest : WebApplicationFactory<Startup>
{
    protected override void ConfigureWebHost(IWebHostBuilder builder)
    {
        builder.ConfigureTestServices(ConfigureServices);
        builder.ConfigureLogging((WebHostBuilderContext context, ILoggingBuilder loggingBuilder) =>
        {
            loggingBuilder.ClearProviders();
            loggingBuilder.AddConsole(options => options.IncludeScopes = true);
        });
    }

    protected virtual void ConfigureServices(IServiceCollection services)
    {
        services.Configure<JwtBearerOptions>(JwtBearerDefaults.AuthenticationScheme, options =>
        {
            var config = new OpenIdConnectConfiguration()
            {
                Issuer = MockJwtTokens.Issuer
            };

            config.SigningKeys.Add(MockJwtTokens.SecurityKey);
            options.Configuration = config;
        });
     }
}

We now have a base class that we can have our integration tests inherit from to customize the configuration further. The trick here is to set the Configuration property on the JwtBearerOptions to the values defined in our mock. When that value has been set, the JWT authentication handler will see that it already has the information it needs to validate the JWT tokens we generate, and will not try to download the required metadata. This is documented behavior:

Configuration provided directly by the developer. If provided, then MetadataAddress and the Backchannel properties will not be used. This information should not be updated during request processing.

Hope this helps :)

Hello friends!

Long time, no blog, but now it's time to for something fun.

I happened upon the awesome BenchmarkDotNet library the other day, which makes it incredibly easy to write good and solid .NET benchmarks.

You can find a good tutorial on how to get started using it by looking at the GitHub page for the project, but I'm just going to dive right into it.

Introduction

In the services we write at CCP Games we often have to do geographical lookups based on the visiting users IP address. For this we use the IP2Location database which saves the IP address lookups as one large unsigned integer instead of a dot-delimited string of four bytes. This makes sense from a database perspective, because it means that the IP address ranges can more easily be stored in an integer column and makes more sense to be able to quickly find out if an IP address is within a specific range.

This does however require an IP address that's represented as a string to be converted into an integer, which means splitting up the string, and adding the elements of the string into an integer. This means that the IP address is split into 4 numbers and each of them represents one byte out of a 32-bit unsigned integer.

We had written a simple helper class to do this.

Code

public static class IP2LocationHelper1
{
    public static uint IPAddressToInteger(string input)
    {
        uint[] elements = input.Split('.').Select(x => uint.Parse(x)).ToArray();
        return elements[0] * 256U * 256U * 256U + elements[1] * 256U * 256U + elements[2] * 256U + elements[3];
    }
}

Result

What this tells us is the following. Each call takes around 785 nanoseconds to execute and it allocates around around 95 bytes of memory. That might not seem like a lot but for a simple IP Address to integer conversion I feel it is rather much. So, we started to experiment.

Watch out for params!

Let's just break things done line by line, which is easy enough since we only have to lines at the moment.

The string.Split() method is actually defined with a params parameter accepting an array of characters as possible delimiters. What a params parameters does is, even provided with just one parameter, it always creates an array, and creating a new array every time we call the method is obviously expensive so we'll start by modifying it into a static variable.

Code

public static class IP2LocationHelper2
{
    private static readonly char[] Delimiter = new[] { '.' };

    public static uint IPAddressToInteger(string input)
    {
        uint[] elements = input.Split(Delimiter).Select(x => uint.Parse(x)).ToArray();
        return elements[0] * 256U * 256U * 256U + elements[1] * 256U * 256U + elements[2] * 256U + elements[3];
    }
}

Results

Small progress. We're not allocating as much but it's probably within margin of error though.

Using appropriate datatypes

Since an IP address is made up of four bytes (32 bits) there is no reason for us to parse the individual elements as int. Let's try converting them to byte and see where that takes us.

Code

public static class IP2LocationHelper3
{
    private static readonly char[] Delimiter = new[] { '.' };

    public static uint IPAddressToInteger(string input)
    {
        byte[] elements = input.Split(Delimiter).Select(x => byte.Parse(x)).ToArray();
        return elements[0] * 256U * 256U * 256U + elements[1] * 256U * 256U + elements[2] * 256U + elements[3];
    }
}

Results

Steady improvements. But were have more to go still!

Why all the parsing?

For all four elements we are parsing the string representation into a byte using byte.Parse. This involves a lot of checking which really shouldn't be necessary. Since a byte can only have 256 different decimal string representations we might as well create a Dictionary<string, byte> mapping the possible string to their byte values. Let's create the following static variable and initialize it in the static constructor.

Code

public static class IP2LocationHelper4
{
    private static readonly char[] Delimiter = new[] { '.' };
    private static readonly Dictionary<string, byte> elementMapper = new Dictionary<string, byte>();

    static IP2LocationHelper4()
    {
        for (uint i = 0; i < 256; i++)
        {
            elementMapper[i.ToString()] = (byte)i;
        }
    }

    public static uint IPAddressToInteger(string input)
    {
        byte[] elements = input.Split(Delimiter).Select(x => elementMapper[x]).ToArray();
        return elements[0] * 256U * 256U * 256U + elements[1] * 256U * 256U + elements[2] * 256U + elements[3];
    }
}

Results

By caching of data and pre-parsing the possible string the performance has drastically improved by 40%. But we're not done yet!

Avoid LINQ in performance critical code

It's been known that although LINQ is great tool to make complex filtering and composition easier to read it does come with a performance impact. Let's try replacing the Select() and ToArray() methods with a standard new.

Code

public static class IP2LocationHelper5
{
    private static readonly char[] Delimiter = new[] { '.' };
    private static readonly Dictionary<string, byte> elementMapper = new Dictionary<string, byte>();

    static IP2LocationHelper5()
    {
        for (uint i = 0; i < 256; i++)
        {
            elementMapper[i.ToString()] = (byte)i;
        }
    }

    public static uint IPAddressToInteger(string input)
    {
        string[] stringElements = input.Split(Delimiter);
        byte[] elements = new[] { elementMapper[stringElements[0]], elementMapper[stringElements[1]], elementMapper[stringElements[2]], elementMapper[stringElements[3]] };
        return elements[0] * 256U * 256U * 256U + elements[1] * 256U * 256U + elements[2] * 256U + elements[3];
    }
}

Results

OK. That's a pretty big improvement right there! LINQ has a pretty hefty performance hit, and performance has improved by almost 60% now. But is there more we can do? Certainly, let's do some math!

Bit-shifting?

For the first three elements,we really are only shifting their bits around in an integer (multiplying an integer by 256 is really only doing a left shift of 8-bits). Fortunately, C# has the bit shifting operators << and >> (left and right shift respectively). Let's see if that improves our code in any way.

Code

public static class IP2LocationHelper6
{
    private static readonly char[] Delimiter = new[] { '.' };
    private static readonly Dictionary<string, uint> elementMapper = new Dictionary<string, uint>();

    static IP2LocationHelper6()
    {
        for (uint i = 0; i < 256; i++)
        {
            elementMapper[i.ToString()] = i;
        }
    }

    public static uint IPAddressToInteger(string input)
    {
        string[] stringElements = input.Split(Delimiter);
        uint[] elements = new[] { elementMapper[stringElements[0]], elementMapper[stringElements[1]], elementMapper[stringElements[2]], elementMapper[stringElements[3]] };
        return (elements[0] << 24) + (elements[1] << 16) + (elements[2] << 8) + elements[3];
    }
}

Results

Not a lot of change, although allocations seem to have been reduced and just a minor speed bump. Looking at this code made me realize another alternative. Just like we created the map to convert strings into bytes, why don't we create four maps that would map all possible strings to pre-shifted unsigned integers.

Pre-shifting with more maps

Let's do some more pre-mapping, now with pre-shifted bytes so we only have to do additions after splitting up the string elements.

Code

public static class IP2LocationHelper7
{
    private static readonly char[] Delimiter = new[] { '.' };
    private static readonly Dictionary<string, uint> element0Mapper = new Dictionary<string, uint>();
    private static readonly Dictionary<string, uint> element1Mapper = new Dictionary<string, uint>();
    private static readonly Dictionary<string, uint> element2Mapper = new Dictionary<string, uint>();
    private static readonly Dictionary<string, uint> element3Mapper = new Dictionary<string, uint>();

    static IP2LocationHelper7()
    {
        for (uint i = 0; i < 256; i++)
        {
            element0Mapper[i.ToString()] = i << 24;
            element1Mapper[i.ToString()] = i << 16;
            element2Mapper[i.ToString()] = i << 8;
            element3Mapper[i.ToString()] = i;
        }
    }

    public static uint IPAddressToInteger(string input)
    {
        string[] stringElements = input.Split(Delimiter);
        return element0Mapper[stringElements[0]] + element1Mapper[stringElements[1]] + element2Mapper[stringElements[2]] + element3Mapper[stringElements[3]];
    }
}

Results

Et voila! More allocations removed and small speed bump along with it. We'll make do with this for now.

Conclusion

The end results is quite something. Our method is now 60% faster than the original and allocates almost 30% less memory as well and we still maintained good readability on our code.

Things can be improved even further as almost all of the rest of the allocations are taking place in the string.Parse method, but then I feel we might be crossing the line between readibility and optimizations, but I'll leave it as an exercise for the reader to take that on ;)

This just goes to show that with a little thinking and some research, we can usually find better and more optimized ways of doing things without sacrificing code readability.

Also, I can't recommend BenchmarkDotNet enough, as it is a wonderful tool to help out with performance optimizations like these, by making it easy to show progress and difference between the different implementations.

Hope this helps!

I have been trying Application Insights for awhile now and while it is a fantastic tool I do feel it is missing several important performance counters out-of-the-box for .NET applications. Among them are several counters from the .NET CLR Memory category which I'm going to use as an example for this blog post.

Application Insights allows you to easily add performance counters to the list of counters it already measures, and that is done by some simple little changes to the ApplicationInsights.config file (for regular ASP.NET web sites).

If you take a look at the ApplicationInsights.config file you'll find the TelemetryModules section near the top and under it is a PerformanceCollectorModule where you can add your own performance counters.

Here is an example of the .NET CLR Memory counters I added (mostly relating to the .NET Garbage Collector):

<Counters>
  <Add PerformanceCounter="\.NET CLR Memory(??APP_CLR_PROC??)\# Bytes in all Heaps" ReportAs="Bytes in all Heaps" />
  <Add PerformanceCounter="\.NET CLR Memory(??APP_CLR_PROC??)\% Time in GC" ReportAs="Percent of time spent in GC" />
  <Add PerformanceCounter="\.NET CLR Memory(??APP_CLR_PROC??)\# Gen 0 Collections" ReportAs="Gen Zero Collections" />
  <Add PerformanceCounter="\.NET CLR Memory(??APP_CLR_PROC??)\# Gen 1 Collections" ReportAs="Gen One Collections" />
  <Add PerformanceCounter="\.NET CLR Memory(??APP_CLR_PROC??)\# Gen 2 Collections" ReportAs="Gen Two Collections" />
  <Add PerformanceCounter="\.NET CLR Memory(??APP_CLR_PROC??)\Allocated Bytes/second" ReportAs="Allocated Bytes per Second" />
  <Add PerformanceCounter="\.NET CLR Memory(??APP_CLR_PROC??)\Gen 0 Promoted Bytes/Sec" ReportAs="Bytes Promoted from Gen Zero per Second" />
  <Add PerformanceCounter="\.NET CLR Memory(??APP_CLR_PROC??)\Gen 1 Promoted Bytes/Sec" ReportAs="Bytes Promoted from Gen One per Second" />
  <Add PerformanceCounter="\.NET CLR Memory(??APP_CLR_PROC??)\Gen 1 heap size" ReportAs="Gen One Heap Size" />
  <Add PerformanceCounter="\.NET CLR Memory(??APP_CLR_PROC??)\Gen 2 heap size" ReportAs="Gen Two Heap Size" />
  <Add PerformanceCounter="\.NET CLR Memory(??APP_CLR_PROC??)\Promoted Memory from Gen 0" ReportAs="Promoted Memory from Gen Zero" />
  <Add PerformanceCounter="\.NET CLR Memory(??APP_CLR_PROC??)\Promoted Memory from Gen 1" ReportAs="Promoted Memory from Gen One" />
</Counters>

The ??APP_CLR_PROC?? string is a "magic variable" that the Application Insights runtime replaces with the identity of the CLR Process that runs the application to select the correct counters.

After adding these counters we can take a look at them using the Metrics Explorer in the Application Insights hub on the Azure Portal.

There is one caveat to remember as well. You need to make sure that the user running the website, for example the IIS Application Pool user, is a member of the local Performance Monitor Users group to be able to read performance counters from the system.

Adding an IIS application pool user to that group can be easily done with PowerShell like this (answer on Stack Overflow from Svein Fidjestøl):

$group = [ADSI]"WinNT://$Env:ComputerName/Performance Monitor Users,group"
$ntAccount = New-Object System.Security.Principal.NTAccount("IIS APPPOOL\ASP.NET v4.0")
$strSID = $ntAccount.Translate([System.Security.Principal.SecurityIdentifier])
$user = [ADSI]"WinNT://$strSID"
$group.Add($user.Path)

If you are running your website as a Windows Azure Web App, you don't need to worry about those permissions.

I hope this helps, and happy performance monitoring!

Docker is a fantastic tool, and the Docker Toolbox is a must to manage it.

If you install the Docker Toolbox on a Windows machine, the installer automatically installs Oracle Virtualbox to run the Docker virtual machine.

That always annoyed me, since Hyper-V has now been a part of the Windows Pro and Enterprise editions since Windows 8.

It is however possible to run the Docker Toolbox using Hyper-V straight out of the box.

First create a Virtual Switch in Hyper-V.

Then, start an Administrative Command Prompt and instead of using the standard documented method to create a Docker VM called default using Virtualbox:
docker-machine create --driver virtualbox default

you simply exchange the virtualbox driver with hyperv like so:
docker-machine create --driver hyperv default

That's all there is to it. If you are running a version of Windows that has Hyper-V, you can safely uninstall Virtualbox and just use Hyper-V instead to manage your Docker VMs.

Hope this helps!

Hello friends!

I've been taking a look at Microsoft's Application Insights recently and decided as a test to see how hard it would be to integrate with the Ghost blogging platform.

Application Insights is Microsoft's application performance monitoring platform. It is build into Azure and provides excellent insights into both server side as well as client side performance.

Turns out adding it to Ghost is relatively easy.

Preparation

First of all you need an account on Microsoft's Azure platform. Once you have that created you can log in to the Azure Portal. From there simply click New -> Developer Services -> Application Insights. Give it a name, select "Other (preview)" under Application Type, make sure "Pin to Startboard" is selected and click "Create".

Once that is completed, go to "Home" and find the Application Insights tile. It should have the name you gave it as well as a purple lightbulb icon. Clicking it should open up the Essentials dashboard. Now look for the Instrumentation Key which is a Guid and copy it to your clipboard, you will need it when configuring Ghost.

Ghost server side integration

First of all we need to add the necessary Application Insights node modules to the application. We do that by using the npm package tool. Simply go to the root folder of the website and run npm install applicationinsights.

The next step is to set Application Insights to start up during the Ghost startup. For that you have to edit index.js in Ghost's root directory.

Here is how my index.js looks like:

// # Ghost Startup
// Orchestrates the startup of Ghost when run from command line.
var express,
    ghost,
    parentApp,
    errors,
    appInsights;

// Make sure dependencies are installed and file system permissions are correct.
require('./core/server/utils/startup-check').check();

// Proceed with startup
express = require('express');
ghost = require('./core');
errors = require('./core/server/errors');
appInsights = require("applicationinsights");

// Initializes app insights
appInsights.setup("xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx").start();

// Create our parent express app instance.
parentApp = express();

// Call Ghost to get an instance of GhostServer
ghost().then(function (ghostServer) {
    // Mount our Ghost instance on our desired subdirectory path if it exists.
    parentApp.use(ghostServer.config.paths.subdir, ghostServer.rootApp);

    // Let Ghost handle starting our server instance.
    ghostServer.start(parentApp);
}).catch(function (err) {
    errors.logErrorAndExit(err, err.context, err.help);
});

See where I call appInsights.setup? That's where your instrumentation key goes, and calling this method will set up all the necessary server side interceptions so you'll start to see server side information such as request times, number of requests and failed requests.

Now, let's see what's needed for the client side integration.

Ghost client side integration

Next. Log in to your Ghost administration portal and got to Settings -> Code Injection. In the "Blog Header" section, add the following snippet, replacing the "xxxxxx" part with your instrumentation key)

<script type="text/javascript">
    var appInsights=window.appInsights||function(config){
        function s(config){t[config]=function(){var i=arguments;t.queue.push(function(){t[config].apply(t,i)})}}var t={config:config},r=document,f=window,e="script",o=r.createElement(e),i,u;for(o.src=config.url||"//az416426.vo.msecnd.net/scripts/a/ai.0.js",r.getElementsByTagName(e)[0].parentNode.appendChild(o),t.cookie=r.cookie,t.queue=[],i=["Event","Exception","Metric","PageView","Trace"];i.length;)s("track"+i.pop());return config.disableExceptionTracking||(i="onerror",s("_"+i),u=f[i],f[i]=function(config,r,f,e,o){var s=u&&u(config,r,f,e,o);return s!==!0&&t["_"+i](config,r,f,e,o),s}),t
    }({
        instrumentationKey:"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
    });
    
    window.appInsights=appInsights;
    appInsights.trackPageView();
</script>

And that's all there is to it. Now you should receive a bunch of metrics from your Ghost application, being it server side requets as well as browser and session information.

Hope this helps!