Wednesday, November 05, 2008

Casting an array of value types

Warning ahead: this article explains a dirty trick that should only be used in very specific circumstances (when memory conditions are low). I’ve personally seen this used when loading a large image from file (as a byte array) and needing to pass this array to a piece of legacy code that only accepts an array of shorts. Due to the size of the image, it was not feasible to copy the data.

Casting arrays of value types can be problematic. Image that you need to cast a large array of unsigned integers to an array of signed integers.

Our first experiment might be like this:

   1: static int[] Cast(uint[] input)
   2: {
   3:     int[] result = new int[input.Length];
   4:     
   5:     Array.Copy(input, result, input.Length);    // throws ArrayTypeMismatchException
   6:  
   7:     return result;
   8: }

This does not work. We get an ArrayTypeMismatchException because the two types are not compatible (the rules for compatible types are explained here).

The solution is to use the Marshal.Copy() method. This requires unsafe code however.
Here is an example:

   1: static unsafe int[] Cast(uint[] input)
   2: {
   3:     int[] result = new int[input.Length];
   4:  
   5:     fixed(uint* pInput = input)
   6:     {
   7:         Marshal.Copy(new IntPtr(pInput),result,0, input.Length);
   8:     }
   9:  
  10:     return result;
  11: }

This seems to work, but instead of casting the array we really made a copy of the array. The differences are subtle but can be important:

  • the copy requires its own memory. If the array is very large, this can be a problem. I’ve encountered situations (e.g. in image processing) where this was not acceptable.
  • when the copy is modified, the original array is left untouched. This is not what should happen when we cast a variable.

Using a dirty trick it is possible to cast an array of value types without copying the data. Look at this sample code:

   1: // WARNING: this is a dirty trick. Use at your own risk!
   2: static int[] Convert(uint[] input)
   3: {
   4:     Converter converter = new Converter();
   5:     converter.UInts = input;
   6:     return converter.Ints;
   7: }
   8:  
   9: [StructLayout(LayoutKind.Explicit)]
  10: struct Converter
  11: {
  12:     [FieldOffset(0)]
  13:     public int[] Ints;
  14:  
  15:     [FieldOffset(0)]
  16:     public uint[] UInts;
  17: }

We’re using a Converter-struct that uses some interop-magic to define two fields that map to the same location in memory. Now we can access the same data as an int[] or as an uint[]. The data is not copied in any way!

Note that this is a dirty trick: we fool the compiler into thinking that the type of the array is changed but it really is not. You can verify this by calling GetType() on the returned array: it will return System.UInt32[].

This trick can be used in other situations as well, for instance to access an array of integers as an array of bytes (without copying the data).

Warning: if the size of the array to be converted is reasonably small, you should just copy the data! Only use this trick if you really cannot afford to copy the data! This trick violates the type-safety of the runtime and is not guarantee to work in all situations or in future versions of .NET!

4 comments:

Anonymous said...

You can also do the following to achieve the same result, without defining the separate struct:

static int[] Convert(uint[] input)
{
object inputObj = input;
return (int[])inputObj;
}

Cast the array to an object and then cast the object to the target type.

SB said...

to Anonymous:
The simple cast doesn't work, at least not for me, to convert bytes to ushort.
Kristof's code does the trick, great solution, thanks!

simi said...

hi,

First of all. Thanks very much for your useful post.

I just came across your blog and wanted to drop you a note telling you how impressed I was with the information you have posted here.

Please let me introduce you some info related to this post and I hope that it is useful for .Net community.

There is a good C# resource site, Have alook

http://www.csharptalk.com/2009/09/c-array.html
http://www.csharptalk.com/2009/10/creating-arrays.html

simi

Ken Smith said...

That's sweet. I've been using Buffer.BlockCopy() to do similar stuff, but it always requires a copy of the underlying data. This is a nice way to do the old-style C casts in C# - until MS pulls the rug out from underneath us in the next version, presumably :-).