A Vista update: integration between System.Transactions and the transacted file system and registry

I have written earlier (here and here) about how to integrate the transactional file and registry services in Vista with System.Transactions.  With the upcoming Vista RC update, there are a number of changes in the APIs for TxF and TxR that impact how to do this integration.

First, TxF and TxR have changed from APIs that automatically  picked up the ambient kernel transaction and joined it, to one where the developer needs to explicitly show an intention to participate in the transaction.  This is accomplished through the addition of several new APIs, such as CreateFileTransacted.  These new APIs are explicitly transaction-aware.  All existing file and registry APIs return to being explicitly transaction-unaware.

(For details on the new transaction-aware APIs, see  http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/fs/win32_functions_changed_by_transactions.asp.)

All of the new APIs take a kernel transaction handle as an explicit parameter.  This means that if I want to use these new routines from inside a System.Transactions TransactionScope I will need to do some work.  That is what I want to concentrate on here in the form of a partial example.

Let's start by setting up a couple of helper classes and declarations.

First, we know that we'll be manipulating a kernel transaction handle, so we'll specialize a SafeHandle appropriately:

using System.Collections.Generic;
using System.Runtime.ConstrainedExecution;
using System.Runtime.InteropServices;
using System.Runtime.Versioning;
using System.Security.Permissions;
using System.Text;

namespace TxFile
{
    [SecurityPermission(SecurityAction.LinkDemand,UnmanagedCode=true)]
    public sealed class SafeTransactionHandle: SafeHandleZeroOrMinusOneIsInvalid
    {

      private SafeTransactionHandle () : base (true)
      {
      }

      public SafeTransactionHandle (IntPtr preexistingHandle, bool ownsHandle)
       : base (ownsHandle)
      {
       SetHandle (preexistingHandle);
      }

      [DllImport("Kernel32.dll", SetLastError=true)]
      [ResourceExposure(ResourceScope.Machine)]
      [ReliabilityContract(Consistency.WillNotCorruptState, Cer.Success)]
      private static extern bool CloseHandle (IntPtr handle);

      [ResourceExposure(ResourceScope.Machine)]
      [ResourceConsumption(ResourceScope.Machine)]
      override protected bool ReleaseHandle ()
      {
          return CloseHandle (handle);
      }
    }
}

We also need to declare the new CreateFileTransacted API.  This looks a lot like CreateFile, but with a couple of extra parameters.  The new declaration is:

[DllImport("Kernel32.Dll",
           CallingConvention=CallingConvention.StdCall,
           CharSet = CharSet.Unicode)]
internal static extern SafeFileHandle CreateFileTransacted(
    String lpFileName,
    int dwDesiredAccess,
    int dwShareMode,
    IntPtr lpSecurityAttributes,
    int dwCreationDisposition,
    int dwFlagsAndAttributes,
    SafeFileHandle hTemplateFile,
    SafeTransactionHandle txHandle,
    IntPtr miniVersion,
    IntPtr extendedOpenInformation
   );

Our final declaration is to define the new COM interface that DTC supports, IKernelTransaction.  This interface is available off MSDTC's ITransaction objects.  It's declaration looks like:

[ComImport]
[Guid("79427A2B-F895-40e0-BE79-B57DC82ED231")]
[InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
public interface IKernelTransaction
{
    void GetHandle( out SafeTransactionHandle ktmHandle );
}

With these helpers and declarations out of the way, we can look at what to do in the mainline code to use CreateFileTransacted in a TransactionScope.  We know that we need the transaction handle, that we can get that from IKernelTransaction, and we know that we can get that from a DTC ITransaction.  That turns into:

SafeTransactionHandle txHandle;

IKernelTransaction kernelTx =
   (IKernelTransaction) TransactionInterop.GetDtcTransaction
    (Transaction.Current);
kernelTx.GetHandle (txHandle);

Putting it all together, we would get a TransactionScope pattern that looks like:

using (TransactionScope s = new TransactionScope ())
{
  SafeTransactionHandle txHandle;

  IKernelTransaction kernelTx =
   (IKernelTransaction) TransactionInterop.GetDtcTransaction
    (Transaction.Current);
  kernelTx.GetHandle( txHandle );
  ...

  // Do the Win32 file operation such as...
  SafeFileHandle fileHandle = CreateFileTransacted(..., txHandle, null );

  s.Complete ();
}