login      my ps      about      contact  



home instructor-led training on-demand screencasts blogs


Jim Johnson's Blog » The Resource Manager Project -- Introduction

The Resource Manager Project -- Introduction

I've been thinking for a while that I should do some posts on building a resource manager -- not just what the interfaces are, but also what the implications and obligations are and what is entailed in meeting them.  The longer I've thought about it, the more that I've realized that it really can't be one post -- instead it needs to be a set, starting with some of the behind the scenes principles.

 

So, this isn't going to be the 'how to write a resource manager' post -- it's going to be the 'what topics do I need to think about?' post.  This one will be followed by sections on each of those topics -- at least that's my plan.

 

At one level a resource manager is simple: it encapsulates some resource and provides a 'transactionally perfect' view of it.  Doing this means that it has to isolate changes appropriately; it has to vote in the commitment process; and it has to recover from any intervening failures.  Once the design has covered these points, then there's room to discuss optimizations available through the System.Transactions or MSDTC interfaces, and how you might decide between a full resource manager or a compensator (e.g. a COM+ CRM).

 

This drove my set of topics.  My first ones are fairly 'meta', but I believe that they're important to get the design and implementation right.  They are:

 

  • What is isolation, what does it need to do, and how can I implement it?
  • How should I think about and plan for faults and failures?  Which ones do I need to handle?  What should my strategy be for recovering from them?
  • What should I do about durability?  How should I think about logging?  What are the naming restrictions around durable resources?
  • What are my responsibilities during the two phase commitment process?
  • What if I want to handle unrecoverable resources (also known as real world actions)?
  • How do I emulate a COM+ CRM?

 

Finally, then, my plan is to bring it all together by building up a resource manager that uses System.Transactions and System.IO.Log (available in WCF).  This will be a fairly long series of articles, and I will undoubtedly be covering other topics in amongst them.

Posted Jan 21 2006, 11:30 AM by jim-johnson
Filed under:

Comments

John Apps wrote re: The Resource Manager Project -- Introduction
on 01-21-2006 12:46 PM
Jim,
sounds like a pretty ambitious plan - I'm looking forward to reading the instalments!

I have one question: the description you provide would seem to imply a resource manager for a resource which has a short transaction time, e.g., databse, queue entry.
Have you considered resources with longer transaction times such as workflow entities or travel reservation systems?

Cheers, john
Jim Johnson wrote re: The Resource Manager Project -- Introduction
on 01-22-2006 6:06 PM
John,

Yeah, it's a pretty big undertaking. I'm hoping that I can see it through in a reasonably timely manner.

Right now I am focusing on a simple recoverable resource manager for an atomic transaction. Assuming I can cover this in a reasonably short timeframe, extending it to include compensations might just be interesting. For now, though, I don't want to make this any more complex than it is going to be anyway.

Jim.
Vijay Srinivasan wrote re: The Resource Manager Project -- Introduction
on 01-24-2006 4:29 AM
Jim,
This is something that will be a great reading. I was fortunate to enough to implement a reusable resource manager and the dispenser in the COM world, using & implementing the OLE Transaction interfaces. we had to implemnet this to plug this into our doc management middleware The two phase commit worked like a charm...
yup extending that to include compensating txn sematics will make it more interesting
Vijay Srinivasan wrote re: The Resource Manager Project -- Introduction
on 01-24-2006 4:34 AM
hey John,
I am not sure OLE Transactions as encapsulated in System.Transactions can be used to model long running transactions
Correct me if I am wrong though.

Vijay Srinivasan wrote re: The Resource Manager Project -- Introduction
on 01-24-2006 4:51 AM
Jim,
to make your life more difficult -
can you please enlighten us on these areas as well -

Nested Txns - How are we going to manage them? Are we to treat them as two seperate physical txns in two phase commit yet to provide a relationship semantics that enables to treat the parent and child txns atomic OR treat them as one physical txn as a whole

Idem Potency - is there a way to see whether we can advertise to RM of concurrent txns that are idem potent, allowing the RM to optimize on locks
Jim Johnson wrote re: The Resource Manager Project -- Introduction
on 01-24-2006 8:39 PM
Vijay,

YOur resource manager experience sounds interesting. I'd certainly be interested in what you learned, and what you think could make the process easier.

For this sequence, I'm going to have to try to keep the resource simple, and stick to the mainline designs. Since System.Transactions, or oletx, don't support nested transactions, I wasn't planning on discussing that.

Idempotency is possibly more relevant. For right now, let's see how the set of articles ends up going. It may make sense as one of the perf optimizations that a resource manager author can consider.

Does that make sense?
Jim.
Vijay Srinivasan wrote re: The Resource Manager Project -- Introduction
on 01-25-2006 2:18 AM
Jim,
Just got to see your msg, I am sitting across the globe here in India.
I am indeed looking forward to have some good learnings from your articles.And would not mind sharing some of my RM code snippets with you if you want to have a look

all of my expriences in building the RM engine was way back in 1999 using COM - ATL/C++ , WIN32 and implementing & using OLETXN Interfaces and MSDTC APIs and I am not sure some of the points that I relate here is relevant anymore. That said here are some of my experience worth sharing -

we realized that OLETXN object model did not specify how the RD needs to bind with the RM to propagate the connection state (whatever connection that it maintains) during txn enlistment. we had to resort to custom IPC between RD and RM - did that using DCOM itself. would have been great if there was a default channel (like what we have in Remoting)that will serve as an abstraction for us to register our RM and RD. The channel can be an extensibility point allowing RDs to connect to RMs thru arbitary transports.

Definitely one needs to have an automated mechanism to process TXN in conflict (those that lost out in commit phase ). I guess MSDTC does not do much on those txns which failed on commit phase but were sucessful during prepare

Ours was an generic RM engine (implementing OLETXN Interfaces - ITransactionResourceAsync, IResourceManagerSink...). we built this as a generic RM engine that abstracted the common behaviour (including the custom connection between RD & RM exposed thru DCOM)while providing extensbility points (thru the APIs the engine published) to allow arbitary RMs to be implemented and get attached to the engine.

just some thoughts memories of my RM ....
Jim Johnson wrote re: The Resource Manager Project -- Introduction
on 01-25-2006 7:24 AM
Vijay,

Thanks for the summary - it sounds like it was an interesting project.

Fwiw, you do hit on some points that I hope to cover in the sequence. I'll be interested in feedback and thoughts as it proceeds.

Jim.
Vijay Srinivasan wrote re: The Resource Manager Project -- Introduction
on 01-25-2006 8:52 AM
Sure Jim, I will read your articles with utmost interest and looking forward to a good learning excercise for me

Is there a way for me to get in touch with you over a mail?

Vijay
shreeman wrote re: The Resource Manager Project -- Introduction
on 02-02-2006 7:31 AM
JIm,

I had written to you previously on some what similar to the
How do I emulate a COM+ CRM?

What I was trying to achieve was to utilise the enterprisesvc usefulness together with system.transaction .That is while i was trying to create a resourcemanager which ll utilize the best of both world .

My stem on this started while replying a group post but i didn't able to worked on that successfully.Now that you had openedup the topic again i need to know what are your thoughts on the Xa transaction support and will it be fully replacement of msdtc as well as any suggestion on Async operation on transaction to span multiple thread and since the same can be delegated can it be delegated to a different app domain ??
Sorry for spamming for differnet questions at one go.Here is the old resource manager post pls suggest what could be a solution??

http://groups.msn.com/-netindiagroup/technotes.msnw?action=get_message&mview=0&ID_Message=3153&LastModified=4675552648936951482&all_topics=1
MADHU BLOG(MSDTC/COM+/WEB SERVICES/WCF) wrote How to implement Resource Manager in SYSTX?
on 04-05-2006 10:16 AM
Following links will help you to write resource manager by using SYSTX classes
 
http://msdn2.microsoft.com/en-us/library/ms229975(VS.80).aspx...

Add a Comment

(required)  
(optional)
(required)  
Remember Me?


home instructor-led training online training blogs

781.749.9238 (East Coast) | 310.251.9901 (West Coast) | Pluralsight, LLC , 185 Lincoln St., Suite 305, Hingham, MA 02043-1741

Copyright © 2008 Pluralsight LLC. All rights reserved.

designed by nukeation