Scott Allan introduces himself

I'm a consultant in Baltimore Maryland, I run a website called odetocode.com and I've been a Microsoft MVP. This is my second year as MVP. I think I got that award because I've participated in code camps, conferences, user groups, I've written for MSDN Magazine and Dr. Dobbs. I've written a couple of books for Rocks and some other publishers.

Windows Workflow Foundation. What is it?

It is part of WinFX which is the next generation of Managed APIs that Microsoft is going to release. That includes Windows Presentation Foundation, Windows Communication Foundation (which used to be called Avalon and Indigo, which everyone liked better than WPF and WCF). WWF is sort of a sleeper technology. I don't think it has got as much press as the other two technologies, but I really think it's the most significant piece of middleware that Microsoft has released since the Distributed Transaction Coordinator. Almost every program encodes some type of workflow; there's some type of process involved that has rules and conditions. I think its going to be very useful in a lot of different types of architectures.

Where do you see WWF being used, specifically to solve what sort of problems that we are facing today?

Typically workflows have some distinct problems. One is they're generally long running. So if I'm tracking a purchase order from beginning to end it takes some time from beginning to end to purchase order, to get approved. By the time it eventually closes out it may be 20 or 30 days later. During that time I have to keep state about that purchase order, I have to know what state it is in, what has happened to it previously, who it goes to next.

After you create a workflow and define the process that it goes through it can just persist itself to a sequel server database, re-hydrate itself later and pick up where it left off. If the person who does the PO approval is in vacation for a week, it can just sit in the database and wait for that person to come back.

Another problem that it tries to address is that workflows typically involve some human judgement. We've all been in these requirements meetings where you say "Please tell me the steps the software has to take. Step a, b, c, d etc."

The person would say "Sometimes we do c, but sometimes not". You think that it's not going to be a random number generator, so we're going to have to receive some sort of human input. Windows Workflow has essentially two ways to model a workflow and arrange activities. One way is the sequential execution - you would arrange the steps in your workflow and it just marches through them to the inevitable conclusion, sometime later. You can still loop and branch and things like that but there's a definite start and a definite end.

The other model is the event drive model which essentially defines what state can my workflow might be in, what event it listens for in those states and when it gets one of those events, what's the next step? Those are perfect for scenarios where you need human judgement to be involved in to say "this purchase order is going to be rejected so leave it to the rejected state". It could be one of the scenarios where you could just not codify the conditions under that was going to happen, you couldn't put all the rules together, someone just has to look at it. That's another challenge that Windows Workflow is trying to solve.

It's not something that has a UI, is it basically a rules engine?

It's a runtime. Hopefully it's a lot like ASP.NET; it's juts a collection of assemblies, DLLs; you need a host process to load that. It will run under a smart client, it will run under ASP.NET but itself it does not present a user interface. It's definitely middleware behind the scenes; it's about services and interfaces more than about displaying widgets and things like that.

In a typical application, let's say a WinForms app, how would it go about integrating with the workflow engine and displaying queued items that are ready for the user?

Typically all you have to do to get a workflow running is to call the work flow runtime constructor and start it up. That will get the runtime in place. You can add additional services into that runtime so it's a bit like going through a drive-through and ordering a hamburger. You can get the basic hamburger; you can get the hamburger with transactions, or that with transactions and sequel persistence. You tell the runtime what services you want and you can then just start creating workflows and having the runtime execute those workflows. There is a tracking service available that can give you events about what is happening inside of a workflow, and you can record all that to a SQLServer database too. You can go back in time and say "This workflow that ran on January 5 when was it created, when did it reach the second state, when did it finally close out?". You can go back and look at that kind of information

Is it almost like an audit trail that can help with things like Sarbanes-Oxely?

You can have an audit trail; you can inject your own audit events that you want to track in there. One thing you can do with a state machine activity that's where you have state and each state has specific events that it listens for and moves to the next state. You can think of a purchase order: when a Windows form client comes up and you type in a purchase order and hit create. Other windows programs and that program itself can go to workflow and say "for purchase order 123, what state is it in?". The engine can say "It's in the open state. It hasn't been processed, it hasn't been shipped". Then you can adjust your UI accordingly. Typically a purchase order that is in the open state has to move to the process state before it moves to the shipped state. If you're driving a UI with your workflow you can go and query that workflow instance and say "what state are you in and what events are you allowed to receive in that state?" That way you can disable the ship-it button because it hasn't been processed yet".

Once your workflows are defined and you have a purchase order. How's that stored and retrieved?

The workflow itself can serialize into a database, your purchase order may or may not be a part of your workflow; you can store that into another table and just keep an ID and correlate the two that way.

Every workflow instance has a GUID we've defined. You can do it the other way. You can say "This purchase order is working with this GUID workflow instance".

Is Microsft "dog-fooding" it, are they going to be included in any of the products that are coming out?

The upcoming SharePoint release is going to have Windows Workflow imbedded inside of it. A lot of people want workflows around their documents that are being put into SharePoint. Instead of just being able to upload a document and have it on the company intranet, you can actually have it go from the approval process from the marketing department, legal department etc. The next version of SharePoint is out before the end of the year.

David: I'm looking forward to it. We try to implement some higher workflow that's going to flow from HR through finance, IT etc, using infopath and share-point. You have to write so much of your pluming code to get all that happen in the right order. Info-path and share-point are great tools to do what they do but they were missing that workflow piece. It's exciting to hear that the workflow stuff is going to be integrated with share-point.

It's going to be there and it's going to be in the next version of BizTalk server. The new thing about SharePoint is that Windows Workflow ships with what they call the base activity library, which is just a set of primitive activities that you can put into your workflow; they're activities like an IF-ELSE branch, a WHILE loop, invoke a web service, be a web service endpoint etc.

What Microsoft expects people to do is to create custom activities; so just like we have user controls and ASP.NET specific to our application, and we have custom controls and windows forms that are specific to what we're doing, we'll be able to create custom activities in Windows Workflow that will be specific to what we're doing; from my understanding SharePoint is going to ship with some custom activities that are specific to it.

Certainly one of the advantages you'll be able to do with Windows Workflow in writing business applications is create a domain specific language. You can have a business person sit down beside you and help you drag and drop these activities, arrange them, define the rules inside. They'll be able to look at the screen and actually decide what's been going on because they won't be seeing things like invoke method and web service, which we don't want to show them. We want to see things like notify purchasing, ship order, check inventory, etc.

Could you elaborate more on exactly a Domain Specific Language is?

If you look at C# and Visual Basic those would be general purpose programming languages. You have to have some training, some experience, you have to know how to construct loops and for each loop to be able to get anything done. The plus side is that you can write any type of application with C# or Visual Basic. You could write a healthcare application, a restaurant inventory application. Domain Specific Languages are tailored to their environment. In theory they should be more productive for people that are constructing software for a specific category of applications. If I write a DSL for a restaurant I could expose language primitives like check inventory, order cheese, etc; not only would it be more productive for a person that's trying to put the pieces together, but you can also get subject matter experts and business people involved a little more in the process.

The other distinction in my mind is that with general purpose programming languages we're typically doing imperative coding and we're telling the computer the exact steps to take to get a process done. We're writing loops and declaring variables and things like that. Typically with DSLs you can move to a slightly higher level of abstraction and just tell the computer "This is what I want to get done, you figure out the steps to make that happen".

Windows Workflow is a lot like that because you can drag and drop these activities and you don't know the specific steps that it has to take to transform those activities into IL.

David: You can recreate the custom activities that must be familiar even to en end user and give them a drag and drop interface to create and modify their own processes.

It's neat because the designer that ships with Windows Workflow and plugs into Visual Studio is part of the redistributable. You can host that same workflow design in your own Windows form application.

How would you apply Workflow in an ASP.NET environment?

In ASP.NET is similar but a little different because the default scheduling service in Windows Workflow is to run a workflow asynchronously; so it picks up a thread from the thread pool to execute that workflow. That's great for Windows form applications because typically you want to keep the UI responsive. So if the Workflow is going to take some time to calculate you just let it do that in the background. In ASP.NET is a different story because ASP.NET is already using the thread pool to process web requests. If we're spinnng up a lot of workflows asynchronously and they are also using the thread pool, that's one last thread that we have to service web requests. There's not a big need for doing stuff on the background thread of a web form; ultimately the user is still going to wait for the result of that page and it's just a question of "do you want to tie up two threads to complete that or just one thread." That's one subtle change.

Persistence is a lot more important in ASP.NET because in windows forms workflows that might run for a few days are going to need persistence too but in ASP.Net you typically do the quicker turn around so you're going to have shorter operations, keep more state in the database.

David: It's far more likely that the network might go away so you don't want long running transactions. You're constantly moving stuff out into the state, constantly saving state.

You can keep things around in memory a little longer, that's not a problem but it's nice to get them in and get them out and get it over with.

What do you see as the architectural patterns that are emerging with WWF and how to best apply it?

I think Windows Workflow is going to be the execution engine. There's certainly the ability to write a lot of code inside of a Workflow, to actually put implementation code inside of a Workflow. I don't think that is going to be the most flexible approach, it's not going to be the best architecture. Workflow has the ability to communicate back to the host process and receive events from the host process during the lifetime of a Workflow through well defined interfaces. You literally define an interface that has public methods and the public events and you tell the Workflow "this is what you're going to be working with".

Let's say I'm building a Workflow to download the log files from a web server. I can certainly create activities that do the ftp transfer, the parsing of the log files that do the insertion into the database. But ultimately I think you want to keep those jobs into separate components.

The Workflow is really just the shell that says "here's the process we follow, here are the rules". It calls back to the host process which does a lot of the work. Another nice think about Workflow is that when you define conditions and rules on certain activities those can be stored in an external XML file which can be edited; after your workflow has been compiled you still have those rules files that you can work with to maybe tweak what the discount percentage is going to be, how much does an order have to be before you have to reject etc.

David: In what format is the external file?

It is an XML file.

So rules can be stored in an XML file externally. But where is the definition of the workflow stored and how is it stored?

Ultimately the definition of a workflow is going to be in an assembly or a XAML file. In Windows Workflow the extension is .xoml but inside the XAML the same extensible mark-up language that WPF is going to use.

When I'm offering a workflow I have a couple of choices. I can write a workflow using straight code, I can write it using XAML or XML mark-up, and I can also use the combination of the two which is really comfortable both for ASP.NET developers and Windows forms developers because we're used to having a visual designer on the screen, dropping some buttons there, and the designer takes care of all that in one file.

But then in another file we have a code file where we can write event handlers and the little bit of logic that has to be behind that. In windows Workflow you can do all that, it works very similar to the designer of both of those technologies and you can have a code behind or beside the class that has the logic; the XAML files and the code files do have to go through a compilation phase.

If you're writing enough XAML, something has to take the XAML and generate some code; all the code has to go through the C# compiler, BB compiler, so ultimately those workflow definitions are in assemblies. The one exception is when you write workflows in pure XAML just XML, no code behind, you can actually feed those to the workflow engine and have it go through a process called activation. So you don't have to compile those workflows, it can just read the XAML definition and conjure up a new instance of that workflow.

I think that will be very useful in scenarios where you have to create a lot of customized workflows. You might have 100 clients every single one of them has customized their workflow somehow and you're just storing the definitions in a database table. You never have to compile those, they're just pure XAML and you can just feed those into the runtime and have them executed.

You gave the example of ftp log files. You have a component that starts to download the file and then it tells the workflow engine "I'm done dowloading, what do I do next?" Is that it?

That's the grey area. I can certainly write a custom activity that is an ftp activity. It would expose properties like server, user name, path, destination directory etc. I could drop that into the workflow designer and it could have all the code to do an actual ftp job. If you have some business logic, let's take the next part of that weblog example where I have to do some transformations; maybe there are some records that I want to filter out of the web log before whatever I'm doing with the generating reports from it or so forth.

That is a perfect place for Windows Workflow because you can define rules that end up in external files and you can easily modify those rules and have it passed to those records. But I think you certainly don't want to build the entire application using just Windows Workflow and just kicking off the workflow and just executing everything in your program; there are lots of cases where you're going to want Windows Workflow just to be the driver, the logic.

Can you use Windows Workflow to drive page navigation?

One of the topics that I'm really interested in is using Windows Workflow to drive page navigation. Instead of embedding links and using hyperlinks etc in my web page I always from page 1 to default .ASPx.

ctually the workflow makes the decision about where the user is going to end up next. ASP.NET isn't greatly suited to that. You typically need to put sort of a layer of indirection between the browser with the url the browser is requesting and the page that you ultimately execute, so usually there's some module involved that's going to do some URL rewriting or something. So that you can tell the workflow what the user did and the workflow can come back and say "ok we've completed that phase, go the next step."

IT doesn't have to be hard coded in your application than the workflow, you can have an outside of an ASP. NET.

If you ever want to change your sign up process you don't have to go into your ASPX files and change links around. You can go into graphical workflow designer and say "it's going to work this way now". That's one of the scenarios if you have a domain specific language that could be a business person that's actually making this tweaks and not a developer.

You can use a front line employee that's less costly than an advanced developer to modify your workflows.

They don't have to know anything about ASP.NET they just have to drag and drop some controls and re-arrange them on the designer.