Deal Groupie – Learning WP7 and ASP.NET MVC 3

Lately, I have been taking some time to learn some new technologies.  For me, the best way to learn a new technology is to jump head first into a project:

Daily deal websites such as GroupOn, LivingSocial, DealFind, etc. have really taken off over the last 6 months.  With so many sites out there, my inbox was being flooded with emails from all these daily deal websites.  This seemed like it might be a fun project,  so I set out to build a daily deal aggregator.

First, I built a Windows Phone 7 app.  I built this app using Visual Studio 2010. Silverlight for WP7, the silverlight toolkit for WP7 and WCF.  Overall, I was pleased with the developer tools for Windows Phone 7 and I am looking forward to building more apps for WP7

ScreenShot5

The app conveniently presents deals for your city in a format very similar to the email client for Windows Phone 7.  Users can flag deals, share deals with friends, and buy deals from over 20 daily deal websites.  If you have a Windows Phone, you can download the Deal Groupie App here.

 

Next, I set out to learn some new web technologies.  I had recently attended a Microsoft Web Camp in Calgary (presented by Jonathan McCracken) and I was impressed with what I saw.  I decided to build a web version of Deal Groupie using ASP.NET MVC3.  I also made use of jQuery, jQuery UI, SQL Server CE 4 and WCF.  Since I had most of the backend already built, the web version did not take me nearly as long to build as the WP7 version did.  I spent more time tweaking the css than anything else.  The website provides daily deals for the selected city in 3 ways: a simple webpage listing all the active deals, a free daily email, and an RSS feed.  The RSS feed was exceptionally easy to implement using Syndication mechanism built in to WCF.

DealGroupieSite

Find deals in your city – http://www.mydealgroupie.com/

Currently, Deal Groupie is finding more than 2000 deals per day in over 300 cities in the USA, Canada, UK, and Australia.

Overall, building the Deal Groupie app and website has been a really good learning experience.  What’s next?  I’m thinking of building an Android version of the app and adding some filtering options to the website.

WF4 Asynchronous Workflows

If you are like me, you may have been surprised when you read the MSDN documentation for System.Activities.Statements.Parallel.

A Parallel activity operates by simultaneously scheduling each Activity in its Branches collection at the start. It completes when all of its Branches complete or when its CompletionCondition property evaluates to true. While all the Activity objects run asynchronously, they do not execute on separate threads, so each successive activity only executes when the previously scheduled activity completes or goes idle. If none of the child activities of this activity go idle, this activity execute in the same way that a **Sequence** activity does.

-MSDN (http://msdn.microsoft.com/en-us/library/system.activities.statements.parallel.aspx)

While the name implies that each branch of activities would be executed on a separate thread, the Parallel activity does not execute activities on separate threads.  In fact, all activities in a workflow execute on the same thread.  This is very similar to how in Windows Forms and WPF, all code that manipulates the user interface must be executed on the UI thread.  You might be asking yourself:  What is the point of this parallel activity?

It is possible for some activity to execute work on a background thread.  When combined with the Parallel activity (or the ParallelForEach activity), this allows the workflow runtime to schedule other activities to be executed while it is waiting for the background thread to complete.  WF4 comes with a number of asynchronous activities that can be used in this way.  Some examples are the Delay, InvokeMethod (when RunAsynchronously is set to true).  If you are creating custom activities, you can also create activities that execute asynchronously.  If you are implementing a code-based activity, you can inherit from AsyncCodeActivity.  If you are implementing more complex activities that need to make use of the workflow runtime, there are a couple strategies for implementing asynchronous activities.  One option is to use the parallel extensions introduced in .NET 4 (Workflow and Parallel Extensions).   Another option is to use bookmarks.

However, if you want to compose a workflow in XAML and have that workflow executed in parallel, your options are more limited.  Here’s an example:

In this example, I would like the 2 workflows to execute on a separate thread.  The output, however, shows that both activities are executed on the same thread:

Luckily, there is a way to accomplish this using a dynamic activity to execute a child activity on a background thread.  The child activity can be any activity (including Sequence or FlowChart), so this allows us to execute any portion of a workflow on a background thread. 


using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Activities;
using System.Threading.Tasks;
using System.ComponentModel;
using Microsoft.VisualBasic.Activities;

namespace ParallelActivities
{
[Designer(typeof(AsyncActivityWrapperDesigner))]
public class AsyncActivityWrapper : AsyncCodeActivity
{
public AsyncActivityWrapper()
{
Body = new ActivityAction();
}

    [Browsable(false)]
    public ActivityAction Body { get; set; }

    protected override IAsyncResult BeginExecute(AsyncCodeActivityContext context, AsyncCallback callback, object state)
    {        
         Activity activity = CreateDynamicActivity(context);
        IDictionary inputs = GetArgumentsAndVariables(context);
        Task task = Task.Factory.StartNew((ignore) =>
        {
            WorkflowInvoker.Invoke(activity, inputs);

        }, state);
        task.ContinueWith((t) => callback(t));
        return task;
    }

    protected override void EndExecute(AsyncCodeActivityContext context, IAsyncResult result)
    {            
    }

    private Activity CreateDynamicActivity(AsyncCodeActivityContext context)
    {
        DynamicActivity result = new DynamicActivity();
        //Create a DynamicActivityProperty for each argument / variable in the current context
        foreach (PropertyDescriptor property in context.DataContext.GetProperties())
        {
            DynamicActivityProperty dynamicActivityProperty = new DynamicActivityProperty();

            dynamicActivityProperty.Name = property.Name;
            dynamicActivityProperty.Type = typeof(InArgument<>).MakeGenericType(property.PropertyType);
            dynamicActivityProperty.Value = Activator.CreateInstance(dynamicActivityProperty.Type);
            result.Properties.Add(dynamicActivityProperty);
        }

        //Copy impors to dynamic activity;
        VisualBasic.SetSettings(result, VisualBasic.GetSettings(this));
        result.Implementation = () => Body.Handler;
        return result;
    }

    private IDictionary GetArgumentsAndVariables(AsyncCodeActivityContext context)
    {
        IDictionary result = new Dictionary();

        foreach (PropertyDescriptor property in context.DataContext.GetProperties())
        {
            result.Add(property.Name, property.GetValue(context.DataContext));
        }

        return result;
    }
}

}

Using the AsyncActivityWrapper inside a Parallel activity accomplishes the desired behavior by executing activities inside the AsyncActivityWrapper as shown here:

With the AsyncActivityWrapper, we can see that each branch of the Parallel activity is executed on a separate thread:

The designer for this activity is actually very simple.  It contains a single WorkfowItemPresenter which allows us to drag and drop an activity into the AsyncActivityWrapper.


<sap:ActivityDesigner x:Class="ParallelActivities.AsyncActivityWrapperDesigner"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:sap="clr-namespace:System.Activities.Presentation;assembly=System.Activities.Presentation"
    xmlns:sapv="clr-namespace:System.Activities.Presentation.View;assembly=System.Activities.Presentation">

    <sap:WorkflowItemPresenter Margin="7" Item="{Binding Path=ModelItem.Body.Handler, Mode=TwoWay}" HintText="Drop Activity"/>

</sap:ActivityDesigner>

In conclusion, the AsyncActivityWrapper can be used to execute portions of a workflow on a separate thread.  When combined with the Parallel or ParallelForEach activities, this allows us to execute multiple activities in parallel.  In situations where a workflow is processing a large amount of data, or doing a large number of complex calculations, the AsyncActivityWrapper can help to improve performance.  The implementation provided here does have some limitations.  As with any parallel programming, you need to make sure that activities that are executing in parallel do not conflict with each other.  If the activities are trying to access and manipulate the same data, you might run into problems.  If the activities have a return value, you may need an alternate implementation of AsyncActivityWrapper that inherits from AsyncCodeActivity<TResult>.  Also, since each activity is being executed as a separate independant workflow, you will lose some workflow functionality such as Persistence and Tracking.

Take the WF4 Speed Test

One of my previous posts talked about the performance improvements in WF4.  With the recent release of Visual Studio 2010 / .NET 4 RC1, I thought I would re-run my performance tests to see if there have been any further improvements with the most recent version.

My test is simply timing how long it takes to invoke an empty workflow (an Sequence activity that does nothing) 2,000,000 times.  This test gives us a good idea of the overhead of invoking a workflow.  With .NET 4.0 RC1, I am able to invoke ~275,000 workflows per second (approximately 15% faster than .NET 4 Beta 2).  I’m glad to see that Microsoft has continued to work on improving the performance of WF4. 

Take the WF4 speed test on your computer and let me know your results.  Create a new console application and past in the code below.  Don’t run the speed test in Visual Studio.  Compile the application and run the executable from outside Visual Studio.


using System;
using System.Linq;
using System.Activities;
using System.Activities.Statements;

namespace WFSpeedTest
{
class Program
{
static void Main(string[] args)
{
Activity activity = new Sequence();
WorkflowInvoker invoker = new WorkflowInvoker(activity);
DateTime startTime = DateTime.Now;
int numberOfInvokes = 2000000;
for (int i = 0; i < numberOfInvokes; i++)
{
invoker.Invoke();
}
DateTime endTime = DateTime.Now;
double workflowsPerSecond = numberOfInvokes / (endTime - startTime).TotalSeconds;
Console.WriteLine("Your WF Speed: {0} workflows per second", workflowsPerSecond);
Console.WriteLine("Press to exit…");
Console.ReadLine();
}
}
}

WF4 RC1, now 15% faster than WF4 Beta 2!

Adding Workflows in VS 2010

In Visual Studio 2010 Beta 1, you specifically selected if you were adding a Sequential Workflow or a Flowchart Workflow to your project.  In Visual Studio 2010 Beta 2, everything is an activity.  Instead of adding specific types of workflows to your project, you add an Activity.

In the Workflow Designer, you then drag either a Sequence activity or a Flowchart activity to the activity you just created.  This might seem like a small difference, but this really improves the composability of workflows in WF4.  Since workflows are no different than activities, we can compose a workflow of other workflows.  It also allows us to execute custom activities in unit tests without needing to create a workflow that wraps that activity (see previous post).

WF4 Performance

I have been playing around with the idea of modeling business rules as a set of small workflows (Flowcharts in WF4).  In the application, this would result in small workflows being called a large number of times.  I decided to investigate the performance overhead of invoking a workflow in WF4, the workflow technology in .NET 4.  Since I am using Visual Studio 2010 Ultimate Beta 2, this also seemed like a good opportunity to evaluate the performance profiler included in this edition of VS 2010.

For this test, I implemented a simple application that performs a calculation 10,000 times, then outputs that value to the console.  As a reference, I first implemented this in simple C# code as follows:

Code1

To determine the performance, I used the Instrumentation Profiler.  The simple C# code took approximately 800 milliseconds to execute.  Most of the time was spent writing to the console.

Profile1

 

Next, I implemented the calculation functionality in a simple flowchart workflow.  My flowchart has a single InArgument y, and an OutArgument x.

Workflow

In the Main method for my application, I modified the loop to call the simple workflow to perform the calculation.

Code2

The performance from this implementation seems a little disappointing at first.  We went from a total execution time of less than 1 second to approximately over 36 seconds.

Profile2

But if we really think about what’s happening here, this performance isn’t that bad.  We know that it takes about 1 second to perform the calculations and output the results.  That means that the remaining 35 seconds was spent on “workflow” related processing.  In that 35 seconds, we were able to create and execute 10,000 workflows.  Clearly this is not fast enough to accomplish what I was originally trying to do.

After digging through the profiler, I noticed that the application seemed to be spending a large percentage of the time initializing the workflow.  This led me to investigate if we could just create the workflow object one time and reuse it whenever we invoke that workflow.  It turns out that we can, and the results are dramatic.

 

The code is as follows:

Code4

Notice that I am now passing the input to the workflow using a Dictionary<string,object>.  I’m not sure why, but when I tried setting the input using workflow.y = i, the workflow would always use the first value that was assigned to y.  In this case, my output was always 0 because the workflow was always using y=0.  Passing the workflow inputs as a Dictionary seems to work around this problem.

Using this method of invoking the workflow, the total execution time was just over 1.15 seconds.

Profile4

Now this looks much better.  By caching the instance of the workflow object, we are able to significantly reduce the overhead of executing a workflow.  In this case, we spent just over 1/2 the time calling Console.WriteLine().  That means we were able to execute 10,000 workflows in the remaining .65 seconds.  Considering the flexibility this can provide in an application, I would consider this overhead to be minimal.  In most real-world applications, the difference in performance will not be noticeable to the end user.

When running the application in release mode with no profiler attached, it took 14.48 seconds to execute the workflow 1,000,000 times.  That works out to 69,039 workflows per second!

I found the performance profiler in VS 2010 to be useful.  While it is much better than no profiler at all, I was a little disappointed.  There are much better profilers on the market today.

Improved Testability in WF4

One of the big complaints with the version of Windows Workflow Foundation that shipped in .NET 3 and .NET 3.5 (WF3) was that it was very difficult to unit test workflows.  Unit testing workflows was not impossible, but it was very difficult and required a lot of code to setup the workflow runtime and manage your workflow.  Since unit testing workflows in WF3 was clumsy and awkward, many developers chose to either not unit test their workflows, or just avoid using WF3 all together.
I am very happy to see that Microsoft has improved on this weakness with Workflow Foundation 4.0 (WF4). 

The first change that improves testability is the concept of In/Out arguments for workflows / activities.  The use of In/Out arguments and variables greatly simplifies the flow of data in WF4.  It is a big improvement over the use of Dependency Properties in WF3.  In WF4, all workflows and activities can accept a number of In arguments and return a number of Out arguments.  There is also support for In/Out arguments.  A more complete description of In and Out arguments and variables in WF4 can be found on MSDN (http://msdn.microsoft.com/en-us/library/dd489456(VS.100).aspx.aspx)).

In this example, I have created a simple workflow called Battle that accepts one SuperHero (the Hero InArgument), and one Villain (the Villain InArgument).  SuperHero and Villain are simple class that I have defined.  Both SuperHeros and Villains have a single SuperPower.  My workflow simulates a battle between the Hero and the Villain, then returns the name of the winner (the Winner OutArgument).  I have added the In and Out arguments using the workflow designer in Visual Studio 2010.

In the workflow xaml definition, the arguments are described simply as:

  <x:Members>
    <x:Property Name="Hero" Type="p:InArgument(w:SuperHero)" />
    <x:Property Name="Villain" Type="p:InArgument(w:Villain)" />
    <x:Property Name="Winner" Type="p:OutArgument(x:String)" />
  </x:Members>

The second big improvement is in the workflow runtime itself.  In WF3, there was a central WorkflowRuntime that created and managed workflow instances for us. The central runtime provided a set of events (WorkflowCompleted, WorkflowTerminated, etc.) that would notify us when a workflow entered specific states.  The problem with the central runtime was that the events would notify us when any workflow was completed (we would often receive notification about workflows that we are interested in).  This approach seemed a little clumsy, and thankfully it has been greatly improved in WF4.  In WF4, the central WorkflowRuntime no longer exists.  Instead, we can directly create instances of our workflow (using the constructor for that specific workflow), set its input arguments, then use a WorkflowInstance object to run that individual workflow.  The workflow instance gives us options to run the workflow synchronously or asynchronously (this was very difficult in WF3), and provides a set of events (WorkflowCompleted, WorkflowTerminated, etc.) to notify us when that specific workflow enters specific states.  Since we are dealing with a single workflow instance, we no longer need to worry about being notified about other workflow instances.  I think this is a much more natural approach than WF3, and really improves the testability of our workflows.  Since we no longer need to worry about initializing the WorkflowRuntime and dealing with its complexities, our unit tests suddenly become a whole lot simpler.

For more information on the changes surrounding the WorkflowRuntime, check out
http://msmvps.com/blogs/theproblemsolver/archive/2009/06/23/the-new-windows-workflow-foundation-4-runtime.aspx.

To make things even easier, WF4 includes a useful static Invoke method on the WorkflowInvoker class.  This allows us to run a workflow synchronously using a single line of code.  Our workflow unit tests become very logic and natural now.  We create a workflow and add any number of InArugments and OutArguments to it.  In our unit test, we create an instance of our workflow by calling its constructor.  We can then set the InArugments by setting the properties that were generated for that workflow class.  Finally, we can call WorkflowInvoker.Invoke to run the workflow.  The Invoke method returns an IDictionary<string, object> containing all the OutArguments from the workflow. We can then test the outputs using Assert as we would in any other unit test.

The only clumsy piece here is the way WF4 has chosen to return the OutArguments. It is using a key/value pair dictionary which can be a error prone.  A more natural approach would be if I could just access the OutArgument properties on the instance of the workflow that I passed in to the WorkflowInvoker.Invoke method.  Who knows, maybe this will change for the final release.

Overall, the changes introduced in WF4 can really improve the testability of our workflows.  If you previously decided not to use WF3 because it was too hard to unit test your workflows, you should take a good look at WF4.

NOTE:  This information and code is based on Visual Studio 2010 Beta 1 and is subject to change.