The autistic coder - Andreas Öhlund's blog

world.Publish(new BlogPost());

This post explains how to manage the NHibernate session when using NServiceBus. In NServiceBus each message becomes a unit of work so the recommendation is to use a new session for each message that gets processed. NServiceBus wraps the handling of each message in a TransactionScope. This scope becomes the unit of work and guarantees consistency between the data store and the underlying queuing technology. Note that this requires the endpoint to be configured as "transactional", the default in NServiceBus.

Configuring the session factory
First we have to configure the NHibernate session factory so that we can create new sessions. Creating a session factory is an expensive operation so we need configure it as a singleton.

ISessionFactory sessionFactory = ConfigureSessionFactory();            
Configure.Instance.Configurer.RegisterSingleton(typeof(ISessionFactory), sessionFactory);

Creating and managing a session per request can be done in a number of ways so lets start with the simplest one.

Explicitly create the session in each handler
This brute force way is to open and use the session in each of your message handlers. An example of this is shown below

public class SaveToDatabaseUsingRawSessionHandler : IHandleMessages<SaveToDatabaseUsingRawSession>
{
private readonly ISessionFactory sessionFactory;

public SaveToDatabaseUsingRawSessionHandler(ISessionFactory sessionFactory)
{
this.sessionFactory = sessionFactory;
}

public void Handle(SaveToDatabaseUsingRawSession message)
{
// the session will automatically enlist in the TransactionScope
// that NServiceBus wraps calls to messagehandlers in (given that the endpoint
// is configured as "transactional")
using (var session = sessionFactory.OpenSession())
{
session.Save(new PersistentEntity { Data = message.DataToPersist });
}
}
}

The fact that NServiceBus wraps message handling in a TransactionScope means that NHibernate will automatically enlist and commit or rollback accordingly. The drawback with this solution is that you need to duplicate the call to create a session in all of your message handlers. If you use some kind of repository pattern you would have to explicitly pass the session to all those repositories in order to have them use the same session. Let's see how we can improve this a bit.

Storing the session in the NHibernate session context

To avoid duplicate code and having to pass the ISession around we can use the context session feature in NHibernate to store our session for the duration of the request. To do this we need a way to create a new session and store it in the session context when message processing begins. This is done using the message module feature in NServiceBus. Message modules are created by implementing the IMessageModule interface. NServiceBus will then automatically detect those classes and invoke them at the various stages of the message processing pipeline.

With this solution the code in our message handler is reduced to this:

public class SaveToDatabaseUsingMessageModuleHandler : IHandleMessages<SaveToDatabaseUsingMessageModule>
{
private readonly ISessionFactory sessionFactory;
public SaveToDatabaseUsingMessageModuleHandler(ISessionFactory sessionFactory)
{
this.sessionFactory = sessionFactory;
}

public void Handle(SaveToDatabaseUsingMessageModule message)
{
sessionFactory.GetCurrentSession().Save(new PersistentEntity { Data = message.DataToPersist });
}
}


And the message module is implemented like this:

 public class NHibernateMessageModule : IMessageModule
{
private readonly ISessionFactory sessionFactory;

public NHibernateMessageModule(ISessionFactory sessionFactory)
{
this.sessionFactory = sessionFactory;
}

public void HandleBeginMessage()
{
CurrentSessionContext.Bind(sessionFactory.OpenSession());
}

public void HandleEndMessage()
{
//session is closed when the transactionscope is disposed so we
//don't have to do anything here
}

public void HandleError()
{
}
}

Note that in order for this to work the NHibernate session context mode needs to be thread_static. A sample of how to configure the session factory can be found here.

Managing the session using your favorite IoC container

This solution is equivalent to the one above but relies on the thread static cache mode of your container of choice. This example uses StructureMap if you prefer one of the other supported containers please use the syntax of that container. Also note that this solution doesn't require a message module to open the session.

In order to get this to work we just configure the container to create a new ISession for each thread and cache it thread static. This means that we can change our repositories and message handlers to depend on ISession directly and let the container do it's magic. The container will then make sure that all code executed on the same thread will use the same ISession.

The container configuration looks like this:

ObjectFactory.Configure(x=>
{
x.For<ISession>().CacheBy(InstanceScope .ThreadLocal)
.Use(ctx => ctx.GetInstance<ISessionFactory>().OpenSession());

});

A sample message handler is seen below

public   class   SaveToDatabaseUsingContainerCacheModeHandler :  IHandleMessages< SaveToDatabaseUsingContainerCacheMode>
{
private readonly ISession session;

public SaveToDatabaseUsingContainerCacheModeHandler(ISession session)
{
this.session = session;
}

public void Handle( SaveToDatabaseUsingContainerCacheMode message)
{
session.Save(new PersistentEntity { Data = message.DataToPersist });
}
}

This solution basically replaces the caching provided by NHibernate with the one provided by the container, with the added benefit of not needing the message module.

Summary
NHibernate sessions in NServiceBus should be used on a "per message basis". Hopefully this post will give you ideas on how you can best solve this in your application. Running endpoints non-transactional is not recommended and should only be used under special circumstances. If you do need non transactional endpoints remember that you have to manage the NHibernate transaction your self!

A full sample can be found here.

As always comments/suggestions are most welcome!

App configs tend to get bloated with all kinds of configuration settings that are not likely to change in production. Those huge files are a pain to read and it's easy to get something wrong causing the whole app to fail. Another thing to take into account is that everything in the config file need to be well documented so that the sysadmins know what to change and when.

So what am I getting at?

Think twice about what you put in you app.config. A lot of things can and should be configured in code instead. You as a developer are in charge of the source code and all decisions that are yours or things that shouldn't be modified in production should go there. This results in configurations that are:
• More readable to both developers and sysadmins
• Easier to modify when the need arise
• Easier to document for developers
• Only contains setting that are eligible for modification in production

What should go in the app.config then?

I found it useful to ask myself the following question when deciding what to make configurable:

Is this something that our sysadmin should be allowed to change when the system is running in production?

If you can answer the above question with a yes then it's most likely a good candidate for your app.config. If not, put in in the code!

Logging - an example

Let's using logging as an example on how to apply this rule. Using log4net logging is usually configured by the following syntax in the app.config

<log4net>
<appender name="RollingLogFileAppender" type="log4net.Appender.RollingFileAppender">
<file value="whatever.log">
</file><appendtofile value="true">
</appendtofile><rollingstyle value="Composite">
</rollingstyle><datepattern value="yyyyMMdd">
</datepattern><maxsizerollbackups value="100">
</maxsizerollbackups><maximumfilesize value="10MB">
<layout type="log4net.Layout.PatternLayout">
<conversionpattern value="%date [%thread] %-5level %logger [%property{NDC}] - %message%newline">
</conversionpattern></layout>
</maximumfilesize></appender>

<root>
<level value="DEBUG">
</level><appender-ref ref="RollingLogFileAppender">
</appender-ref></root>
</log4net>

As you can see we allow configuration of:

  1. What appender to use, a rolling file appender in this case
  2. Where the log output should be stored
  3. That the appender should create a new file for each 10 MB of logs
  4. That the appender should keep the most recent 100 files
  5. The logging threshold
  6. The layout if each row in the log output

Applying the control question to the above statements reveals that the only thing that make sense to change on the fly in production is 2 and 5.

You might argue that it's good to allow for adding new appenders without redeploying code. But come on, how often do you get in a situation where you need to change that on short notice in a production environment? Your CM procedures should allow you to patch and redeploy quickly anyway.

What about settings that should go into the code that might vary between the different environments during development, test and production? This should be handled by making your application lifecycle aware.

By configuring logging in code you can make sure that only the relevant settings are exposed to your sysadmin. By doing this we end up with the following config. Much cleaner don't you think?


<appsettings>
<add key="Logging.Threshold" value="INFO"/>
<add key="Logging.OutputDirectory" value="c:/logs"/>
</appsettings>

The icing on the cake
Let's take this approach on step further by applying some convention over configuration in order to totally remove the need for logging configuration for normal production operations. This is easily done by defaulting the threshold to WARN and the output directory to either the current working directory or ./logs. If the admin doesn't like this he can always override as he likes. This gives us the following of benefits

  • Forgetting to configure logging will result in the default settings beeing used.
  • No need to have your deployment scripts modify the settings when you move between environments
  • Less risk for misconfiguration
  • Cleaner and more readable config files

So next time you add something to you config file consider configuring it in code instead!

Today I committed support for running NServiceBus on the Azure cloud to the trunk. Also included is a sample application that demonstrates  how to build NServiceBus solutions for Azure.

Running the sample

  1. To run the sample you have to install the Azure SDK and the Azure tools for Visual Studio. The easiest way to do this is through the  Web Platform Installer. The azure tools are located under “Developer tools” ( If you can’t see them you have to enable them in “options” for the installer)
  2. Once you have tools installed you can get the latest NServiceBus trunk from subversion or download the latest build from our CI. The sample is included in the samples folder
  3. Start up the “AzureService” solution in Visual Studio and hit F5 (NServiceBus defaults to Development Storage if no configuration section is present, more on this later) so you should see the Development. Storage and Development Fabric services start up.

That should be it, the sample should now run in the development fabric on your machine.

Configuration

The Azure queues support is configured using the fluent API

var config = Configure.With()
.SpringBuilder()
.XmlSerializer()
.UnicastBus()
.LoadMessageHandlers()
.AzureQueuesTransport()
.IsTransactional(true)
.PurgeOnStartup(false);


If no configuration section is found NServiceBus will default to your local development storage. To use your personal Azure storage with the sample add the following to the configuration files (web.config, app.config):



<AzureQueueConfig 
AccountName="{Your account name}"
Base64Key="{Base64 key for your account}" />


This configures NServiceBus to create and use queues on your Azure storage account. If you hit F5 again you should see the application start in your local fabric but this time the data is stored on Azure. Cloud Storage Studio is a great tool for managing you storage accounts both locally and on the cloud. The final step is to deploy the entire solution and run it completely on Azure. To do this “publish” the project in Visual Studio and upload the AzureService.cspkg and the ServiceConfiguration.cscfg to your Azure account. Detailed instructions on how to deploy solutions to Azure can be found here.



The sample application is currently running on my own Azure account, please try it out!http://nservicebus-demo.cloudapp.net (And yes my web design skills sucks :)



The sample is doing some simple send/receive and publish/subscribe operations between a WebRole and a WorkerRole. Please take look at the source code for the details.



Considerations when running NServiceBus on Azure



Azure has some limitations that might affect the way you have to design your NServiceBus solutions:




  • Maximum message size is 8kb. Message size has always been a consideration given that MSMQ has a fixed limit of 4 Mb but the 8kb limit will probably cause problems for users that transports large messages on NServiceBus. There are plans to introduce a “data bus” concept to NServiceBus to allow users to transport larger messages.


  • Messages are kept in the Azure Queues for at most 7 days. This shouldn’t be a big problem for normal usage (unless your endpoints are down for more than 7 days that is). But this also means that we can’t use the queues for subscription storage in the same way as we do with MSMQ.


  • All queue names on Azure must be lower case. Complete naming rules here.





What’s next



Right now we support Azure Queues for transporting messages but we can only store subscriptions and sagas using in memory storage. This is of course only acceptable for demos so we are working on adding support for the available storage types that Azure offers . The fact that we have NHibernate implementations for both subscription storage and saga persistence hopefully means that we already support SQL Azure Services but this has not been tested yet. (Ayende has more on this). The next step in this area is to implement support for storing subscriptions and sagas using the Azure Table Storage.



We’re constantly trying to improve NServiceBus both in terms of technology and guidance so stayed tuned for more!

Designing applications that can easily be reconfigured for the different stages in the development lifecycle is important when it comes to shorten the time between development and production. To achieve this we need ways to easily change our applications behavior as it moves through the various stages of software development. Having frameworks that help in this is of course a great start but it’s ultimately up to you as a developer to design and code your application to be lifecycle aware. Udi Dahan has a nice post on the subject that also describes how NServiceBus tries to help in this matter.

Enough talk let’s see an example on how we can combine NServiceBus and StructureMap to create easily reconfigurable applications.

Lifecycle awareness NServiceBus style

In my last post I explained how we could send emails in a “transactional” way using NServiceBus. The solution boiled down to a simple message handler that sends emails through it’s IEmailGateway dependency. Let’s see how we hook up to the built-in profiles of NServiceBus and configure our own dependencies accordingly.

The message handler was implemented as follows:

public class SendEmailMessageHandler : IMessageHandler< SendEmailRequest >
{
private readonly IEmailGateway emailGateway;

public SendEmailMessageHandler(IEmailGateway emailGateway)
{
this.emailGateway = emailGateway;
}

public void Handle(SendEmailRequest message)
{
emailGateway.SendEmail(message.ToAdress,
message.Subject,
message.Body);
}
}


To ease our integration testing we decide to swap out the implementation of IEmailGateway to something that just logs the request to a folder on disk so that we can verify the results either automatically or using our testers.  To do this we configure our endpoint as follows:



public class EndpointConfigWithAwareness : IConfigureThisEndpoint,
As.aServer,
IDontWant.Sagas,
ISpecify.ToRun< DemoEndpoint >,
IWantCustomInitialization,
IHandleProfile< Integration >
{
private static bool IsInProductionMode = true;

public void Init(Configure configure)
{
if (IsInProductionMode)
configure.Configurer.ConfigureComponent< SmtpGateway >(ComponentCallModelEnum.Singlecall);
else
configure.Configurer.ConfigureComponent< FileLoggingOnlyEmailGateway >(ComponentCallModelEnum.Singlecall);
}

public void Init(IConfigureThisEndpoint specifier)
{
IsInProductionMode = false;
}
}


As you can see we implement IHandleProfile<Integration> and uses the call to Init(…) to adjust the mode variable. With this in place we can easily configure the IEmailGateway dependency accordingly. So now when we start the NServiceBus in integration mode , NServiceBus.Host.exe /integration, our emails will be logged to disk instead of being sent to the real SMTP-server. 



Lifecycle awareness vNext



As you can imagine this solution gets kind of ugly when you have more dependencies and more profiles.  As a nice “coincidence” my container of choice, StructureMap, also has the concept of profiles. Please read Chad Myers post for an in-depth explanation. Let’s modify our configuration to combine the both:



public class EndpointConfigUsingStructuremap : IConfigureThisEndpoint,
As.aServer,
ISpecify.ContainerTypeToUse< StructureMapObjectBuilder >,
IDontWant.Sagas,
ISpecify.ToRun< DemoEndpoint >,
IHandleProfile< Integration >
{
private static readonly string IntegrationProfile = typeof (Integration).Name;

public EndpointConfigUsingStructuremap()
{
ObjectFactory.Configure(x =>
{
x.ForRequestedType< IEmailGateway>()
.TheDefaultIsConcreteType< SmtpGateway >();

//setup the alterations for the integration profile
x.CreateProfile(IntegrationProfile)
.For< IEmailgateway >()
.UseConcreteType< FileLoggingEmailGateway >();
});
}

public void Init(IConfigureThisEndpoint specifier)
{
//switch profile to "integration"
ObjectFactory.Profile = IntegrationProfile;
}
}


The code above starts with configuring NServiceBus to use StructureMap as the container. Then we move on to configure all our components using the StructureMap API. Using “CreateProfile” we creates a “Integration” profile where we configure the special version of IEmailGateway to use when running in that profile. With all this setup it’s straight forward to use the NServiceBus profile to activate the correct StructureMap profile. 



Conclusion



Building software that is easily configurable for different purposes is becoming increasingly important to enable rapid development and tight release cycles. To succeed make sure to pick frameworks that enables you to be lifecycle aware but in the end it’s up to you to “connect the dots”.



The source code an a fully working sample of all this can be found here.

Inconsistency problems in your application often occur when you are using technologies that are non transactional. Sending emails using SMTP is one example where there is no “rollback” in case something goes wrong. I want to talk about how asynchronous messaging in general and NServiceBus in specific can help you solve your inconsistency problems with minimal effort.

Let examine a common situation where a user has signed up for some service that you provide and you need to create the account and send a confirmation email to the user. The code would look something along these lines:

  
private IRepository repository;
private IEmailGateway emailGateway;

public void CreateUser(string userName,string password)
{
using(var transaction = new TransactionScope())
{
var user = new User
{
UserName = userName,
Password = password
};

repository.Save(user);

emailGateway.SendEmail(userName,
"Account created",
"Please click the following link to activate account: " +
"http://mysite/confirm&id=" + user.Id);


transaction.Complete();
}
}


Where the IEmailGateway interface is implemented like this:



  
public class SmtpGateway:IEmailGateway
{
private readonly string host;
private readonly string fromAdress;

public SmtpGateway(string host, string fromAdress)
{
this.host = host;
this.fromAdress = fromAdress;
}

public void SendEmail(string toAdress, string subject, string body)
{
var smtpHost = new SmtpClient(host);

smtpHost.Send(fromAdress,toAdress,subject,body);
}
}


At a first glance this code looks ok but the fact that SmtpClient doesn’t support transactions means that if our database call rolls back we’ll send the email without saving the user to our database. This inconsistency will result in users getting confirmation emails without being able to login. Inconsistency bugs is very painful to debug and usually only shows up when the system is under heavy load so they should be avoided at all costs. It’s also obvious that the duration of the synchronous call to the SMTP – server is a direct factor of the total call time and it’s probably just a matter of time before the performance police will come knocking on our door. Further our availability will now have to factor in the availability of the SMTP – server which will make it much harder to achieve high availability. I’ll save the in depth explanation of the availability aspects for a future post so let’s move on to see how we can solve our inconsistency problem by adding pinch of asynchronous messaging.



NServiceBus to the rescue



In order to be consistent we need to wrap the SMTP-call with something that supports transactions. Let’s accomplish that using NServiceBus combined with it’s MSMQ-transport to send a SendEmailRequest instead of actually calling the SMTP-server. The fact that MSMQ supports transactions means that if either the database call to save the user or the call to send the message fails nothing will be committed and we will still be in a consistent state. The modified code looks like this:



  
private readonly IRepository repository;
private readonly IBus bus;

public void CreateUser(string userName, string password)
{
using (var transaction = new TransactionScope())
{
var user = new User
{
UserName = userName,
Password = password
};

repository.Save(user);

var emailRequest = new SendEmailRequest
{
ToAdress = userName,
Subject = "Account created",
Body = "Please click the following link to activate account: " +
"http://mysite/confirm&id=" + user.Id
};

bus.Send(emailRequest);


transaction.Complete();
}
}


As you can see the code is almost identical as before with the important difference in that we’re now sending our email using a durable asynchronous call.



To process the message, on the same server or on a different server, we just implement a message handler for the SendEmailRequest message and host that handler using the generic host that ships with NServiceBus:



  
public class SendEmailMessageHandler : IMessageHandler
{
private readonly IEmailGateway emailGateway;

public SendEmailMessageHandler(IEmailGateway emailGateway)
{
this.emailGateway = emailGateway;
}

public void Handle(SendEmailRequest message)
{
emailGateway.SendEmail(message.ToAdress,
message.Subject,
message.Body);
}
}


If the message handler fails to process the message after the configured number of retries the message will be moved to an error queue where the administrator need to decide what needs to be done in order to resolve the issue.



Conclusion



With a minimal change to our original code we can now guarantee consistency in our application. This setup will also be more testable because we only have to worry about stubbing out the SMTP-server when we are testing the message handler. The fact that we no longer have to wait for the SMTP– server will shorten our call times and boost performance. As a final benefit our application can now function with out the actual SMTP-server being up and running.  This means that our availability is no longer limited by the availability of the SMTP-server.

 

The packaging of NServiceBus 2.0 has been slightly changes compared to 1.9 The reason for this is that the merged NServiceBus.dll started to get to big. While we still strive to keep the number of assemblies down we decided that a split was necessary.

NServiceBus 2.0 will ship the following assemblies:

NServiceBus.dll

Contains only the interfaces and classes that enables you to implement your business logic. This means message handlers sagas, classes with a dependency to the IBus interface etc. This is essentially a lightweight assembly containing the public Api of NServiceBus

NServiceBus.Core.dll

This assembly contains all the core parts of NServiceBus, transports, subscriptions storages, serializes, etc. Any projects that hosts NServiceBus endpoints needs to reference this assembly. NServiceBus and NServiceBus.Core are the only mandatory assemblies so if you are providing your own hosting solution these are the only things you have to reference. (most likely if you are hosting NServiceBus within a asp.net web application)

NServiceBus.Host.exe

NServiceBus 2.0 comes with a new generic host that will take care of hosting your business logic in a flexible way. (Based on topshelf). Justin Ramel has great post on how to use the host

Additional Containers

2.0 will ship with Spring.Net as the default container. If you prefer to use one of the other supported containers, AutoFac, StructureMap, Windsor and Unity you have to reference the NServiceBus.ObjectBuilder.[Name of Container].dll that ships with NServiceBus. We also supply the versions of the actual containers that NServiceBus is compiled against. If you prefer you own version just reference the specific version and assembly redirect accordingly.

The current trunk contains the above packaging and should hopefully be quite stable as we are closing in on the release of NServiceBus 2.0.

Team City has a killer feature called remote run. This gives you the ability to test your changes on the build server before actually committing anything.

Why is this needed, you might ask? 

Let me try to explain with an example. I’ve recently committed some features to NServiceBus where the tests where using Sqlite for in memory testing of NHibernate mappings. The coding went smooth and all test where green “on my machine”. I’ve danced the checkin dance and finally committed the changes.

What do you think happened?

Well the “it works on my machine” devil paid me a visit and made sure that the Continuous Integration choked with a classic “Bad Image Format” exception. It turns out that the build agent over at http://teamcity.codebetter.com is 64-bit and that caused the NUnit test runner to fail because I’ve compiled against the x86 version of Sqllite. But before I’ve got this fixed I had caused 10 failed builds, I don’t have a x64 machine so I had to do trial and error debugging. And trust me repeatedly failing the build is a great way to piss off your fellow developers. If I had been using the “personal build” feature before I committed  this wouldn't have happened.

To start using this feature do the following:

  1. Download the Team City Visual Studio plug in (login to your Team City server and click on “My Settings & Tools”)
  2. Download the CollabNet Subversion client and install it
  3. Open visual studio and
    1. Login to Team City (from the “Team City” menu)
    2. Click on “Remote run”
    3. Make sure subversion support is enabled (see picture below)

personal_build

That should be it. Another great “move” added to my check in dance!

Thanks to Tuna Toksoz for helping me with the Sqlite issue and reminding me of the “personal build” feature!