We have been doing some work with NServiceBus recently and observed some unusual behaviour which was caused by our mistake and seemed worthy of a small post.
The Scenario
In our solution we were doing some standard NServiceBus stuff by pushing a message to a queue using NServiceBus. We had a direct send/receive scenario rather than a publish/subscribe one.
The background process which was meant to collect the message and then process it was a normal NServiceBus message handler. We would run the NServiceBus.Host.exe which would find the handler and then do the usual NServiceBus magic.
The Problem
In this solution we were creating some automated tests around this module of the integration process to ensure that it would work well. We had two tests.
Test 1
This test would start NServiceBus.Host.exe using the Process object, then seed a message to the queue via our web service façade sitting above the queue which wrapped NServiceBus. The background process would then process the message and the test would check the message had been processed fine.
If all was well then the NServiceBus.Host.exe process was stopped.
Test 2
In test 2 we would do a very similar thing except that instead of starting the process the test would install NServiceBus.Host.exe as a windows service and then start the service before the test and once the test was executed it would stop the test.
The Results of the Tests
Test 1 worked really well, however in test 2 we found that it didn’t really work at all, instead of doing the background process we were finding that between mqsvc.exe and NServiceBus.Host.exe the CPU on the machine was maxed and nothing was really happening.
The Solution
After trying a few things we found it was the permissions on the queue were not set correctly. Once this was resolved it all worked fine and CPU was not excessive and ran just like the console application.
I think the couple of take aways from this are:
Make sure you set the windows service for NserviceBus Generic Host to the right credentials
When you install the generic host as a windows service then by default it will use the default windows credentials. For any production like scenario you should be using a domain account to run the process as via the windows service.
Make sure you have the queue set with the right permissions
For the credentials you have used to configure the generic host as a windows service you should ensure that this user has the appropriate permissions for any queues it will interact with.
Make sure you turn on the right logging configuration in NServiceBus
When this wasnt working correctly we didnt know there was an issue, we were just experiencing the high CPU condition. I am a little surprised that there wasnt something logged and that the process didnt crash. I guess this could be by design bearing in mind that the process could be monitoring many queues. In this point Im just saying that originally we didnt have all of the log4net logging which is available from NServiceBus turned on. Its probably a good idea to have this turned on and configured until you are happy your solution is working fine.
Thanks to Ahmed Hashmi on my team who got this working in the end.
