JetBrains TeamCity is a wonderful product that we use for build management and continuous integration in our .NET and Java projects. The latest version adds support for .NET 4 among other things. However, it does not come with support for running Silverlight unit tests out of the box. In this post I will describe what I did to set this up in TeamCity 5.1.
Building the Silverlight project
First of all I am assuming that you have setup a configuration in TeamCity that is able to build your Silverlight project successfully on the server.
I had some problems with building our Silverlight project at first because we are using WCF RIA Services and there does not seem to be any way to install the SDK without having VS2010 installed. Well, actually there is one way but it doesn’t install the Silverlight client libraries that you need to build but only the things you need to run RIA projects on the server. I finally gave in and installed VS2010 on the server although I didn’t like it. However, that solved the build issues for me.
Setting up StatLight configuration
Next we take advantage of a lovely little tool called StatLight. It is used to run Silverlight tests more efficiently when you are practicing TDD. It runs the tests without showing the browser window that the regular SLUT framework uses and it has a really nice “continuous” mode that can monitor your project and re-run tests automatically whenever you rebuild your solution. Last but not least, it has support for producing TeamCity compatible reports of the test run!
To use StatLight with TeamCity we need to create a new build configuration to run a command line build. I set it up like this:
General Settings
You can set the general settings as you like but I used the defaults for most of it.
Version Control Settings
Under the VCS settings I made sure to set the same checkout directory as I have in my main build configuration. The idea here is to reuse the output from that project when we run this one.
Build Runner
Next is the Build Runner settings. Here I have specified that I want to run a command line tool and the path to it is:
BuildTools\StatLight\statlight.exe
This will run StatLight from within the checked out sources where I have put my tools used for building.
If you want you can of course put StatLight in some other path on your server, but I like to include the tools needed to build in the version control system so the right versions of the tools for a project are always present. This way I can upgrade tools in one project and simply commit them in version control and have the build server pick it up automatically.
Next I also needed to configure the parameters to give StatLight:
-x="Source\Tests.Gws3.Client\bin\Release\Tests.Gws3.Client.xap" -v=April2010 –teamcity
This tells StatLight where to find the XAP-file with the tests. (This file was actually built using our main build configuration so we need to setup a dependency on that.) We also specify which version of the testing toolkit we want to use and that we want StatLight to output a TeamCity compatible report.
Build Triggering
Next up is Build Triggering settings. When do we want to run our tests? Well, I think it is a good time to run them whenever our main build has been built successfully so I setup a build dependency trigger for that.
Dependencies
So are we done yet? No, not quite…We also need to specify that our build configuration is dependent on stuff from another build project. This means that if that project is out of date it will be rebuilt before we run ours.
Conclusion
It seemed like a lot of steps to get it all up and running but it really isn’t that much work and once it is setup you can enjoy full continuous integration bliss for your Silverlight projects as well!
If you haven’t tried out TeamCity yet I suggest you check it out! It is free for up to 20 build configurations which should be more than enough to get you started.
Comments on: "Running Silverlight unit tests in TeamCity using StatLight" (26)
Thanks for putting this together.
FYI: You don’t have to specify what version of the Microsoft testing framework to use (-v={version}). StatLight detects it automatically.
I’m looking to remove this feature because if you’re build against the April2010 version and tell StatLight to use the March2010 (It will not correctly).
I had some problems at first when I left this flag out. Maybe it was because I did not have any tests at all in my XAP-file. I got an exception pointing me to including this option. I tried removing it now and it seems to work well without it so I’m not sure what my original problem was.
Thanks for the input!
You’re using TeamCity as Service and StatLight.v1.0 working?,
whenever i launch StatLight also in local it show a form,
so when i run it from TeamCity it throw exception
Yes, I’m using StatLight v1.0 and TeamCity build agent is running under Local System and I haven’t had any problems. StatLight does not show any form when I run it, but it briefly displays an IE browser window that quickly closes when I run it locally. What is the exception you get?
i launch with this command
C:\src\MyApp\Libs\StatLight.v1.0\Statlight.exe -x=MyApp.Views.Tests\Bin\Release\MyApp.Views.Tests.xap –teamcity
i’m running under administrator account my error is :
Eccezione non gestita: System.InvalidOperationException: Impossibile visualizzare un form o una finestra di dialogo modale quando l’applicazione Š in esecuzione in una modalit… diversa da UserInteractive. Specificare lo stile ServiceNotification o DefaultDesktopOnly per visualizzare una notifica ricevuta da un’applicazione di servizio
Ok, I don’t speak Italian so I’m afraid that doesn’t help much…
Do you get the same error if you run it from a command prompt on your build server? Does your test display dialogs or message boxes? Try removing all test but a simple one that only runs Assert.IsTrue(true) to see if that works.
/Pontus
from server if i run command prompt all work correct but it show this form :
http://yfrog.com/59statlightj
i run it with this command :
libs\statlight.v1.0\statlight.exe -x=.\servicecenter.services.tests\bin\debug\servicecenter.services.tests.xap
what’s wrong?
I’ve found a match with my problem
http://statlight.codeplex.com/wikipage?title=TeamCity%20Integration&referringTitle=Documentation
Excellent!
/Pontus
I use StatLight to run unit tests in two RIA Services solutions.
I have a VS solution containing 2 .xap files with unit tests.
In TeamCity, if I follow your article, I have to create one build configuration for each .xap file.
Is there a way to run StatLight in a unique build configuration that run tests contained in more that one .xap file?
For example, by adding multiple -x options, or by creating a build script designed to run StatLight on multiple .xap file at one in order to merge unit tests statistics.
Any suggestion? Thanks
I’m sorry but I don’t know if that’s possible. Maybe you should try asking in the Statlight forum to see if anyone can help you: http://statlight.codeplex.com/discussions
Yes, please do ask questions on the forum.
However as of the latest 1.3 release you can now specify multiple -x={file} parameters.
Let me know how it goes. (over at http://statlight.codeplex.com)
Hello Jason,
What a good news! Tomorrow I will try the 1.3 release and I will give you a feedback.
@Adriano,
yes, u can execute multiple xap files in Statlight V1.3. All u have to do is specifiy multiple -x arguments. eg.) -x=”test1.xap” -x=”test2.xap” and also u can log the combined report in single file.
[…] line tool which can be used to run a Silverlight testing project on the dev machine and in a TeamCity build server or using […]
This blog post is very useful, but I would not specify custom checkout directory. TeamCity is smart enough to choose the same directory for different build configurations if VCS settings are the same.
Please read carefully notes which appear if you set custom checkout directory.
This is working great for me except that I expected my output to appear more like NUnit does. What should the output look like? Right now I have the output xml attached as an artifact to make it a little easier to find. Do you have any suggestions? I have everything set up how you have shown above. Any help is much appreciated.
Thanks,
Bryan
I’m not sure what you mean. Make sure that you include the –teamcity option at the end of the command line statement when you run StatLight and it should show up just like NUnit tests does for regular projects.
I don’t see the tab (like NUnit tests).
My parameters are as follows:
-x=”Membership\Client\MA.Client.Membership.Tests\Bin\Release\MA.Client.Membership.Tests.xap” -v=April2010 –teamcity -r=lib\test.xml
Please let me know if I’m doing something wrong here.
Thanks,
Bryan
I don’t think you need that last part with the -r parameter.
I removed and still no luck. I was using that to write the .xml file to a location so that I could at least pick it up as an artifact which I know is kind of a hack.
There’s a problem with your blog post. The command parameters should have a double dash for the teamcity argument, like this:
-x=”Source\Tests.Gws3.Client\bin\Release\Tests.Gws3.Client.xap” -v=April2010 –teamcity
In your post the double dash got converted to a single long dash. Fixing it allows TeamCity to report on the number of unit tests that have passed and failed.
Thanks for reporting this. It’s something with WordPress. Don’t know how to fix it. But it’s good you noticed so people can fix it if they copy/paste from here.
Darn, the double dash in my comment also got converted to a long dash.
Thank you, that helped me
When I run my Silverlight unit tests from VisualStudio, it works fine. But when run using statlight from command prompt, it is causing errors. What could be the problem?