Live Fast, Code Hard, Die Young

Archive for the ‘Guides’ Category

Python rocks – on Windows too

I always thought of Python as a language mostly useful for doing stuff on Linux machines. You know, like a pretty nice little language for scripting tasks but not useful for doing advanced stuff.

Well, I have changed my mind…This past year I have had the pleasure of working a lot in Python and I have come to like it very much. Python is the bomb!

So what I have done with it? Well…

  • I have built massively scalable cloud services running on Google App Engine.
  • I have built command line tools for intelligently packing sprite sheets, extracting font information, performing custom encryption, modifying XCode projects and more…
  • I even started working on a cross platform Git client application…

…and I must say that I have really enjoyed it very much.

Python has some really nice benefits:

  • The code is easy to read – very clean and compact
  • It is cross platform
  • Compiles to exe
  • Has lots and lots of third party libraries to do anything
  • Good integration with C
  • With Cython you can transform Python to fast performing C code
  • Interactive console (easy to try things out)
  • Large community
  • It is wrist friendly – no curly bracket typing 🙂

But is it really useful for any serious development? Well, you tell me! Many well known applications and services are written in Python such as GMail, Google Maps, Dropbox, Spotify and Mercurial to name a few…

But…can I really use it on Windows? Python originated in the Unix world and must therefore not work properly on Windows, right? Hmmm, I think this is the biggest hurdle for Python acceptance. Many people thinks it is hard to use on Windows and yes…that was my notion too until I tried it. As a matter of fact it is easy to install and it works really well. Let’s take a look!

How to get started – Python essentials

Before getting started, make sure you have Chocolatey – the lovely package manager for Windows.

I believe there are three essential things you need to have a basic Python setup on your system:

  1. Python itself
  2. A Python package installer
  3. Virtualenv for Python

The first two are the most important so I will leave the last out one for a future post.

Installing Python
With Chocolatey it is very easy to install the latest version of Python:

C:> cinst python.x86

This will install the 32 bit version which I recommend for now.

UPDATE: This command will install the latest version which is now Python 3.4. If you want to run an older version you can supply a version argument like this:

C:> cinst python.x86 -Version 2.7.6

This installs a specific version of Python.

Once this is done you can actually start writing code using any text editor you like. Notepad works but it is nice to have some syntax highlighting and I recommend either Sublime Text or Notepad++. Even better is to use a full featured IDE such as PyCharm which I use. It has more advanced features that makes your life easier!

Installing PIP
Once you start working with Python you realize that you sometimes need 3rd party packages. To install them you need the Python package manager PIP. You can install PIP using easy_install and easy_install can be installed with Chocolatey. Isn’t that lovely? 🙂

C:> cinst easy.install

After you have run this, close the command prompt window and open a new one to ensure that the environment variables are reloaded. Then run this:

C:> easy_install pip

When this is complete you are all set to go. You can now run Python stuff directly from the command prompt. You can even install Python scripts using pip if you just want to get some nice tools written in Python in this way.

What about virtualenv?
I told you there are three essentials you need on your Python system. The last thing is virtualenv. All packages you install with PIP will be installed globally unless you use virtualenv. For now, that is ok. In a future post I will explain how to use virtualenv to setup isolated development environments.

Using Python

To show a little example of how you can use Python, open up a text editor and paste in the following Python code. This short snippet will calculate the SHA-1 hashes of all files in the directory that you are in.

import hashlib, os
for f in os.listdir('.'):
  if os.path.isfile(f):
    print hashlib.sha1(open(f, 'rb').read()).hexdigest(), f

Save it out as hash.py and run it (from the command prompt) and it will show something like this:

27286076704f65d178baa46e66232a9e20d7e3dc diskstation-dir.bat
b8f7f52219520b64cac9c8ad49f533ed40cbbac8 fix-vs.py
6ebe48a8f124447600cb2bcf4135b5586933d0d2 import-itunes.bat
200b0b05edd6638aa43f1281f7a5f77bb2517303 killsteam.cmd
c86b8442e1e0561ca1715874a9d8fddda4a0d965 md5hash.py

 

That’s quite nice for only four lines of code. Python is often like that – few lines of code and high productivity. I like that.

For some reason Python is not very popular on Windows. I hope this will change in the future. If you have never tried it – give it a spin. You may like it!

 

Advertisements

Chocolatey delicious package manager goodness

As a long time Windows user I have always found it nice and easy to use UI’s to perform various tasks. However, some things are actually quite a bit nicer to do from the command line. One such thing is installing programs. Installation is mostly a tedious procedure where you have to go and find an installer to download, run the installer and carefully select not to install any crap with the software etc. It is a series of steps that differ from program to program and they are not especially fun – just something you have to do.

So what can we do about this? Well, we can use Chocolatey!

With Chocolatey all program installation become streamlined into a single command line operation. This means you have to think less about what to do, just look up the package you need and type it in. Package names are also easy to remember or maybe put in a text file so you have them when you need to re-install or upgrade apps. It’s very nice.

Chocolatey has been around for a few years and I actually tried it when it was released but found it to be a bit rough on the edges back then. Now it works much better! And the amount of packages available has also gone up considerably.

Installing Chocolatey

It is super easy to install Chocolatey. Just open up a command prompt (hit Windows+R, type cmd and press Enter) and then paste in the following:

C:\> @powershell -NoProfile -ExecutionPolicy unrestricted 
-Command "iex ((new-object net.webclient)
.DownloadString('https://chocolatey.org/install.ps1'))" && 
SET PATH=%PATH%;%systemdrive%\chocolatey\bin

 

Yeah, this is a mouthful of stuff. What it does is to run a Powershell command that downloads and runs a script that installs Chocolatey. This could have been done with a regular installer program but this is a way to promote installation via the command prompt to make it easy to get going.

Using Chocolatey

Once you have installed Chocolatey you can go ahead and install programs from the command prompt by simply typing cinst:

C:\> cinst vlc

 

This command will install the latest VLC for you in one simple step (you need to click Yes in the UAC prompt though). This works even if you already have VLC installed. Go ahead and try it now!

Just for fun I recorded a video of the installation procedure just to show you how easy it is to set it up.

 

So, there you have it – package management for Windows. With my Mac I use Homebrew and it is lovely and I’m really happy to see the same type of tool coming to Windows too. If you haven’t tried it yet – now is the time! 🙂

Rake tutorial

Rake is a great tool for creating build scripts even if you primarily develop on Windows and normally don’t use Ruby. In my last article I showed you how easy it is to set it up and get started. This time I am going to show you some more examples of how it can be used.

One thing I really like about Rake build scripts is that you can divide the work into many small tasks that can be developed and tested individually. When you got them all working it is just a matter of chaining them together using the Rake dependency mechanism and you’re done. This speeds up the development of build scripts quite a lot in my opinion.

Dependencies

So how do we setup dependencies between tasks in Rake? Well, it’s very easy. The Ruby language makes the syntax pretty sleek as well:

task :deploy do
  puts "Deploying..."
end

task :default => :deploy do
  puts "Done!"
end

In this example I have two tasks called deploy and default. The default task is dependent on deploy, which means that Rake will run the deploy task first and if it is successful it will continue with the default task. Pretty simple really.

You can also create empty tasks that simply depend on others. If you have more than one dependency just surround the tasks in brackets:

task :release => [:newversion, :deploy, :package]

By default Rake will run the default task. If you want Rake to run the release task instead you can do that:

C:\>rake release

Internal and public tasks

Once I started filling my build script with different tasks I ran into the situation where I wanted some tasks to be considered “internal”. I wanted to run them on my own for testing purposes, but they were not meant to be run by others. Instead I wanted to expose some sort of public API of tasks to be run from the command line to guide the user of what was possible. Rake has a way of handling this by decorating tasks with the special keyword “desc”. If you put a “desc” statement before the task it becomes the public documentation for that task, which to me is a great way of exposing it as part of a public API. This is how you do it:

desc "Perform a deploy and package a release version"
task :release => [:newversion, :deploy, :package]

To view a list of documented tasks you simply run Rake with the -T parameter. Here is an example of how this looks in one of my projects:

An example of output from Rake

This gives a nice list of all the commands available to a user that is new to my project. I like this a lot. 🙂

If you want a little bit more involved example you can take a look at the rakefile that I use for one of my projects. You can find it on GitHub.

Next time I will show you how to use Rake to build .NET projects. Stay tuned!

Using Rake in .NET projects

Last year I started using Rake for my .NET build scripts and it has been a quite pleasant experience that I wanted to share with you. “Hmmm, isn’t Rake one of those scary Ruby tools from the other side of the fence?” you might ask. Yes, indeed, it is a Ruby tool! Many good things have come from the Ruby community but sadly many .NET developers seem to be scared of even learning about them. “Yuck, strange Ruby stuff from the Linux world! I can’t use my .NET knowledge and that must be a bad thing, doesn’t it?” Well, that’s a shame! Rake can be used on Windows! And besides, I think it never hurts to at least dip a toe in the ocean once in a while and experience something different than the usual stuff you work with. When a tool helps you to accomplish things with simple elegancy it couldn’t hurt to try it.

So what is Rake?

Rake is a tool for creating build scripts. It is a modern variant of the good old Make tool where you describe different tasks and the dependencies between them. Each task is a series of Ruby statements to be executed when the task runs. The flexible nature of the Ruby language makes the task descriptions very brief and elegant.

Here is an example of a task called “deploy” that copies files:

task :deploy do
    src_path = File.join(BASE_PATH, "Output/bin/.")
    dest_path = File.join(BASE_PATH, "Deploy")
    puts "Deploying files..."
    FileUtils.cp_r src_path, dest_path
end

Why use Rake?

Okay, now we know a little about what Rake is, but you may still ask why do we need build scripts at all? Can’t we do everything with Visual Studio and some pre/post build commands?

Sure, we can do quite well with just Visual Studio for small projects. But in larger and more complex projects we often run into situations that require some form of automation and we don’t want to do them every time we build with VS. We need scripts to do various stuff such as deployment and packaging. Maybe we need to update some files with version info, compile an installation kit, upload it via ftp to a server and so on. I’m sure you can come up with a range of time consuming stuff you do on a regular basis that could be done more quickly and reliably by a script instead. The more you can automate the tedious repetitive tasks the more time you can spend doing things that really matter to the customer.

So why not use a simple bat file to run some commands? Yes, that’s one solution but bat files have some significant problems. One is that the error handling is quite cumbersome, you have to use goto statements for flow control and you cannot easily reuse functionality across different bat files. In my experience, as soon as the complexity of the build script rises it will quickly blow up in your face and become a maintenance nightmare. Another glaring omission is the ability to express dependencies between tasks. That is one pretty important thing for structuring a build script successfully that bat files lack.

But what about MSBuild? Wasn’t it created for this purpose exactly? Yes, I have used MSBuild (and before that NAnt) and it sort of works, but it feels very limited. Both MSBuild and NAnt are XML based and declarative. For me, declarative programming is not very intuitive (XSLT anyone?). I feel that I have to think harder before I get things right, instead of just describing what to do in the order I would do it when performing the task manually. With MSBuild I get these set of predefined tasks that I can use, but if they don’t do exactly what I want I’m out in the cold. To me, it feels much more powerful and flexible to write code to do what I want in a build task, and easier to understand and maintain too. Maybe it’s a matter of taste so ultimately you’ll have to decide for yourself…

Getting started

Many people are scared because they think it is difficult to setup Ruby and Rake on Windows. This is not the case. To get started you simply run the Ruby Installer that you can download here:

http://rubyinstaller.org/

After installing Ruby, hit the Windows key and type “Ruby”. Then click “Start Command Prompt with Ruby” in the list that appears. This will bring up a command window where you can use Ruby commands.

To install stuff in Ruby you use “gem”. To install Rake simply type:

C:\>gem install rake

Now you are good to go!

Your first script

By default Rake will look for a file called “rakefile.rb” so create a text file with that name and write your first build task like this:

task :default do
    puts "Wonderful world!"
end

Then launch this script like this:

C:\>rake
Wonderful world!

Pretty simple, wasn’t it?

In a future post I will show some more examples of what the build scripts may look like, so stay tuned!

Watching DVB-C cable TV in Media Center using Anysee E30C

Getting digital cable TV (DVB-C) to work in Windows 7 Media Center (MCE) can be really difficult and there isn’t much information available on what to do. Watching encrypted digital cable TV isn’t even supported by Microsoft so how do we solve this? Well, there are a couple of products that can help and one of them is the Anysee E30C digital cable TV card. It has a pretty cool driver that pretends to be a DVB-T card (which MCE supports) and it properly handles encrypted pay TV channels (provided that you have a valid smart card and subscription of course).

 

Preparations

This should probably be obvious, but the first thing you need to do is to install the latest drivers for the Anysee card. The drivers on the CD you got when you bought the card are probably outdated so make sure you get them from the Anysee website.

Remember to reboot after installing the drivers! Installation doesn’t require you to restart but I’ve had weird problems when I skipped this…

Now, before even trying to get the card working in MCE you should ensure that TV works in the Anysee Viewer application. (You may need to manually scan for channels if your network provider is not one of the supported ones.)

 

Watching Canal Digital Sweden

The Anysee card comes preconfigured with settings for different cable TV networks around Europe. However, Canal Digital Sweden is not one of the supported providers. We need a little bit of investigation and configuration to make this work. (This is something you can do for your provider too if needed.)

First of all we need to figure out the settings that Canal Digital uses in Sweden. I did this by examining the information menus in my set top box which had a screen listing channel information with frequencies and symbol rate:

IMG_0221

This revealed the channel frequencies and that symbol rate 6952 is used. Some searching on the internet also revealed that QAM 64 is used (which I think is the most common modulation in Sweden). After we have this information along with the frequencies for the channels we are ready to start configuring.

 

Configuring Anysee CNO

The Anysee CNO application is responsible for simulating the DVB-T card that MCE uses for watching TV. By default it resides in the C:\Program Files\anysee\Driver directory (or C:\Program Files (x86)\anysee\Driver if you are running 64-bit Windows).

If you look in the subdirectory Transponders\Cable you will find a bunch of preconfigured files for different networks. Unfortunately there is no settings file suitable for Canal Digital Sweden here with QAM 64 and symbol rate 6952. What I did was that I just copied a similar one and renamed it to EU_64QAM_6952.TPL as shown below:

image

Then I edited the file to make sure the symbol rate said 6952 and verified that the frequencies matched the ones listed by my set top box. In my case everything looked alright so all I had to do was to change one line in the .TPL file:

Symbolrate: 6952

After adding a new settings file we also need to tell the CNO application about it. This is done by editing the CNO.CNO file found in the application directory C:\Program Files\anysee\Driver:

image

In the CNO.CNO file at about line 143 or so you should find a [Cable-List] section. Just add your new file here similar to what I did:

image

It should look something like this when you are done:

[Cable-List]

// ±¹°¡ÄÚµå, Áö¿ª, ¼­¹ö½ºÁ¦°øÀÚ, ¸ñ·ÏÆÄÀÏ

-1, , Europe_64QAM_6875, .\Transponders\Cable\EU_64QAM_6875.TPL

-1, , Europe_64QAM_6900, .\Transponders\Cable\EU_64QAM_6900.TPL

-1, , Europe_64QAM_6952, .\Transponders\Cable\EU_64QAM_6952.TPL

-1, , Europe_128QAM_6000, .\Transponders\Cable\EU_128QAM_6000.TPL

-1, , Europe_128QAM_6875, .\Transponders\Cable\EU_128QAM_6875.TPL

-1, , Europe_128QAM_6900, .\Transponders\Cable\EU_128QAM_6900.TPL

-1, , Europe_256QAM_6875, .\Transponders\Cable\EU_256QAM_6875.TPL

-1, , Europe_256QAM_6900, .\Transponders\Cable\EU_256QAM_6900.TPL

358, , Iisalmen_Puhelin_Oy(IPY), .\Transponders\Cable\Finland_IPY.TPL

358, , TSF, .\Transponders\Cable\Finland_TSF.TPL

358, , Entire, .\Transponders\Cable\Finland_Cable.TPL

Now you need to restart CNO and change the settings. You can exit CNO by right clicking its icon in the tray bar and selecting Exit.

image

After shutting down CNO just relaunch it from the program files directory (C:\Program Files\anysee\Driver\CNO.exe). Then locate the tray icon again and select Settings from the context menu. This will bring up the settings dialog where you can now find your new settings file that is suitable for the Canal Digital Sweden digital cable TV network:

image

Configuring Media Center

Next thing to do is to actually get it working in MCE so start it up and go to the Settings->TV->TV Signal to setup the new card:

image

In the next screen select Setup TV Signal and then select your region and enter your postal code. I’ve experimented a bit with these settings but as far as I know it doesn’t matter if you enter 00000 for postal code or the correct one. I suspect it may have something to do with the guide listings but I’m not sure. Things seems to work no matter what I choose here but try entering correct data to make sure you don’t screw up things.

After agreeing to the license you get to choose the type of signal to receive and this is important. You need to select “Antenna” here because that is what MCE supports best and it is also what Anysee CNO simulates:

image

Then select “No” for set-top box and in the following screen choose Digital Antenna (DVB-T) signal:

image

When it asks if you want to set up any other TV-signals select “No” and then proceed to scan the channels. This should take a while but when it is ready you should be able to watch Live TV in MCE using your Anysee card!

Running Silverlight unit tests in TeamCity using StatLight

JetBrains TeamCity is a wonderful product that we use for build management and continuous integration in our .NET and Java projects. The latest version adds support for .NET 4 among other things. However, it does not come with support for running Silverlight unit tests out of the box. In this post I will describe what I did to set this up in TeamCity 5.1.

 

Building the Silverlight project

First of all I am assuming that you have setup a configuration in TeamCity that is able to build your Silverlight project successfully on the server.

I had some problems with building our Silverlight project at first because we are using WCF RIA Services and there does not seem to be any way to install the SDK without having VS2010 installed. Well, actually there is one way but it doesn’t install the Silverlight client libraries that you need to build but only the things you need to run RIA projects on the server. I finally gave in and installed VS2010 on the server although I didn’t like it. However, that solved the build issues for me.

Setting up StatLight configuration

Next we take advantage of a lovely little tool called StatLight. It is used to run Silverlight tests more efficiently when you are practicing TDD. It runs the tests without showing the browser window that the regular SLUT framework uses and it has a really nice “continuous” mode that can monitor your project and re-run tests automatically whenever you rebuild your solution. Last but not least, it has support for producing TeamCity compatible reports of the test run!

To use StatLight with TeamCity we need to create a new build configuration to run a command line build. I set it up like this:

General Settings

You can set the general settings as you like but I used the defaults for most of it.

image

Version Control Settings

Under the VCS settings I made sure to set the same checkout directory as I have in my main build configuration. The idea here is to reuse the output from that project when we run this one.

image

Build Runner

Next is the Build Runner settings. Here I have specified that I want to run a command line tool and the path to it is:

BuildTools\StatLight\statlight.exe

This will run StatLight from within the checked out sources where I have put my tools used for building.

If you want you can of course put StatLight in some other path on your server, but I like to include the tools needed to build in the version control system so the right versions of the tools for a project are always present. This way I can upgrade tools in one project and simply commit them in version control and have the build server pick it up automatically.

Next I also needed to configure the parameters to give StatLight:

-x="Source\Tests.Gws3.Client\bin\Release\Tests.Gws3.Client.xap" -v=April2010 –teamcity

This tells StatLight where to find the XAP-file with the tests. (This file was actually built using our main build configuration so we need to setup a dependency on that.) We also specify which version of the testing toolkit we want to use and that we want StatLight to output a TeamCity compatible report.

image

Build Triggering

Next up is Build Triggering settings. When do we want to run our tests? Well, I think it is a good time to run them whenever our main build has been built successfully so I setup a build dependency trigger for that.

image

Dependencies

So are we done yet? No, not quite…We also need to specify that our build configuration is dependent on stuff from another build project. This means that if that project is out of date it will be rebuilt before we run ours.

image

 

Conclusion

It seemed like a lot of steps to get it all up and running but it really isn’t that much work and once it is setup you can enjoy full continuous integration bliss for your Silverlight projects as well!

If you haven’t tried out TeamCity yet I suggest you check it out! It is free for up to 20 build configurations which should be more than enough to get you started.

Using MEF as an IoC container

The Managed Extensibility Framework is a new wonderful addition to .NET 4 and Silverlight 4. The main purpose of MEF is to handle the extensibility and plug-in capability of an application. It features a very simple and elegant method for creating objects and resolving dependencies by decorating your code with import and export attributes.

Managing and resolving dependencies and creating objects is pretty much what a simple IoC (Inversion of Control) container does. So, can we use MEF for this?

The MEF man Glenn Block once said that ”you should use MEF to manage your unknown dependencies and an IoC container to manage your known dependencies.” However, I have found that MEF can work pretty well as your one stop solution for all dependencies. It is especially nice if you are building a Silverlight application that already uses MEF (for example to download XAPs dynamically). Why include a separate third party IoC container when there is one built in already?

Here is a short guide for those who want to use MEF for dependency injection in Silverlight.

MEF libraries

First of all, you need to add a reference to the MEF libraries. They are included with Silverlight 4:

image

System.ComponentModel.Composition – You need this reference anywhere you use the basics of MEF such as importing and exporting.

System.ComponentModel.Composition.Initialization – You need this reference where you actually initialize and configure the container.

ServiceLocator class

To use MEF for resolving dependencies I have built a simple class to set things up and provide a way to create or retrieve instances. This is the code for a simple version of this class:

    public static class ServiceLocator
    {
        private static CompositionContainer _container;
        private static AggregateCatalog _catalog;

        public static void Initialize()
        {
            _catalog = new AggregateCatalog(new DeploymentCatalog());
            _container = CompositionHost.Initialize(_catalog);
        }

        public static T GetInstance<T>()
        {
            return _container.GetExportedValue<T>();
        }
    }

The Initialize() method sets up an AggregateCatalog and feeds it with a DeploymentCatalog. The AggregateCatalog can contain many MEF catalogs that are added at runtime. Using an AggregateCatalog is not absolutely necessary in this simple example but it is a preparation for loading dependencies dynamically in the future.

The DeploymentCatalog is used in Silverlight for working with XAP files. It is a very handy class for loading external XAPs if your application is split up in modules. Creating a DeploymentCatalog without any parameters will create a catalog for the main application XAP file and this will allow us to access the exported types in all the assemblies of our main XAP file.

When we call CompositionHost.Initialize() our assemblies will be scanned by MEF and all exports are discovered. We are then ready to call the GetInstance<T>() method whenever we need an instance of an exported class.

Exporting types

Traditional IoC containers often involve configuring components using either XML or registering them using code. With MEF you simply put an [Export] attribute on your class to indicate that it should be available for composition:

    [Export]
    public class Car
    {
        ...
    }

This will expose the type Car to MEF and lets you retrieve it in IoC fashion like this:

    var car = ServiceLocator.GetInstance<Car>();

Often you want to expose a certain interface that your type implements. You can do this by supplying the type in the [Export] attribute like this:

    [Export(typeof(ICar)]
    public class Car : ICar
    {
        ...
    }

The default behavior in MEF is to treat exports as singletons. That is, if you don’t specify anything to override this behavior you will only get one instance of the Car object in your application no matter how many times you call GetInstance<Car>(). Actually, if the exported type does not specify anything the default is to allow either singleton or non shared so it is up to the caller to decide what he wants.

If you want unique instances to be created every time you retrieve an instance of your type you can specify this using an extra attribute on your class:

    [Export]
    [PartCreationPolicy(CreationPolicy.NonShared)]
    public class Car
    {
        ...
    }

Dependency injection

Of course an application is seldom built with only a single class. Most likely you have many parts that fit together and you want to use the IoC container to resolve everything smoothly for you using dependency injection. MEF can do this as well, but you have to decorate the constructor to use with the [ImportingConstructor] attribute:

    [Export]
    public class Car
    {
        [ImportingConstructor]
        public Car(Engine engine)
        {
        }
    }

    [Export]
    public class Engine
    {

    }

When retrieving a Car instance MEF will automatically create and inject the Engine in the constructor of the class just like an ordinary IoC container would.

All in all, those 20 lines of code is all you need to get started with MEF as an IoC container. You can now retrieve instances of your objects with injected dependencies and that’s pretty much what you need in most cases.

Conclusion

I wanted to keep this example as simple as possible and have deliberately left out topics like object lifetime, dynamically loading XAPs and so on. I am planning to write about those things in upcoming posts.

I think MEF provides a great alternative to the traditional IoC containers. It is very easy to setup and get started with. I also like the fact that all configuration sticks with the class, which makes it easy for newcomers in your project to pick things up quickly and write new components simply by looking at an existing class.

I believe MEF is going to be central in all .NET 4 development. Don’t miss out!