So what kind of tools do you use when writing an operating system?
Well, before I delve deeper into that discussion let me talk about where I come from tech-wise. I admit that I’m a Microsoft junkie. I’ve been using Microsoft tools and technologies since back in 1995 when I wrote my first C++ programs for Windows. Before that I used TASM/TLINK and wrote everything in x86 assembler in a primitive DOS environment and compiled it all with a good old BAT file…
I really like Visual Studio and would love to use it for OS development. After all, you get quite spoiled with the feature rich IDE, integrated debugging, intellisense and what not, but unfortunately it doesn’t cut the mustard when it comes to OS development. At least not the compiler/linker that comes with it. It is simply not suited for that task since it makes too many assumptions about what to include in the final executable. When you create your own OS there is no runtime library to rely on. You have to write all code yourself, including primitive functions such as memcpy() and strlen().
Instead I had to make a journey back in time to my old DOS days and the BAT file. This time however, instead of using TASM/TLINK, I picked out some tools based on advice from the OS dev community.
Compiler and linker
First of all you need a good compiler and linker toolset. I decided early on to develop in C since it is a high level language that still gives you raw access to low level structures and memory pointers. C++ is a viable alternative that I may investigate later on, but for now I use C.
GCC is a widely used C/C++ compiler with plenty of settings. It can produce highly optimized, tight code without unnecessary junk tying it to a specific OS.
Along with GCC I use the LD linker which takes the object files that the compiler generates and puts it all together in one executable file. The good thing about this linker is that it can create many different types of executables and even raw binary images which is very handy when starting out in kernel development.
These tools are readily available on Unix systems but what about Windows? Well, you can either use Cygwin but this seemed quite messy to me. I’m not interested in getting a Unix shell in Windows…Instead I believe the best option is to use DJGPP which is a collection of Unix tools that have been ported to Windows. This way you get both GCC and LD and some other handy utilities that work perfectly from the Windows command line. You can choose exactly which tools you need from the DJGPP toolset and installing is just a matter of unpacking the archives and updating the PATH environment variable.
I should also warn you not to use MinGW. This was the first choice I evaluated but it turned out to be a real mess with GCC and LD not supporting all the stuff I needed to compile and link correctly.
When you write an OS kernel you will inevitably need an assembler. Some things simply cannot be done in C because you need precise control over the stack and CPU registers. It is also more convenient to write longer pieces of code in a dedicated assembler file instead of writing messy inline assembler in C which is a total pain.
The GNU toolset contains an assembler (GAS) but I don’t like its weird syntax. Instead I have chosen NASM which has a similar syntax to TASM that I used before. It is open source and works really well.
Next up you need an emulator to run the OS in (yeah, you didn’t think I was going to keep rebooting my own system over and over again a million times while testing did you?)
There are quite a few emulators to choose among so you can just pick one that suits you. However, you should think about whether you need an emulator or a virtualization software. An emulator can emulate things that your current computer does not have, like multi core CPUs and exotic hardware, while virtualization programs often just simulate a "copy" of your current hardware (and thus it can run many many times faster as well).
Now it would be really nice to suggest a great development IDE here, but the fact is that I haven’t found any good one yet. I tried out Dev-C++ but it is too tied up to MinGW for my taste. And besides it was kinda quirky when editing code because it kept screwing up the indentation. I could probably work around the MinGW dependency in Dev-C++ but I rather try to do that with Visual Studio instead, i.e. have VS run a custom make tool.
In other words, right now I’m just using Notepad++ for editing and compile all source using a makefile. Although Notepad++ is a great editor, it’s not a development IDE. I feel I definitely need to improve this work environment as the project grows bigger…