On the shoulders of others – standing or leaning?

I’ve obviously been doing a lot of research recently, specifically into the entire booting and kernel loading process. Here are a few examples of such tutorials. But probably the most quoted and widely-used tutorial currently out there is the famous Bran’s Kernel Development Tutorial (bkerndev). 

This tutorial is so successful because it is fairly well-written, and because it gently guides the reader through some fairly thorny tasks that are necessary when building an OS on the IA-32 architecture. The tutorial takes us through the booting process, through setting up protected mode, and then shows how to set up interrupt handling and basic I/O with hardware such as the keyboard. 

I haven’t done any number crunching, but I think it’s fair to assume that the publishing of Bran’s tutorial led to a bit of a population explosion in the OS Dev world. Since its release, there has been a proliferation of OS projects on OSdev.org and other similar sites. The reason for this is that many developers work their way through Bran’s tutorial, and having done this, start up their own OS projects.

In reading through the forums on osdev.org, I get the sense that new OS projects based on Bran’s tutorial are sorely frowned upon. Just recently, someone announced a new version of their OS, and they were promptly beat down for having done nothing but make a few changes to the bkerndev code. I can see how frustrating such projects must be for OS dev veterans, and such beatdowns are probably warranted in many cases, but I would like to take a moment to explore the pros and cons of “standing on the shoulders of those that have come before” as an approach to OS development.

The first time that I tried to develop M3, about 10 years ago, there was no Bran’s tutorial. There wasn’t anything like it. Information about boot sectors, and loading kernels, etc, was hard to come by. Besides the Intel manuals, there weren’t too many reliable sources of information for those interested in tinkering with OS dev. And in my case, the result of this lack of information was that I gave up. After getting a very rudimentary boot sector working, that basically just booted from an actual floppy disk and printed a welcome message to the screen in real mode, I gave up. As a one-man team, I didn’t have the time or resources to figure out how to set up the necessary scaffolding for my kernel. I was interested in memory management, and process scheduling, and file systems, but I never got anywhere near that point, because of the immense amount of work that needs to be done just to get the environment initialized and in a state such that control can be handed off to the kernel.

Fast forward to today. With Bran’s tutorial and others like it, I could use the code provided and build myself that necessary scaffolding in a matter of days, not months or years. Bran’s Kernel Dev tutorial does not teach you much of anything about developing the bulk of the kernel itself. Instead, it walks you through the environment setup that needs to be done to be at the point where you can really focus on kernel development. bkerndev is somewhat of a misnomer. It should be called bkernenvsetup or something like that.

Back to the forum beatdowns. If someone merely takes Brandon’s code, tweaks a line here or there, and tries to pass it off as his own OS, then absolutely he should be raked over the coals (or perhaps just ignored, if flamewars aren’t your MO). However, if someone is using such code as the means to an end, the end being that they get to start working on their actual kernel, which will run on top of all the Bran scaffolding, then I would argue that that’s a valid use of the code. In fact, Brandon himself says so, right in the introduction. He lists the steps that the tutorial will take, with the final step being:

12) …and the rest is up to you!

Using Bran’s code to get to the meat of kernel development is a legitimate and pragmatic approach to OS development. Such (re)usage of code is an example of standing on the shoulders of those that have come before, which has been a valid and acceptable (even encouraged) practice in the open source world for.. well, ever.

At this point (if you’ve actually read this far), you’re saying to yourself “Great, this guy is just trying to justify cutting corners in his OS project”. Not so. I’m not trying to do any such thing. I’ll admit that I have read Bran’s and many other tutorials, and have used them as reference implementations for some of my stuff, but I don’t plan to just copy the code and use it without having any understanding of what’s going on. Quite the opposite – I use such tutorials precisely so that I can understand what’s going on. And I think that’s the critical difference. By reading through the tuts, by going through the provided code line by line and really understanding what’s going on, you come to “own” the code. If you know the code back and forth, and you get to the point where you understand it as well as if you had written it yourself, then you are truly standing on the shoulders of those who have come before. They have helped to lift you up to a level of understanding. And that’s something great. That’s not something to beat down.

So is my plan with M3 to simply build on top of Bran’s environment? No – that’s not the plan. But I do plan to use such tutorials to give me a leg up, so that I have a fair chance of actually getting to the point where I’m working on the real meat of the kernel. I plan to use to the code as a reference implementation of the concepts explained in the Intel manuals and other reference books. Being able to read something in the manual, and then see some actual code that illustates the concept is a great way to learn and understand. 

I’d like to get to step 12 at some point in my lifetime, and standing on the shoulders of others is by far the most pragmatic way to do it.