M3 Operating System Development

A chronicle of the development of the M3 operating system

Archive for the ‘Personal’ Category

On the shoulders of others – standing or leaning?

leave a comment »

I’ve obviously been doing a lot of research recently, specifically into the entire booting and kernel loading process. Here are a few examples of such tutorials. But probably the most quoted and widely-used tutorial currently out there is the famous Bran’s Kernel Development Tutorial (bkerndev)

This tutorial is so successful because it is fairly well-written, and because it gently guides the reader through some fairly thorny tasks that are necessary when building an OS on the IA-32 architecture. The tutorial takes us through the booting process, through setting up protected mode, and then shows how to set up interrupt handling and basic I/O with hardware such as the keyboard. 

I haven’t done any number crunching, but I think it’s fair to assume that the publishing of Bran’s tutorial led to a bit of a population explosion in the OS Dev world. Since its release, there has been a proliferation of OS projects on OSdev.org and other similar sites. The reason for this is that many developers work their way through Bran’s tutorial, and having done this, start up their own OS projects.

In reading through the forums on osdev.org, I get the sense that new OS projects based on Bran’s tutorial are sorely frowned upon. Just recently, someone announced a new version of their OS, and they were promptly beat down for having done nothing but make a few changes to the bkerndev code. I can see how frustrating such projects must be for OS dev veterans, and such beatdowns are probably warranted in many cases, but I would like to take a moment to explore the pros and cons of “standing on the shoulders of those that have come before” as an approach to OS development.

The first time that I tried to develop M3, about 10 years ago, there was no Bran’s tutorial. There wasn’t anything like it. Information about boot sectors, and loading kernels, etc, was hard to come by. Besides the Intel manuals, there weren’t too many reliable sources of information for those interested in tinkering with OS dev. And in my case, the result of this lack of information was that I gave up. After getting a very rudimentary boot sector working, that basically just booted from an actual floppy disk and printed a welcome message to the screen in real mode, I gave up. As a one-man team, I didn’t have the time or resources to figure out how to set up the necessary scaffolding for my kernel. I was interested in memory management, and process scheduling, and file systems, but I never got anywhere near that point, because of the immense amount of work that needs to be done just to get the environment initialized and in a state such that control can be handed off to the kernel.

Fast forward to today. With Bran’s tutorial and others like it, I could use the code provided and build myself that necessary scaffolding in a matter of days, not months or years. Bran’s Kernel Dev tutorial does not teach you much of anything about developing the bulk of the kernel itself. Instead, it walks you through the environment setup that needs to be done to be at the point where you can really focus on kernel development. bkerndev is somewhat of a misnomer. It should be called bkernenvsetup or something like that.

Back to the forum beatdowns. If someone merely takes Brandon’s code, tweaks a line here or there, and tries to pass it off as his own OS, then absolutely he should be raked over the coals (or perhaps just ignored, if flamewars aren’t your MO). However, if someone is using such code as the means to an end, the end being that they get to start working on their actual kernel, which will run on top of all the Bran scaffolding, then I would argue that that’s a valid use of the code. In fact, Brandon himself says so, right in the introduction. He lists the steps that the tutorial will take, with the final step being:

12) …and the rest is up to you!

Using Bran’s code to get to the meat of kernel development is a legitimate and pragmatic approach to OS development. Such (re)usage of code is an example of standing on the shoulders of those that have come before, which has been a valid and acceptable (even encouraged) practice in the open source world for.. well, ever.

At this point (if you’ve actually read this far), you’re saying to yourself “Great, this guy is just trying to justify cutting corners in his OS project”. Not so. I’m not trying to do any such thing. I’ll admit that I have read Bran’s and many other tutorials, and have used them as reference implementations for some of my stuff, but I don’t plan to just copy the code and use it without having any understanding of what’s going on. Quite the opposite – I use such tutorials precisely so that I can understand what’s going on. And I think that’s the critical difference. By reading through the tuts, by going through the provided code line by line and really understanding what’s going on, you come to “own” the code. If you know the code back and forth, and you get to the point where you understand it as well as if you had written it yourself, then you are truly standing on the shoulders of those who have come before. They have helped to lift you up to a level of understanding. And that’s something great. That’s not something to beat down.

So is my plan with M3 to simply build on top of Bran’s environment? No – that’s not the plan. But I do plan to use such tutorials to give me a leg up, so that I have a fair chance of actually getting to the point where I’m working on the real meat of the kernel. I plan to use to the code as a reference implementation of the concepts explained in the Intel manuals and other reference books. Being able to read something in the manual, and then see some actual code that illustates the concept is a great way to learn and understand. 

I’d like to get to step 12 at some point in my lifetime, and standing on the shoulders of others is by far the most pragmatic way to do it.

Written by m3os

April 1, 2009 at 12:22 pm

What’s in a Name – M3

leave a comment »

I thought I’d take a bit of time to explain where the name M3 comes from, just in case anyone is curious.

I first became interested in OS design and development about twelve years ago, in 1997. Being a recent graduate of college, a newlywed, and just plain young, I got the idea that I could write my own OS. Call it youthful optimism, call it hubris, call it insanity if you want, but that was my thinking at the time.

Of course, one of the most important things that one must do when writing an OS is to come up with a good name. A good name is way more important than, say, actual code. One trend at the time was to incorporate the suffix “ix” or “ux” to the name of your OS, if it in any way resembled or derived from Unix. Another trend was to work the letters “os” into the name, such as BeOS, or FreeDOS. I wanted to cut my own path, and stay away from such trends. I was a rebel, a renegade, a trailblazer.

In 1997, the new millennium was starting to come into view. Seeing as I was setting off to build an operating system that would serve as a platform for the new millenium, nay, the platform for the new millenium, I wanted to capture that idea in the name. After kicking around names like MilleniOS and Milleniux, I settled on the simple yet solid M3.

M3 – the operating system for the third millenium.

Well, M3 didn’t really get too far down the path to development back in 1997, but when I decided to restart my efforts in 2009, I kept the name. Seeing as we’re not too far into the 3rd millennium, it’s still applicable. Besides, I can’t think of anything better.

Plus, there’s all sorts of clever things that I could do to play off the name, if I were a clever marketer. M3 could be three things that start with M, such as Modular Modern Microkernel, or Multitasking Modular Microkernel, or well, you get the idea. But before I get clever with the marketing, I’d like to have a working product. Otherwise, M3 will just be another word for vaporware.

Written by m3os

March 23, 2009 at 12:44 pm

Posted in Personal

Develop an Operating System – Am I Crazy?

leave a comment »

It is no small task to build an operating system. In fact, I often question my own sanity when contemplating the task before me. Why on Earth would I want to build my own operating system, and why would I try to do such a thing myself?

My interest in operating systems started in my junior year of college, when I took a course on operating systems as part of my Computer Science degree. From the start, I was totally into the internal workings of the kernel – schedulers, task management, memory management – it was really cool stuff. We wrote a couple simple programs in C++ that would handle different responsibilities, such as a round-robin scheduler, or a “memory manager” that was basically an implementation of a stack, etc. However, we never got to tie all the stuff together into an actual operating system. The course was more about the concepts than actually putting together an OS. 

The next semester, I took a course in computer architecture, which took me down to the hardware level. I learned about CPUs and what was inside them, clock timing, system busses, RAM, everything. There was a lab that went along with this class, and while the class was cool, the lab was really cool. The lab was where we got to actually put our knowledge into practice. 

In the lab, we would all be given breadboards, wires, and various CPUs or other logic chips. First, we would have to figure out how to wire things together to get the system that we wanted. Then we actually hooked the breadboard up to a running computer and wrote simple programs to run on our boards. The chips were 8086s, so we used 8086 assembly language to program the things. Up until this point, I’d worked only with high-level languages like C/C++ and Pascal, and assembly language just blew my mind. Now I needed to worry about the individual bits, and needed to be familiar with the processor itself to program it successfully. Others got frustrated, but I fell in love.

Once I graduated, I took a couple of courses in grad school, including an operating systems course. Once I was done with that, I continued to research and learn about operating systems on my own. It’s funny – when I was in school, I barely cracked open the textbooks. But once I graduated, I found myself buying textbooks on operating system design and construction and actually reading them for fun!

So why would I want to write one myself? Well, I think the idea of having total control of the hardware is pretty cool. When you’re writing an OS, it’s just your code going directly to the metal. And to be able to completely organize and manage the allocation of that hardware and those resources is just a really interesting problem to try to solve. I’ve never thought of myself as a control freak, but perhaps in this regard I am a little bit. The system is MIIINE, completely under my control – Mwuhahahah!

OK – I feel better now. 🙂

I think that the size and complexity of the problem is also part of the draw. If some day, years down the road, I get a solid kernel up and running, and applications (however simple) executing on my OS, I’ll feel a great sense of accomplishment. In my day job, I write business applications – and really, where’s the excitement in that? Building an operating system though, well, that’s for the hard-core developers. If I build my own OS, I’ll be part of a very small club. A true hacker.

Why try to do this myself? Well, it is going to be a hobby for me, a pet project. I’m going to write all the code myself because I want to learn all the gritty details. I love to learn, and I’m treating this project as a giant learning opportunity. If someday I get it to a point where there’s something bootable and somewhat stable, perhaps I will invite other programmers to join the fray. 

But just because I’m planning to write the code myself doesn’t mean that I’m doing it alone. There’s a pretty good community of “os-devers” out there, and most are willing to lend a hand with advice or problem-solving or both. I kind of liken it to the phenomenon of NaNoWriMo – National Novel Writing Month. The point of NaNo is to write a 50,000-word novel in a month. A daunting task, no? Well, I participated in NaNo in 2005, and the best part about the project was the supportive community of fellow writers who were all toiling along beside me, sharing the same trials and tribulations. The OS Developer community is very much like that.

I plan on being an active participant in the OS Dev community. I’m not going to be afraid to ask questions, and I will answer other people’s questions when I can. And I’ll document the entire process on this blog, so that other people can learn from my mistakes and successes, and perhaps offer me guidance or encouragement when I need it.

So am I crazy? At this point, I would have to say – nope. I understand the monumental task before me, and I’m going into it with realistic expectations. I’m doing it because I enjoy the work, and the learning, and the challenge.

Am I a little intimidated? You betcha! I would be crazy if I wasn’t.

Am I excited? Again, you betcha! I’m totally looked forward to the adventure that developing M3 will be. I hope that you’ll stick around for the ride.

Written by m3os

February 26, 2009 at 7:20 pm

Posted in Personal