M3 Operating System Development

A chronicle of the development of the M3 operating system

Archive for February 2009

Instant Gratification – Building a Simple Bootloader

leave a comment »

I am all about instant gratification. Perhaps that’s a bad personality quirk for someone that has set out to write his own operating system, but hey, I’m me, and I can’t be anything else.

For someone that has just set out on the long road to building an OS, there is nothing more thrilling than seeing your code boot up for the first time. I can’t describe it – it’s just a total rush to hand-craft some assembly code, build a binary, write that binary to the boot sector and see your computer boot up and be completely under the control of your code.

In addition to satisfying my need for instant gratification, this exercise allowed me to start to become familiar with the various tools in my toolbox. When you’re coding this close to the metal, it is important to be very familiar with your tools.

So, without further ado, here it is – the very first iteration of M3’s boot loader, running on Q (QEmu on OS X):

M3 Boot Loader on Q Emulator (OS X)

M3 Boot Loader on Q Emulator (OS X)

In the next post, I will describe the steps that I took to do this.

Ahhh – instantly gratifying!

Written by m3os

February 28, 2009 at 12:52 pm

Posted in Status Update

Development Environment for M3 Operating System Development

leave a comment »

One of the first things that an operating system developer must do is decide on a development environment and a set of development tools. In this post, I will describe the development environment that I plan to use for M3 development.

In 2001, I bought my first Mac. I was intrigued by OS X. A really nice GUI built on top of rock-solid UNIX sounded like a winning combination, and oh boy is it a rockin’ combination! Since then, I have not purchased another PC. In recent years, many other developers have followed suit, and Macs have become quite a popular development platform. In my current job, I develop ASP.Net websites using a MacBook Pro running VMWare Fusion. I’ve got a virtual Windows 2003 server instance that I do my .Net development on, but for everything else, I use native OS X software. It’s a sweet setup.

So I decided right off the bat that my development environment would be OS X. Because it’s UNIX, there are a zillion dev tools available, many of them bundled with OS X on the included developer tools CD.

I also mentioned VMs. I decided that my life would be a whole lot easier if I ran M3 on a VM or an emulator during development, rather than try to run it on the actual target hardware. When I first started getting into OS development, about 10 years ago now, I remember having to write the boot loader and kernel to an actual 3.5″ floppy disk. To test, I would need to boot the computer off the floppy. If it didn’t work (as was often the case), I would need to figure out what went wrong, fix the code, compile, write the code to the floppy, and reboot the computer. Not an optimal development cycle by any stretch of the imagination.

Today, my Mac doesn’t even have a floppy drive. I haven’t bought or owned a floppy disk in years. That fact, combined with the painful floppy development cycle had me looking for more optimal ways to test operating system code. And I decided that what I needed was an emulator.

Right now, there seem to be two popular choices for IA32 emulation: Bochs and QEMU.

Bochs is an open-source IA32 emulator that runs on a variety of platforms. It is well-maintained, and there is plenty of help available out there if you need it.

QEMU is another open-source emulator that runs on a variety of platforms. For OS X, there is the Q project, which provides a nice Cocoa front-end to QEMU.

I haven’t yet decided which emulator I’ll use for my testing. I’m currently evaluating them to determine which will better suit my needs. 

Coding Tools

I’m getting a bit ahead of myself though. Before I can get to the testing stage, I need to have actually written and built some code.

The core M3 kernel will be written in C. Perhaps some of the other pieces will be written in a higher-level language, but the core – the task schedule, the memory manager, etc – will be written in C.

For C programming, I’m not going to look any further than the GNU C Compiler (gcc). It’s not even up for discussion. gcc is an excellent compiler, and available on a zillion platforms. There’s copious documentation, and it’s open-source. 

M3 will need to have certain parts written in assembly, primarily the boot loader. For assembly code, I’m currently evaluating two options: GNU Assembler (gas) and NASM.

GNU Assembler is the back-end assembler used by gcc. The benefit of using gas would be that I would remain within the GNU toolset. However, gas uses a different syntax than a lot of other assemblers (AT&T syntax as opposed to Intel), and I think that might make it more difficult for me to incorporate sample assembly code available on the Internet. I don’t want to have to translate from Intel to AT&T syntax.

Nasm – the Netwide Assembler is an open-source assembler project that is portable across many platforms and supports a wide variety of output formats. It is currently a very popular choice for assembly programming. 

I’m currently leaning towards using Nasm. I just like the assembly syntax a lot better.

Other Tools

For my text editor, I’ll most likely be using TextMate. It’s not free, but it’s well worth the money.

I’ll use make to script the build/linking process and make that easier. I don’t want to sit there and type a bunch of commands everytime I want to build M3. I’d rather just type

make m3

and have make do all the heavy lifting. Yeah, I’m lazy like that.

If I come across other tools that I find useful, I will be sure to note them in the blog.

Written by m3os

February 27, 2009 at 10:37 am

Develop an Operating System – Am I Crazy?

leave a comment »

It is no small task to build an operating system. In fact, I often question my own sanity when contemplating the task before me. Why on Earth would I want to build my own operating system, and why would I try to do such a thing myself?

My interest in operating systems started in my junior year of college, when I took a course on operating systems as part of my Computer Science degree. From the start, I was totally into the internal workings of the kernel – schedulers, task management, memory management – it was really cool stuff. We wrote a couple simple programs in C++ that would handle different responsibilities, such as a round-robin scheduler, or a “memory manager” that was basically an implementation of a stack, etc. However, we never got to tie all the stuff together into an actual operating system. The course was more about the concepts than actually putting together an OS. 

The next semester, I took a course in computer architecture, which took me down to the hardware level. I learned about CPUs and what was inside them, clock timing, system busses, RAM, everything. There was a lab that went along with this class, and while the class was cool, the lab was really cool. The lab was where we got to actually put our knowledge into practice. 

In the lab, we would all be given breadboards, wires, and various CPUs or other logic chips. First, we would have to figure out how to wire things together to get the system that we wanted. Then we actually hooked the breadboard up to a running computer and wrote simple programs to run on our boards. The chips were 8086s, so we used 8086 assembly language to program the things. Up until this point, I’d worked only with high-level languages like C/C++ and Pascal, and assembly language just blew my mind. Now I needed to worry about the individual bits, and needed to be familiar with the processor itself to program it successfully. Others got frustrated, but I fell in love.

Once I graduated, I took a couple of courses in grad school, including an operating systems course. Once I was done with that, I continued to research and learn about operating systems on my own. It’s funny – when I was in school, I barely cracked open the textbooks. But once I graduated, I found myself buying textbooks on operating system design and construction and actually reading them for fun!

So why would I want to write one myself? Well, I think the idea of having total control of the hardware is pretty cool. When you’re writing an OS, it’s just your code going directly to the metal. And to be able to completely organize and manage the allocation of that hardware and those resources is just a really interesting problem to try to solve. I’ve never thought of myself as a control freak, but perhaps in this regard I am a little bit. The system is MIIINE, completely under my control – Mwuhahahah!

OK – I feel better now. 🙂

I think that the size and complexity of the problem is also part of the draw. If some day, years down the road, I get a solid kernel up and running, and applications (however simple) executing on my OS, I’ll feel a great sense of accomplishment. In my day job, I write business applications – and really, where’s the excitement in that? Building an operating system though, well, that’s for the hard-core developers. If I build my own OS, I’ll be part of a very small club. A true hacker.

Why try to do this myself? Well, it is going to be a hobby for me, a pet project. I’m going to write all the code myself because I want to learn all the gritty details. I love to learn, and I’m treating this project as a giant learning opportunity. If someday I get it to a point where there’s something bootable and somewhat stable, perhaps I will invite other programmers to join the fray. 

But just because I’m planning to write the code myself doesn’t mean that I’m doing it alone. There’s a pretty good community of “os-devers” out there, and most are willing to lend a hand with advice or problem-solving or both. I kind of liken it to the phenomenon of NaNoWriMo – National Novel Writing Month. The point of NaNo is to write a 50,000-word novel in a month. A daunting task, no? Well, I participated in NaNo in 2005, and the best part about the project was the supportive community of fellow writers who were all toiling along beside me, sharing the same trials and tribulations. The OS Developer community is very much like that.

I plan on being an active participant in the OS Dev community. I’m not going to be afraid to ask questions, and I will answer other people’s questions when I can. And I’ll document the entire process on this blog, so that other people can learn from my mistakes and successes, and perhaps offer me guidance or encouragement when I need it.

So am I crazy? At this point, I would have to say – nope. I understand the monumental task before me, and I’m going into it with realistic expectations. I’m doing it because I enjoy the work, and the learning, and the challenge.

Am I a little intimidated? You betcha! I would be crazy if I wasn’t.

Am I excited? Again, you betcha! I’m totally looked forward to the adventure that developing M3 will be. I hope that you’ll stick around for the ride.

Written by m3os

February 26, 2009 at 7:20 pm

Posted in Personal

Welcome to M3

leave a comment »

Welcome, dear reader, to the M3 Development Blog. This goal of this blog is to completely chronicle the development of the M3 operating system – a pet project of yours truly.

However, this blog isn’t just about M3. It will be about operating system development in general.

As I see it now, there will be several types of postings on this blog:

  • Status updates on M3 development. Brief updates of where the current project stands.
  • In-depth posts that detail specific pieces of M3. For example, I plan to describe the boot loader and kernel in detail, so that people that are interested in learning about such things can.
  • Tutorials related to operating system development.
  • Operating system development news. There are a ton of OS development projects going on out there.
  • Random posts about whatever, but I’ll try to keep it operating system-related. My musings on existing OSes, speculations on the future of operating systems, etc.

Again, I welcome you to the blog, and I hope that you will find it a useful resource.

Written by m3os

February 25, 2009 at 10:23 pm

Posted in Random