M3 Operating System Development

A chronicle of the development of the M3 operating system

Archive for the ‘Random’ Category

Modifications needed for M3 development under Windows

leave a comment »

For giggles, I downloaded the latest source code to my Windows box and attempted to build and run M3. It didn’t work. After some tinkering with build and linker settings, I have gotten my development environment working smoothly. I can now build and run M3 under Windows. I thought I would take a moment to explain the changes necessary to setup a M3 development environment on Windows.

Some assumptions:

  • You have installed cygwin, and the proper packages (nasm, gcc, binutils, etc).
  • You have installed Qemu and it is on the system path.

Once you’ve met these assumptions, make the following modifications to the M3 source files:

boot/init.asm :

  • you need to prefix an underscore to the extern symbol init_main. It should be _init_main, since gcc under cygwin prefixes function names with an underscore.

link.ld :

  • change OUTPUT_FORMAT from elf32-i386 to pe-i386. This is the default output format supported by the cygwin binutils.
  • change .rodata to .rdata . For some reason, this section is named differently under cygwin. If you don’t make this change, your code will still compile, but you won’t see any output to the screen, since the strings defined in your code won’t be linked into the binary.

Makefile :

  • change CFLAGS:
    • remove -fno-stack-protector . This isn’t supported by gcc under cygwin.
  • change EMUFLAGS:
    • add -L “c:\path\to\biosfolder” . There should be a Bios folder underneath the folder you installed Qemu into. Provide the path to this folder. Otherwise Qemu will complain about not being able to load BIOS.
    • add -no-kqemu
  • change OBJCOPY flags:
    • add -I pe-i386   This tells objcopy that object files you want to convert to binary are currently in PE format.

Once you make these changes, you should be good to go. You should be able to build M3 without errors and run it in the Qemu emulator.

EDIT: 4/12/09 – Modified to reflect the changes made during a small code reorg.


Written by m3os

April 10, 2009 at 8:33 am

On the shoulders of others – standing or leaning?

leave a comment »

I’ve obviously been doing a lot of research recently, specifically into the entire booting and kernel loading process. Here are a few examples of such tutorials. But probably the most quoted and widely-used tutorial currently out there is the famous Bran’s Kernel Development Tutorial (bkerndev)

This tutorial is so successful because it is fairly well-written, and because it gently guides the reader through some fairly thorny tasks that are necessary when building an OS on the IA-32 architecture. The tutorial takes us through the booting process, through setting up protected mode, and then shows how to set up interrupt handling and basic I/O with hardware such as the keyboard. 

I haven’t done any number crunching, but I think it’s fair to assume that the publishing of Bran’s tutorial led to a bit of a population explosion in the OS Dev world. Since its release, there has been a proliferation of OS projects on OSdev.org and other similar sites. The reason for this is that many developers work their way through Bran’s tutorial, and having done this, start up their own OS projects.

In reading through the forums on osdev.org, I get the sense that new OS projects based on Bran’s tutorial are sorely frowned upon. Just recently, someone announced a new version of their OS, and they were promptly beat down for having done nothing but make a few changes to the bkerndev code. I can see how frustrating such projects must be for OS dev veterans, and such beatdowns are probably warranted in many cases, but I would like to take a moment to explore the pros and cons of “standing on the shoulders of those that have come before” as an approach to OS development.

The first time that I tried to develop M3, about 10 years ago, there was no Bran’s tutorial. There wasn’t anything like it. Information about boot sectors, and loading kernels, etc, was hard to come by. Besides the Intel manuals, there weren’t too many reliable sources of information for those interested in tinkering with OS dev. And in my case, the result of this lack of information was that I gave up. After getting a very rudimentary boot sector working, that basically just booted from an actual floppy disk and printed a welcome message to the screen in real mode, I gave up. As a one-man team, I didn’t have the time or resources to figure out how to set up the necessary scaffolding for my kernel. I was interested in memory management, and process scheduling, and file systems, but I never got anywhere near that point, because of the immense amount of work that needs to be done just to get the environment initialized and in a state such that control can be handed off to the kernel.

Fast forward to today. With Bran’s tutorial and others like it, I could use the code provided and build myself that necessary scaffolding in a matter of days, not months or years. Bran’s Kernel Dev tutorial does not teach you much of anything about developing the bulk of the kernel itself. Instead, it walks you through the environment setup that needs to be done to be at the point where you can really focus on kernel development. bkerndev is somewhat of a misnomer. It should be called bkernenvsetup or something like that.

Back to the forum beatdowns. If someone merely takes Brandon’s code, tweaks a line here or there, and tries to pass it off as his own OS, then absolutely he should be raked over the coals (or perhaps just ignored, if flamewars aren’t your MO). However, if someone is using such code as the means to an end, the end being that they get to start working on their actual kernel, which will run on top of all the Bran scaffolding, then I would argue that that’s a valid use of the code. In fact, Brandon himself says so, right in the introduction. He lists the steps that the tutorial will take, with the final step being:

12) …and the rest is up to you!

Using Bran’s code to get to the meat of kernel development is a legitimate and pragmatic approach to OS development. Such (re)usage of code is an example of standing on the shoulders of those that have come before, which has been a valid and acceptable (even encouraged) practice in the open source world for.. well, ever.

At this point (if you’ve actually read this far), you’re saying to yourself “Great, this guy is just trying to justify cutting corners in his OS project”. Not so. I’m not trying to do any such thing. I’ll admit that I have read Bran’s and many other tutorials, and have used them as reference implementations for some of my stuff, but I don’t plan to just copy the code and use it without having any understanding of what’s going on. Quite the opposite – I use such tutorials precisely so that I can understand what’s going on. And I think that’s the critical difference. By reading through the tuts, by going through the provided code line by line and really understanding what’s going on, you come to “own” the code. If you know the code back and forth, and you get to the point where you understand it as well as if you had written it yourself, then you are truly standing on the shoulders of those who have come before. They have helped to lift you up to a level of understanding. And that’s something great. That’s not something to beat down.

So is my plan with M3 to simply build on top of Bran’s environment? No – that’s not the plan. But I do plan to use such tutorials to give me a leg up, so that I have a fair chance of actually getting to the point where I’m working on the real meat of the kernel. I plan to use to the code as a reference implementation of the concepts explained in the Intel manuals and other reference books. Being able to read something in the manual, and then see some actual code that illustates the concept is a great way to learn and understand. 

I’d like to get to step 12 at some point in my lifetime, and standing on the shoulders of others is by far the most pragmatic way to do it.

Written by m3os

April 1, 2009 at 12:22 pm

Useful Linux Commands for Operating System Development

leave a comment »

As I was attempting to figure out what was going on with my C kernel and my “missing” strings. I happened upon some rather useful linux commands for examining and debugging object files, so I thought I would share them here.

nm – print names from object file

strings – dump ASCII strings from object file. It’s interesting to run this command on other binaries to see what they contain. This command gives you an excellent reason as to why you shouldn’t embed sensitive information in your application as plaintext. Seeing such text is a simple command away for anyone that might have malicious intent with your code.

xxd – dumps contents of a binary file – gives you a hexdump – shows you what address information can be found at within your binary.

objdump – displays information about object file.

All of these commands have many command-line options, which you can see if you click on the command names above. For me, these have already proved useful, because they allowed me to determine whether my “missing” strings were actually in my kernel binary or not. I can see them providing similar help in the future, as kernel development progresses.

Written by m3os

March 21, 2009 at 10:34 am

Posted in Random

Welcome to M3

leave a comment »

Welcome, dear reader, to the M3 Development Blog. This goal of this blog is to completely chronicle the development of the M3 operating system – a pet project of yours truly.

However, this blog isn’t just about M3. It will be about operating system development in general.

As I see it now, there will be several types of postings on this blog:

  • Status updates on M3 development. Brief updates of where the current project stands.
  • In-depth posts that detail specific pieces of M3. For example, I plan to describe the boot loader and kernel in detail, so that people that are interested in learning about such things can.
  • Tutorials related to operating system development.
  • Operating system development news. There are a ton of OS development projects going on out there.
  • Random posts about whatever, but I’ll try to keep it operating system-related. My musings on existing OSes, speculations on the future of operating systems, etc.

Again, I welcome you to the blog, and I hope that you will find it a useful resource.

Written by m3os

February 25, 2009 at 10:23 pm

Posted in Random