M3 Operating System Development

A chronicle of the development of the M3 operating system

Archive for March 2009

Tutorial – Building an i386-elf cross-compiler and binutils on OS X

with 2 comments

As I mentioned in my previous post, I managed to build a cross-compiler and binutils toolchain targeting i386-elf executables on OS X. While the process isn’t too difficult, there are a couple of gotchas that you should be aware of.

Before I start, I’d like to reference this tutorial for building an ARM cross-compiler on OS X. It provided a pretty good roadmap for me as I set out on my task.

OK – the first thing that you need to do is get the source code for both gcc and binutils. To do this, we’ll use a handy tool called wget. Unfortunately, OS X does not come with wget as one of its pre-installed apps, so we’ll have to install it. You can grab a copy of wget compiled for OS X here. Follow the instructions in the README to install it into the proper directories and add the proper folders to your PATH.

If you don’t want to install wget, OS X comes with a similar utility called curl. If you are comfortable using that tool, then adjust the wget commands below accordingly.

Now, drop to a command prompt and issue the following commands to download and extract the source code for binutils and gcc:

$ mkdir ~/crossgcc && cd ~/crossgcc
$ wget ftp://sourceware.org/pub/binutils/snapshots/binutils-2.18.50.tar.bz2
$ tar jxf binutils-2.18.50.tar.bz2
$ wget http://ftp.gnu.org/pub/gnu/gcc/gcc-4.2.3/gcc-4.2.3.tar.bz2
$ tar jxf gcc-4.2.3.tar.bz2

I've chosen a target directory of  /usr/local/i386elfgcc

We’ll have to create that directory and make it world writable for now:

$ sudo mkdir /usr/local/i386elfgcc
$ sudo chmod 777 /usr/local/i386elfgcc

First we build the binutils:

$ mkdir build-binutils && cd build-binutils
$ ../binutils-2.18.50/configure --target=i386-elf 
--prefix=/usr/local/i386elfgcc/ 2>&1 | tee configure.log
$ make all install 2>&1 | tee make.log

The compiler will do its thing for a few minutes. When it’s done, your cross-compiled binutils will be installed in the bin folder of the target folder.

Now we'll build the gcc compiler with C support. This is someplace that we need to specialize the configuration for OS X. Be sure to include the parameter --disable-libssp when you are running configure for gcc. Otherwise, your build will blow up with errors about libssp.

$ cd ../gcc-4.2.3
$ cd ..
$ mkdir build-gcc && cd build-gcc
$ ../gcc-4.2.3/configure --target=i386-elf 
--prefix=/usr/local/i386elfgcc/   
--with-gnu-as --with-gnu-ld --disable-libssp --enable-languages=c 2>&1 | tee configure.log
$ make all install 2>&1 | tee make.log

The compiler will again do its thing for a few minutes. When it is done, your cross-compiled gcc will be installed in the bin folder of the target folder.

All the cross-compiled tools in our dev toolchain are now installed into

/usr/local/i386elfgcc/bin

Because they are cross-compiled versions, they have different names from the standard tools, to avoid naming conflicts. For example, our cross-compiled gcc is called i386-elf-gcc. Basically, all the tools have the prefix i386-gcc.

If you’d like these tools to be generally accessible, you can add the target path to your PATH, in .bash_profile or a similar shell configuration file.

You’ve now got a complete cross-compiled development toolchain for targeting the IA-32 platform on OS X. Pat yourself on the back, and if you’re feeling especially amazed and encouraged by the many excellent free development tools available with source, show your support by donating to the Free Software Foundation.

Advertisements

Written by m3os

March 29, 2009 at 1:57 pm

False Start – Setting up for development on OS X

leave a comment »

I mentioned in an earlier post that I had switched over to my EeePC once I started work on my C kernel. I was forced to switch because the development tools that come with OS X do not include support for ELF binaries, and many of the GNU development tools are not available. The version of gcc that is installed only supports the creation of Mach-O executable files, and I want to target ELF binaries, so that I can enjoy the benefit of full usage of the GNU toolchain.

In that previous post, I had mentioned that rather than go through the hassle of building a cross-compiler on my Mac that would target i386-elf binaries, I preferred to keep my development momentum going and just switch over to develop under Linux on my EeePC.

Developing on the EeePC is fine, very portable and handy. However, I often wish I could work on M3 on my Mac, with its full-size keyboard and bigger screen. So I did some research today into exactly what it would take to build a cross-compiler on OS X.

What’s a cross-compiler, you ask? Good question! A cross-compiler is a compiler that generates executables that target another platform. In my case, I would like to target ELF binaries that run on the IA-32 architecture. The flavor of Unix at the heart of OS X is called Darwin, and it does not currently support the ELF executable format. Lucky for me (and for the development world), the GNU project has built their most-excellent C compiler to be very portable, and to support a wide variety of target architectures and executable formats. So what I am able to do is to compile a version of the gcc compiler that targets ELF binaries on the i386 architecture. Once I have a cross compiler, I will be able to compile ELF binaries on OS X, but obviously will not be able to execute them on OS X. That’s perfectly OK though, since I don’t want to execute them on OS X, I want to execute them on my IA-32 system emulated by Q(emu).

There is a project out there on the interwebs that is called DarwinPorts. Essentially, it’s a repository for tools that have been built for Darwin, with a nice client application that manages dependencies and handles downloading, building and installing the software for you. Easy! And hey, they have packages for a cross-compiler toolchain. Great! I figured I would install binutils and gcc and be off to the races.

Unfortunately, things didn’t quite work out that way. There is some problem in the binutils package that was causing the compile to error out. After thoroughly googling the issue and attempting several fixes over a couple of hours, I threw in the towel. As nice as DarwinPorts is, it didn’t work out for me in this case. 

I wasn’t ready to throw in the towel on building a cross-compiler on OS X yet though. I just had to roll up my sleeves, get the source code for the packages that I needed, and build the tools myself. After a bit of work, I got everything built.

The first litmus test of course was to run my make script and have the cross-compiler build my binaries. Yes! They compiled!

Then I needed to test that the binutils tools that I used would work with my binary file, and create a floppy disk image properly. Yes! It looks like everything worked OK, and I’ve got a floppy image sitting in my folder.

Then for the big test – see if the floppy image will boot in Q… Ding! Ding! Ding! We have a winner!

Phew. In the next post I will describe exactly how I built the toolchain, because while it’s not too difficult, it’s not a walk in the park either. The end result is totally worth it though – now I can continue development of M3 on my Mac!

Written by m3os

March 29, 2009 at 9:56 am

Posted in Status Update

Rough plan of action for the near future

leave a comment »

This might be completely off, and is completely subject to change by the minute, but here’s my current thinking about what to work on in the near future. I’d like to keep development rolling, while I read up and do some research into some of the major pieces of the kernel such as memory management, process management, etc.

  • Interrupts – set up an Interrupt Descriptor Table (IDT) and write some Interrupt Service Routines (ISRs) so that the kernel can be responsive to certain important interrupts, such as clock ticks, page faults, etc.
  • Write a driver that handles the keyboard.
  • Write some test routines. I’d like to build as many tests as possible along the way, so that I know that everything is working exactly how I think it is.

I think that a two-pronged approach to this project will keep the momentum going. When I feel like coding, I will. When I don’t feel up to coding, I’ll be reading and researching and planning.

Written by m3os

March 28, 2009 at 8:50 am

Posted in Status Update

What’s in a Name – M3

leave a comment »

I thought I’d take a bit of time to explain where the name M3 comes from, just in case anyone is curious.

I first became interested in OS design and development about twelve years ago, in 1997. Being a recent graduate of college, a newlywed, and just plain young, I got the idea that I could write my own OS. Call it youthful optimism, call it hubris, call it insanity if you want, but that was my thinking at the time.

Of course, one of the most important things that one must do when writing an OS is to come up with a good name. A good name is way more important than, say, actual code. One trend at the time was to incorporate the suffix “ix” or “ux” to the name of your OS, if it in any way resembled or derived from Unix. Another trend was to work the letters “os” into the name, such as BeOS, or FreeDOS. I wanted to cut my own path, and stay away from such trends. I was a rebel, a renegade, a trailblazer.

In 1997, the new millennium was starting to come into view. Seeing as I was setting off to build an operating system that would serve as a platform for the new millenium, nay, the platform for the new millenium, I wanted to capture that idea in the name. After kicking around names like MilleniOS and Milleniux, I settled on the simple yet solid M3.

M3 – the operating system for the third millenium.

Well, M3 didn’t really get too far down the path to development back in 1997, but when I decided to restart my efforts in 2009, I kept the name. Seeing as we’re not too far into the 3rd millennium, it’s still applicable. Besides, I can’t think of anything better.

Plus, there’s all sorts of clever things that I could do to play off the name, if I were a clever marketer. M3 could be three things that start with M, such as Modular Modern Microkernel, or Multitasking Modular Microkernel, or well, you get the idea. But before I get clever with the marketing, I’d like to have a working product. Otherwise, M3 will just be another word for vaporware.

Written by m3os

March 23, 2009 at 12:44 pm

Posted in Personal

Useful Linux Commands for Operating System Development

leave a comment »

As I was attempting to figure out what was going on with my C kernel and my “missing” strings. I happened upon some rather useful linux commands for examining and debugging object files, so I thought I would share them here.

nm – print names from object file

strings – dump ASCII strings from object file. It’s interesting to run this command on other binaries to see what they contain. This command gives you an excellent reason as to why you shouldn’t embed sensitive information in your application as plaintext. Seeing such text is a simple command away for anyone that might have malicious intent with your code.

xxd – dumps contents of a binary file – gives you a hexdump – shows you what address information can be found at within your binary.

objdump – displays information about object file.

All of these commands have many command-line options, which you can see if you click on the command names above. For me, these have already proved useful, because they allowed me to determine whether my “missing” strings were actually in my kernel binary or not. I can see them providing similar help in the future, as kernel development progresses.

Written by m3os

March 21, 2009 at 10:34 am

Posted in Random

Linker woes and missing strings

leave a comment »

Boy – getting my print function to work took some doing. I wrote the initial version of this function last night, and ran it to no avail. My clearscreen function was clearing the screen, and that’s it. My strings were not getting printed to the screen by my print function.

Poking around on OSDev.org led me to believe that there was a problem with my rodata section (for read-only data, I suppose), which is the section where string constants reside. I was declaring the following string constant in my code:

char *welcome = "Welcome to M3\nLine 2 here\n";

Which should put the string between the quotes into the rodata section (along with a terminating null char).

The first place that I checked was my linker script. I noticed that I was aligning everything to start at 0x1000. I’m not sure why I did this – probably had something to do with the fact that controlling the linker is completely new to me. My guess is that this had the effect of putting all the sections on top of each other.

I changed the linker script to put rodata right after the text section, and aligned the data section to start on the next page boundary, so it’s well out of the way. After linking the file with this modified script, I could see that things were no longer being dumped on top of each other. The -M switch to the ld command was helpful here, as was the trace switch (-t). Here’s the command I was using:

ld -T link.ld -M -t -o kernel.o start.o main.o

Executing this binary yielded something at the top of the screen – several seemingly miscellaneous characters on the first line, and couple of miscellaneous characters on the second line. This was definitely better, but not the desired goal.

Then I looked at my C code, and yeah, I’m definitely rusty. I guess it was kind of late when I wrote the first version of this code, because it was just plain wrong. I wasn’t iterating through my string properly – for some reason I was jumping two characters at a time. Bah. I’ll just chalk it up to the late hour.

After de-stupiding my C code, I rebuilt and re-ran, and voila! I saw my two-line string printed nicely at the top of my blank screen. Excellent.

Baby steps…

Written by m3os

March 21, 2009 at 10:27 am

Posted in Uncategorized

Plan for the kernel – baby steps

leave a comment »

Now that I have successfully booted into my very rudimentary C kernel, I can start working to expand that into something that is truly functional.

I’m not just going to jump right into the heavy stuff yet though. Instead, I will ease myself into the shallow waters, and once I’m comfortable I’ll wade into greater depths. It’s been many years since I wrote C code, and even back then I didn’t really get into anything too difficult. Therefore, the first order of business is to refamiliarize myself with programming in C. 

I’ve spent the last couple of days reading up, and I’m ready to start writing some code.

I’ll start exercising my C muscles by coding up some routines that will be immediately useful as I start to build out the rest of the kernel. First up is a clearscreen routine, which will clear the entire screen. Next up will be a print routine, a function that will print a null-terminated string to the screen at the current cursor position. That means it will also need to track the current cursor position. I’ll also need to add cursor handling into the clearscreen function, which will reset the screen position to line 0, position 0.

From there, I plan to write several functions that will assist in debugging, by dumping the contents of the registers to the screen, or the contents of a certain section of memory, etc. Whatever I think will be useful for debugging purposes as I start to tread into the gnarly areas of paging, memory management, etc.

Written by m3os

March 20, 2009 at 7:18 pm

Posted in Status Update

Tagged with , ,