M3 Operating System Development

A chronicle of the development of the M3 operating system

Archive for the ‘Tutorial’ Category

Modifications needed for M3 development under Windows

leave a comment »

For giggles, I downloaded the latest source code to my Windows box and attempted to build and run M3. It didn’t work. After some tinkering with build and linker settings, I have gotten my development environment working smoothly. I can now build and run M3 under Windows. I thought I would take a moment to explain the changes necessary to setup a M3 development environment on Windows.

Some assumptions:

  • You have installed cygwin, and the proper packages (nasm, gcc, binutils, etc).
  • You have installed Qemu and it is on the system path.

Once you’ve met these assumptions, make the following modifications to the M3 source files:

boot/init.asm :

  • you need to prefix an underscore to the extern symbol init_main. It should be _init_main, since gcc under cygwin prefixes function names with an underscore.

link.ld :

  • change OUTPUT_FORMAT from elf32-i386 to pe-i386. This is the default output format supported by the cygwin binutils.
  • change .rodata to .rdata . For some reason, this section is named differently under cygwin. If you don’t make this change, your code will still compile, but you won’t see any output to the screen, since the strings defined in your code won’t be linked into the binary.

Makefile :

  • change CFLAGS:
    • remove -fno-stack-protector . This isn’t supported by gcc under cygwin.
  • change EMUFLAGS:
    • add -L “c:\path\to\biosfolder” . There should be a Bios folder underneath the folder you installed Qemu into. Provide the path to this folder. Otherwise Qemu will complain about not being able to load BIOS.
    • add -no-kqemu
  • change OBJCOPY flags:
    • add -I pe-i386   This tells objcopy that object files you want to convert to binary are currently in PE format.

Once you make these changes, you should be good to go. You should be able to build M3 without errors and run it in the Qemu emulator.

EDIT: 4/12/09 – Modified to reflect the changes made during a small code reorg.

Advertisements

Written by m3os

April 10, 2009 at 8:33 am

Tutorial – Building an i386-elf cross-compiler and binutils on OS X

with 2 comments

As I mentioned in my previous post, I managed to build a cross-compiler and binutils toolchain targeting i386-elf executables on OS X. While the process isn’t too difficult, there are a couple of gotchas that you should be aware of.

Before I start, I’d like to reference this tutorial for building an ARM cross-compiler on OS X. It provided a pretty good roadmap for me as I set out on my task.

OK – the first thing that you need to do is get the source code for both gcc and binutils. To do this, we’ll use a handy tool called wget. Unfortunately, OS X does not come with wget as one of its pre-installed apps, so we’ll have to install it. You can grab a copy of wget compiled for OS X here. Follow the instructions in the README to install it into the proper directories and add the proper folders to your PATH.

If you don’t want to install wget, OS X comes with a similar utility called curl. If you are comfortable using that tool, then adjust the wget commands below accordingly.

Now, drop to a command prompt and issue the following commands to download and extract the source code for binutils and gcc:

$ mkdir ~/crossgcc && cd ~/crossgcc
$ wget ftp://sourceware.org/pub/binutils/snapshots/binutils-2.18.50.tar.bz2
$ tar jxf binutils-2.18.50.tar.bz2
$ wget http://ftp.gnu.org/pub/gnu/gcc/gcc-4.2.3/gcc-4.2.3.tar.bz2
$ tar jxf gcc-4.2.3.tar.bz2

I've chosen a target directory of  /usr/local/i386elfgcc

We’ll have to create that directory and make it world writable for now:

$ sudo mkdir /usr/local/i386elfgcc
$ sudo chmod 777 /usr/local/i386elfgcc

First we build the binutils:

$ mkdir build-binutils && cd build-binutils
$ ../binutils-2.18.50/configure --target=i386-elf 
--prefix=/usr/local/i386elfgcc/ 2>&1 | tee configure.log
$ make all install 2>&1 | tee make.log

The compiler will do its thing for a few minutes. When it’s done, your cross-compiled binutils will be installed in the bin folder of the target folder.

Now we'll build the gcc compiler with C support. This is someplace that we need to specialize the configuration for OS X. Be sure to include the parameter --disable-libssp when you are running configure for gcc. Otherwise, your build will blow up with errors about libssp.

$ cd ../gcc-4.2.3
$ cd ..
$ mkdir build-gcc && cd build-gcc
$ ../gcc-4.2.3/configure --target=i386-elf 
--prefix=/usr/local/i386elfgcc/   
--with-gnu-as --with-gnu-ld --disable-libssp --enable-languages=c 2>&1 | tee configure.log
$ make all install 2>&1 | tee make.log

The compiler will again do its thing for a few minutes. When it is done, your cross-compiled gcc will be installed in the bin folder of the target folder.

All the cross-compiled tools in our dev toolchain are now installed into

/usr/local/i386elfgcc/bin

Because they are cross-compiled versions, they have different names from the standard tools, to avoid naming conflicts. For example, our cross-compiled gcc is called i386-elf-gcc. Basically, all the tools have the prefix i386-gcc.

If you’d like these tools to be generally accessible, you can add the target path to your PATH, in .bash_profile or a similar shell configuration file.

You’ve now got a complete cross-compiled development toolchain for targeting the IA-32 platform on OS X. Pat yourself on the back, and if you’re feeling especially amazed and encouraged by the many excellent free development tools available with source, show your support by donating to the Free Software Foundation.

Written by m3os

March 29, 2009 at 1:57 pm

Tutorial: Setting up an OS development environment on the EeePC (running Xandros)

leave a comment »

As I said in a previous post, I abandoned my OS X development environment for the time being, since the dev tools that come with OS X only support Mach-O binaries, and don’t have utilities such as objcopy. I know that there is a darwintools package for binutils, which I believe has a cross-compiler that will target the ELF format, but for now I decided to take the quick route and switch to developing under Linux on my EeePC. 

The EecPC runs the Xandros flavor of Linux, and by default it comes with some development tools, but not everything that we are going to need. For one thing, make is missing. Ya gotta have make – it simplifies the entire tedious compile/link/disk image creation process.

To set up the EeePC with all the dev goodies, we’ll follow this howto for installing development tools.

Before you can use apt-get, make sure that you have already added the apt-get repository that we added in our previous tutorial about getting our boot sector to run on the EeePC.

Once you’ve got that repository in your list, open a terminal window (by pressing ctrl-alt-T) and type the following commands:

sudo su
apt-get update
apt-get install build-essential

The first command, sudo su will put you into superuser mode. You may be prompted to enter the superuser password at this point. The second command, apt-get update will query the known repositories and update the known package lists. The final command, apt-get install will download and install the build-essential package, which has make and other necessary development utilities.

To test things out, type make at the command prompt. You should see an error message about no targets being specified, and no Makefile found. Congrats – if you see that error then all is good, and you are ready to go.

The last development tool that we’ll need is a very important one – subversion. Subversion is an excellent version control system. I’d advise anyone serious about software development to learn about subversion or some other version control system. Not having your code in a version control system is like walking a fraying tightrope without a net. For any new project, get your code under version control early.

Subversion is part of the same repository that we’ve been using, so we merely need to type:

 

sudo apt-get install subversion

 

and let apt-get work its magic.

My subversion repository is actually hosted by Google Code. They’ve got free project hosting, which is really swell.

Written by m3os

March 17, 2009 at 9:44 am

Tutorial – Simple Boot Sector running on eeePC

with one comment

Because I love gadgets, I got myself an eeePC 701 shortly after they came out. This was one of the first of the netbooks, and I was really enamored by the form-factor. I also really like the fact that it runs Linux.

Because I was bored, and slightly intimidated about working on my actual bootsector, I thought it would be fun to see if I could get the simple bootsector described here running on my eeePC.

Before I could do this, I needed to get and install the proper tools. Namely, Nasm and Qemu. This is pretty easy, thanks to the apt-get package manager. To install these, packages on an eeePC, follow these instructions:

sudo su
nano /etc/apt/sources.list

We need to add a repository for apt-get to look at, so add the following line to the file (note that this should all be on one line):

deb http://xnv4.xandros.com/xs2.0/upkg-srv2 etch main contrib non-free

Once you’ve added the line, hit Ctrl+O to save and Ctrl+X to quit. Now you need to tell apt-get to refresh its package lists (and grab lists from the new repository):

apt-get update

Now you can search your cached lists for the necessary packages. That’s how I found the ones for Nasm and Qemu named, appropriately, nasm and qemu. To install these, run the following commands:

apt-get install nasm
apt-get install qemu

You may be warned that the packages couldn’t be verified, but just type Y to install them anyway.

Voila! The tools are installed. Easy.

Now on to building the boot sector and running it on the emulator. We’ll use the same source code as in the other tutorial, and issue the same command to nasm to compile it:

nasm bootsect.asm -f bin -o bootsect.bin

The commands to create the floppy disk image using dd are also the same:

dd if=/dev/zero of=floppy.img bs=512 count=2880

dd if=bootsect.bin of=floppy.img conv=notrunc

Once we’ve got the floppy disk image, we have to use it to boot qemu. This is the main difference – the eeePC uses the command-line qemu, while OS X has the nice Q GUI. In this case, we want to tell qemu to use the image file as floppy disk drive A, and we’ll boot the virtual system using the command:

qemu -fda floppy.img

This tells Qemu to boot the system from floppy drive A, using the disk image floppy.img. A new window will open, and you should see the exact same boot screen as in the other tutorial, with a red A in the top left corner of the screen. Pretty painless, no?

I think that I’ll be using my Macbook for the bulk of the development, but it’s nice to know that I have options.

Written by m3os

March 5, 2009 at 8:43 am

Posted in Tutorial

Tagged with , , ,

Tutorial – Building a Simple Boot Sector on OS X

with 5 comments

A quick Google search on “boot sector tutorial” or “boot loader tutorial” yields slightly less than 1 kazillion matches. Folks, there’s a lot of information out there on how to write a boot sector.

If I ever hope to have a chance at finishing my own OS, I know that I will need to stand on the shoulders of those that have come before me. Some developers choose to reinvent the wheel rather than reuse freely available source code by others, but I’m not one of them. I prefer to take a pragmatic approach – I figure that if I try to write every single line of code without assistance, I’ll finish my operating system in around 156 years. That is, if I don’t go mad first.

That said, I found several good resources on the net that I used to guide me when creating a simple boot sector. And I mean simple – all this thing does is print a message to the screen before entering an infinite loop.

OK – first let me link to a helpful resource that I found and used when running my boot loader test. Robbert Haarman has put up some helpful x86 Development Tutorials on his website. He walks you through the steps to create a simple boot sector and a simple kernel (both written in assembly), and even shows keyboard support and starts getting in to system calls. As he goes, he explains some concepts you’ll need to know along the way. Unfortunately, he wrote these tutorials in 2005 and hasn’t continued on with the series. I enjoyed reading these and found them very helpful, so I wish there were more.

Like I said, for my first project I just wanted to get a simple boot sector to boot on the emulator that I have chosen to target – Q (Qemu on OS X). Q is basically a the QEmu emulator wrapped in a nice Cocoa interface.

First, let me show you the source code for the simple boot sector. I’ve seen variations of this code on countless sites – who knows who wrote it originally.

push word 0b800h
pop es
xor di, di
mov [es:di], word 441h

jmp $

times 510 – ($ – $$) db 0

db 55h
db 0aah

Copy the above code and paste it into a text file named bootsect.asm. I’m not going to go into depth about what each instruction in this code does – the point of this tutorial is to describe the tools and steps necessary to compile a boot sector and get in running on the emulator. We’ll get into the boot sector code at a later date, but at a high level, this code will compile into a binary that is 512 bytes in size. When it runs, it will display a red A in the upper left corner of the screen and then enter an infinite loop. If we see the red A, all is good.

As I mentioned before, I’m using Nasm as my assembler. I was pleased to discover that nasm (v 0.98.40) is already installed on my Mac. Thanks, Apple! 

When we’re building a boot sector, we aren’t building a typical executable program – so we want to compile it to a flat binary file. Nasm can output other executable formats, such as ELF, a.out,  PE, etc, but for the boot loader, we’re only concerned with flat binaries. This is the default output format for nasm, so you could leave out the target format (the -f flag), but to be explicit, run the following command:

nasm bootsect.asm -f bin -o bootsect.bin

This tells nasm to compile the bootsect.asm file into a flat binary file.

For this exercise, we want to create a disk image that could be written to a 3.5″ floppy disk (remember those?). For creating disk images and reading/writing to files at a low level, we’ll use the dd command. To create a disk image for a 3.5″ floppy disk, enter the following command:

dd if=/dev/zero of=floppy.img bs=512 count=2880

This says – create an output file named floppy.img, containing 2880 blocks, each block having a size of 512 bytes. We’ll fill the image with all zeroes, which we get from reading our input file: /dev/zero.

Now we need to insert our boot sector into our floppy disk image. The boot sector should be placed at the very beginning of the file. Again, we’ll use dd:

dd if=bootsect.bin of=floppy.img conv=notrunc

This says – write the contents of input file bootsect.bin to the start of output file floppy.img. The conv=notrunc tells dd to preserve the existing size of the output file instead of truncating it to the size of the input file.

OK – that’s all we need to do to make our disk image. Nasm and dd make it pretty easy.

Now we need to setup our virtual machine on Q. If you don’t yet have Q, you can grab it from the Q website. Once you’ve installed it, launch Q. To add a virtual machine, click on the big yellow plus button. Then you can name your virtual machine. Since we’re installing a custom OS (as opposed to Windows or Linux), we’ll choose Q Standard Guest as our OS.

 

Adding a New Virtual Machine to Q

Adding a New Virtual Machine to Q

Now we need to configure our machine. Since this is a very simple boot loader, we don’t need to the virtual machine to have networking, a lot of RAM, or even a hard disk drive. All we need is a floppy disk. Here are screenshots from the other steps of the new VM configuration.

 

General Tab

General Tab

Advanced Tab

Advanced Tab

The important tab for our purposes today is the Hardware tab. 

Remember, we created a disk image for a 3.5″ floppy, so let’s set up the virtual machine to boot from floppy. Be sure to specify that we want to boot from floppy. For our floppy drive, select “choose imagefile” and select the floppy.img file that we created earlier.

 

Hardware Tab - set it up to boot from our floppy disk image

Hardware Tab - set it up to boot from our floppy disk image

Important Tip – Q will only allow you to select image files that have the .img file extension. If you build an image file, but give it an extension that is something other than .img, Q won’t let you pick it. Don’t make the same mistake I did.

OK – now for the moment of truth. Once you’ve got all your settings correct, click on the Update PC button to save them. This will take you back to the Q Control screen, and you’ll see your new VM in the list. To start up your VM, double-click on your machine name or click the play button. If you’ve done everything properly, you’ll see a new window open that looks like this:

Q - Booting our VM from our Floppy Image!

Q - Booting our VM from our Floppy Image!

There it is – our red A in the upper left corner of the screen (we wrote over the massages displayed by the emulated BIOS).

Success! Take a few moments and savor the thrill.

Obviously, our boot sector at this point is essentially useless, but we’ve learned how to compile our boot sector assembly code into a binary, place that binary into a floppy disk image file, and boot our virtual PC using that image. Not a bad piece of work.

Written by m3os

March 2, 2009 at 8:56 am

Posted in Tutorial