NetBSD is an extremely flexible operating system that is designed to be portable across various architectures. This feature makes it attractive for embedded developers. In this article, I will demonstrate a process for creating a very small kernel that can boot, either to a shell prompt or to a login screen.
Booting to the shell prompt allows the developer to quickly give life to a system and perform some basic interactions. The shell itself can be a powerful management tool, combined with the right collection of programs. Doing this requires only the kernel and two binaries.
Booting to a login prompt provides a little added security by controlling access to the machine. This would be useful in devices where the console can be accessed by the user, and you need to control what the user has access to. There are a few added files in this setup, and to keep things small, you will need to manually trim some of them. But this is still a very easy process.
The first stage of embedded development usually involves getting a skeleton operating system up and running. Assuming you are using one of NetBSD many supported platforms, this is quite easy. I used a PC booting off of a floppy drive. But this method can also be easily adapted to handle diskless clients that boot off a network.
At an absolute minimum, you will need the kernel, the /dev nodes for your system, and a /sbin/init file. You can craft the init program yourself, putting all of your application code into it. This may be a good solution when you only have one application that you would like to run.
Another option is to use the stock init program and a shell to call your application. This is a bit easier than writing your own init program, and it also allows you to add functionality to your system by simply adding more tools from the stock system. The shell itself is a powerful tool that can make future development much easier. This is the method that will be illustrated here.
$ tar -xzvpf gnusrc.tar.gz -C / $ tar -xzvpf sharesrc.tar.gz -C / $ tar -xzvpf src.tar.gz -C / $ tar -xzvpf syssrc.tar.gz -C / $ tar -xzvpf xsrc.tar.gz -C / $ cd /usr/src $ ./build.sh -m i386 toolsThis script builds the tools needed to produce code that will run on an i386 based computer. You can change this to another architecture, such as sparc64, if you want your tiny NetBSD to run on that platform. Once completed you should have a bunch of tools in usr/src/tooldir.
Once the tools are built the next thing that needs to be done is to build a kernel with a built in ramdisk. The ramdisk will hold the kernel's root filesystem. A good kernel configuration to start with is the INSTALL_TINY configuration. This kernel has all the important stuff in it, without a lot of bloat. Make sure that it has the following lines in it.
in sys/arch/i386/conf
$ cp INSTALL_TINY MYTINY $ vi MYTINY
# Enable the hooks used for initializing the root memory-disk. options MEMORY_DISK_HOOKS options MEMORY_DISK_IS_ROOT # force root on memory disk options MEMORY_DISK_SERVER=1 # make the ramdisk writeable #options MEMORY_DISK_ROOT_SIZE=2880 # 1.44M, same as a floppy options MEMORY_DISK_ROOT_SIZE=8192 # 4Meg options MEMORY_RBFLAGS=0 # don't force single userThis configuration includes ramdisk support, has the kernel boot from the ramdisk, allows the ramdisk to be written to, sets the size a 8192 sectors (512 bytes each), and allows the kernel to boot into multiuser mode (if the files are available).
Once this is done, build your kernel. nbmake will give you a report on the file and you can get some more information with the 'file' command. Once the kernel is built, you can copy it to wherever you want, to make things easier to work with.
$ /usr/src/tooldir.NetBSD-3.0.1-i386/bin/nbconfig MYTINY $ cd ../compile/MYTINY $ /usr/src/tooldir.NetBSD-3.0.1-i386/bin/nbmake-i386 depend $ /usr/src/tooldir.NetBSD-3.0.1-i386/bin/nbmake-i386 $ file netbsd $ copy netbsd ~/NetBSD/netbsd.ramdisk
Although you could copy the binaries from your host into your mini filesystem, a more efficient way (sizewise) to do this is to use a utility called crunchgen. Many programs in the NetBSD system are linked statically. For example, each program that uses the utility library (libutil) has portions of the library linked to it. Several programs on the system can produce redundant copies of same library code used by the programs. Crunchgen takes the programs' object files and merges them into one uber-program. This crunched binary is then linked to the libraries, so only one copy of the library code is needed for the whole system.
"But how do I use the different programs?", you ask. The answer lies in hard links. A hard link is like a filesystem's alias for a program. For example, If I have a program called print_my_error, I can link that to the alias myerr. Then when I invoke myerr, the filesystem simply follows the link and runs the program print_my_error. I can even make the two program names have slightly different behavior. This is done by having print_my_error look at argv[0] that is passed to the main() function. If it is myerr, then I can have it do a special task.
This is how crunchgen works. It takes all of its constituent programs and globs them together. Then in the crunched binary's main routine, there is logic that examines argv[0] and calls the main routine for the appropriate constituent.
To make a crunched binary, you need to know what binaries you want, where their sources are located, and what libraries they use. Once you settle on what binaries to use, simply search the /usr/src folder for the sources. The layout of /usr/src is straightforward. To look for the source for /sbin/init, you would look in /usr/src/sbin/init. To find the /usr/bin/login, go to /usr/src/usr.bin/login. To find out which libraries are used, open up the program's Makefile, and look for the LDADD lines. You should see entries like "-lutil -lcrypt".
Sometimes, programs in the NetBSD distribution are simply hardlinks to another program. For example, mount_mfs is an alias for newfs. You will find no /usr/src/sbin/mount_mfs directory. So how do you find out where the sources are for mount_mfs? Search through the makefiles for "LINKS= ${BINDIR}/newfs ${BINDIR}/mount_mfs". This shows you that the Makefile will link mount_mfs to newfs.
$ find /usr/src -name Makefile -exec grep mount_mfs {} \;
MAN= newfs.8 mount_mfs.8
LINKS= ${BINDIR}/newfs ${BINDIR}/mount_mfs
MLINKS= mount_mfs.8 mfs.8
Armed with this information, you can now create your crunchgen configuration file. This is just a list of the above information. There are some additional features, which I will outline below. Replace the /usr/src lines with the full path to your extracted sources. I'm building my crunchgen in the ~/NetBSD/work folder.
srcdirs /usr/src/bin /usr/src/sbin progs init sh reboot ls ln sh -sh special init objpaths /usr/src/sbin/init/init.smallprog.o # libraries used by the programs # ---------------- Minimum single user files # init : -lutil -lcrypt # sh : -ll -ledit -ltermcap # ---------------- Useful utilities # ls : - # reboot : -lutil # libs -lutil -lcrypt -ll -ledit -ltermcapThe first line tells crunchgen where to look for sources. It looks for the sources by appending the program name onto the listed directories. The progs line is the list of programs that you want included in your binary. The ln lines tell crunchgen about aliases that are used for some of the programs. The shell is sometimes invoked as -sh, so the crunched binary will recognize "-sh" as "sh". Also, as noted above, some binaries are simply aliases for other programs. The crunched binary needs to be on the lookout for these aliases as well.
The special line tells crunchgen that init should not be built from source, but rather just use the specified object file. In this case, I built a modified init program with the SMALLPROG #define, so I don't get the annoying "Enter pathname of shell or RETURN for sh:" prompt when the system starts in single user mode. Instead, it will silently drop to the shell prompt. To do this, I build my special init program this way.
$ cd /usr/src/sbin/init $ make -D SMALLPROG $ cp init.o init.smallprog.oThis functionality makes it easy to customize your system without too much trouble. You could create a customized version of a standard program and use that in your embedded builds, while keeping the original for other purposes.
Finally, the last line tells crunchgen which libraries to link to. Again, you can get this information by looking through the Makefiles for the constituent programs and noting the LDADD lines. I've had trouble with library ordering, so if you have errors indicating unresolved externals, and you know you included the library, try moving the "missing" library closer to the front of the list. This worked for me.
Once your crunchgen file is crafted, build it with the following commands. The finished executable will be named after your conf file, but without the conf extension.
$ if [ ! -x work ]; then $ mkdir work $ else $ rm -rf work $ fi $ cd work $ crunchgen -m Makefile mytiny.conf $ make -f mytiny.mk objs exe $ ls -l mytiny $ cd ..
$ if [ ! "root" = `whoami` ]; then $ echo You need root permissions to run MAKEDEV. $ exit 1 $ fi $ mkdir files $ mkdir files/bin files/sbin files/dev files/etc $ cp /dev/MAKEDEV files/dev $ cd files/dev $ ./MAKEDEV floppy ramdisk wscons $ cd ../.. $ cp work/mytiny files/sbin/init $ ln files/sbin/init files/bin/sh $ ln files/sbin/init files/bin/ls $ ln files/sbin/init files/sbin/reboot $ echo "#!/bin/sh" > files/.profile $ echo "echo Hello World!" >> files/.profile $ du -sh filesWith the filesystem populated, we can wrap it up into an image file that can be embedded into a kernel. This is done with the makefs command. This tool lets you take a directory and bundle it up into a single file. You can adjust your MEMORY_DISK_ROOT_SIZE to an appropriate value to hold your filesystem. Once the image is created, link it to the kernel with the mdsetimage command. Your kernel is now ready to go!
You can compress your kernel if you like. The standard NetBSD bootloader knows how to decompress gzipped kernels. Remember, your spartan OS includes 4 megabytes of mostly empty space in the filesystem. I compressed my kernel down to 825,544 bytes.
# makefs -s 4m -t ffs crunch.image files # mdsetimage netbsd.ramdisk crunch.image # gzip -c netbsd.ramdisk > netbsd.tiny # ls -l netbsd.tiny -rw-r--rw- 1 brose users 1015387 Sep 14 22:28 netbsd.tiny
A multi user system is built in the same manner, but you need a few extra programs and data files. I also use the standard init program. Notice that I commented out the special init line, so that I use the stock init for the multiuser configuration. You may want to go back to your init source and make a fresh init, just in case the .o files are from your single user build.
$ cd /usr/src/sbin/init $ make clean; makeHere's my crunchgen configuration.
srcdirs /usr/src/bin /usr/src/sbin /usr/src/usr.bin /usr/src/usr.sbin /usr/src/libexec progs init mount newfs mount_ffs sh ttyflags getty pwd_mkdb passwd login reboot ls ln sh -sh ln newfs mount_mfs # special init objpaths /usr/src/sbin/init/init.smallprog.o # libraries used by the programs # ---------------- Minimum single user files # init : -lutil -lcrypt # mount : # newfs : -lutil # mount_ffs : # sh : -ll -ledit -ltermcap # ---------------- Minimum multiuser files # ttyflags : # getty : -lutil -ltermcap # pwd_mkdb : -lutil # passwd : -lrpcsvc -lcrypt -lutil -lkrb5 -lcrypto -lasn1 -ldes -lcom_err -lroken # login : -lutil -lcrypt -lskey -lkrb5 -lasn1 -lkrb -lcrypto -lroken -lcom_err # ---------------- Useful utilities # ls : # reboot : -lutil # umount : # libs -lutil -ll -ledit -ltermcap -lcrypt -lrpcsvc -lkrb5 -lkrb -lcrypto -ldes -lasn1 -lcom_err -lroken -lskeyAnd my shell commands for populating the filesystem. Note that the script must be pointed to your crunched file. These commands also need to be run as root.
CRUNCHED_FILE=crunch/mytiny mkdir files mkdir files/bin files/sbin files/usr files/etc files/var files/dev files/tmp files/root files/home mkdir files/usr/bin files/usr/sbin files/usr/libexec mkdir files/var/run files/var/db files/var/crash mkdir files/usr/share mkdir files/usr/share/misc cp /dev/MAKEDEV files/dev cd files/dev ./MAKEDEV floppy ramdisk wscons cd ../.. echo "/dev/md0a / ffs rw 1 1" > files/etc/fstab echo "echo Initializing system..." > files/etc/rc echo "export PATH=/sbin:/bin:/usr/sbin:/usr/bin" >> files/etc/rc echo "mount -ua" >> files/etc/rc echo "ttyflags -a" >> files/etc/rc cp $CRUNCHED_FILE files/sbin/init ln files/sbin/init files/bin/ls ln files/sbin/init files/sbin/mount ln files/sbin/init files/sbin/mount_ffs ln files/sbin/init files/sbin/mount_mfs ln files/sbin/init files/sbin/umount ln files/sbin/init files/bin/sh ln files/sbin/init files/usr/libexec/getty ln files/sbin/init files/sbin/ttyflags ln files/sbin/init files/sbin/pwd_mkdb ln files/sbin/init files/usr/bin/passwd ln files/sbin/init files/usr/bin/login ln files/sbin/init files/sbin/reboot ln files/sbin/init files/sbin/newfs cp /etc/ttys files/etc cp /etc/master.passwd files/etc cp /etc/pwd.db files/etc cp /etc/spwd.db files/etc cp /etc/passwd files/etc cp termcap.mini files/usr/share/misc/termcap cp /etc/gettytab files/etc
The termcap.mini file is simply a hand trimmed version of the termcap file. This is used by the getty program when initializing the console. Since it is very large (over 500k) and most of it is useless, you can trim out the terminal types that you don't use. I trimmed mine down to about 8k by only selecting the "NetBSD consoles" section.
Once you have done this, simply place this filesystem into a kernel and boot. As you can see, a multi user setup is a bit larger. But most of the extra space is from incorporating the extra libraries for the login and getty programs. As you add more code to the system, your code growth should not be as dramatic.
# makefs -s 4m -t ffs crunch.image files # mdsetimage netbsd.ramdisk crunch.image # gzip -c netbsd.ramdisk > netbsd # ls -l netbsd -rw-r--rw- 1 brose users 1066057 Aug 24 22:28 netbsd