ARM Linux Booting Process
This will be a series of articles explaining how Linux kernel boots up on ARM architecture. This is part one.
ARM Linux Boot Process:
We will explain boot process of AT91RM9200 system-on-chip, built around the ARM920T ARM Thumb processor. Kwickbyte builds an embedded board called kb9202 based on AT91RM9200. We will take this board as an example and see how Linux boots up on this board.
Before you start reading this you need to read AT91RM9200 data sheet (specification).
You also need to read ARM Architecture Reference Manual for better understanding the boot process.
Components in Linux Boot Process:
Linux boot process involves the following components.
Bootloader
Kernel Image
Root Filesystem
Before we see how the above components work, the following is the call flow of Linux Kernel boot process for arm architecture. This gives a big picture on whole Linux boot process. We use U-boot bootloader.
ARM Linux Boot Process: Big Picture
U-boot:
_start (cpu/arm920t/start.S)
start_code (cpu/arm920t/start.S)
start_armboot (lib_arm/board.c)
board_init (board/kb9202/kb9202.c)
timer_init (cpu/arm920t/at91/timer.c)
serial_init (drivers/serial/at91rm9200_usart.c)
main_loop (lib_arm/board.c)
Now u-boot is up and running and is in u-boot prompt and ready to accept commands. Assume that kernel image is loaded into RAM and issued bootm command.
do_bootm (common/cmd_bootm.c)
bootm_start (common/cmd_bootm.c)
bootm_load_os (common/cmd_bootm.c)
do_bootm_linux (lib_arm/bootm.c)
stext (linux/arch/arm/kernel/head.S)
Control is given to linux.
Linux Kernel:
stext (arch/arm/kernel/head.S:78)
__lookup_processor_type (arch/arm/kernel/head-common.S:160)
__lookup_machine_type (arch/arm/kernel/head-common.S:211)
__create_page_tables (arch/arm/kernel/head.S:219)
__arm920_setup (arch/arm/mm/proc-arm920.S:389)
__enable_mmu (arch/arm/kernel/head.S:160)
__turn_mmu_on (arch/arm/kernel/head.S:205)
__switch_data (arch/arm/kernel/head-common.S:20)
start_kernel (init/main.c:529)
start_kernel (init/main.c:529)
tick_init(kernel/time/tick-common.c:413)
setup_arch (arch/arm/kernel/setup.c:666)
setup_machine (arch/arm/kernel/setup.c:369)
lookup_machine_type ( )
setup_command_line (init/main.c:408)
build_all_zonelists (mm/page_alloc.c:3031)
parse_args (kernel/params.c:129)
mm_init (init/main.c:516)
mem_init (arch/arm/mm/init.c:528)
kmem_cache_init (mm/slab.c, mm/slob.c, mm/slub.c)
sched_init (kernel/sched.c)
init_IRQ (arch/arm/kernel/irq.c)
init_timers (kernel/timer.c:1713)
hrtimers_init (kernel/hrtimer.c:1741)
softirq_init (kernel/softirq.c:674)
console_init (drivers/char/tty_io.c:3084)
vfs_caches_init (fs/dcache.c:2352)
mnt_init (fs/namespace.c:2308)
init_rootfs ()
init_mount_tree (fs/namespace.c:2285)
do_kern_mount (fs/namespace.c:1053)
set_fs_pwd(fs/fs_struct.c:29)
set_fs_root(fs/fs_struct.c:12)
bdev_cache_init (fs/block_dev.c:465)
chrdev_init (fs/char_dev.c:566)
signals_init (kernel/signal.c:2737)
rest_init (init/main.c:425)
kernel_thread (431, arch/arm/kernel/process.c:388)
kernel_thread() creates a kernel thread and control is given to kernel_init().
kernel_init (431, init/main.c:856)
do_basic_setup (888, init/main.c:787)
init_workqueues (789, kernel/workqueue.c:1204)
driver_init (793, drivers/base/init.c:20)
do_initcalls (796, init/main.c:769) /* Calls all subsytems init functions */
prepare_namespace (906, init/do_mounts.c:366)
initrd_load (399, init/do_mounts_initrd.c:107)
rd_load_image (117, init/do_mounts_rd.c:158) /* if initrd is given */
identify_ramdisk_image (179, init/do_mounts_rd.c:53)
handle_initrd (119, init/do_mounts_initrd.c:37) /*if rd_load_image is success */
mount_block_root (45, init/do_mounts.c:233)
do_mount_root (247, init/do_mounts.:218)
mount_root (417, init/do_mounts.c:334) /* if initrd not given */
mount_block_root (359, init/do_mounts.c:233)
do_mount_root (247, init/do_mounts.c:218)
init_post (915, init/main.c:816)
run_init_process (847, init/main.c:807)
kernel_execve (810, arch/arm/kernel/sys_arm.c:81)
User Space
init() /*userspace /sbin/init */
Bootloader:
A bootloader is a small program which will load the kernel image into RAM and boots up the kernel image. This is also called bootstrap as it brings(pulls) up system by loading an operating system. Bootloader starts before any other software starts and initializes the processor and makes cpu ready to execute a program like an operating system. Most processors have a default address from which the first bytes of code are fetched upon power is applied or board is reset. Hardware designers use this information to store the bootloader code at that address in ROM or flash. Since it should initialize the cpu and should run a program which is located at architecture specific address bootloaders are highly processor specific and board specific. Every embedded board comes with a bootstrap to download the kernel image or standalone application into the board and start executing the kernel image or application. Bootloader will be executed when power is applied to a processor board. Basically it will have some minimal features to load the image and boot it up.
It is also possible to control the system using a hardware debug interface such as JTAG. This interface may be used to write the boot loader program into bootable non-volatile memory (e.g. flash) by instructing the processor core to perform the necessary actions to program non-volatile memory. Generally done for first time to download the basic bootloader and for some recovery process. JTAG is a standard and popular interface provided by many board vendors. Some micro controllers provide special hardware interfaces which can’t be used to take arbitrary control of a system or directly run code, but instead they allow the insertion of boot code into bootable non-volatile memory (like flash memory) via simple protocols. Then at the manufacturing phase, such interfaces are used to inject boot code (and possibly other code) into non-volatile memory. After system reset, the micro controller begins to execute code programmed into its non-volatile memory, just like usual processors are using ROMs for booting. In many cases such interfaces are implemented by hardwired logic. In other cases such interfaces could be created by software running in integrated on-chip boot ROM from GPIO pins.
There are some other third party bootloaders available which provide rich set of features and easy user interface. You can download these third party bootloaders into board and can make them default bootloaders for your board. Generally bootloaders provided by board vendors are replaced with these third party bootloader. There are a quite few third party boolader available and some of them are open source (or free bootloaders) and some are commercial. Some of them are Das U-Boot, Red boot, GRUB (for desktops), LILO, Loadlin,, bootsect-loader, SYSLINUX, EtherBoot, ELILO.
We will take U-boot boot loader as our boot loader. U-boot is the widely used boot loader in embedded systems. We will explain code from the u-boot-2010.03 source. You can download U-boot from the following site. http://www.denx.de/wiki/U-Boot
How U-boot is built:
Based on the configuration of U-boot, all the assembly files (.S) and C files (.c) are compiled using cross compiler which is built for a particular architecture and object files(.o) will be generated. All these object files are linked by linker and an executable file will be created. An object file or executable file is a collection of sections like.text,.data,.bss etc. Object files and executable files have a file format like elf. All the sections of the object files will be arranged in the executable file based on a script called linker script. This script tells where all the sections are to be loaded in the memory when it runs. Understanding this script is very important to know how boot loader and kernel are composed and how different sections of boot loader or kernel are loaded in the memory.
Generally, when a program is run (executed) a loader reads executable file and loads different sections of the executable file in the specified memory location and starts executing the start function(entry point) specified in the linker script. But, if you want to run(load) a boot loader there will not be any loader to load(basically to understand the file format) different sections of executable file into the memory. Then you need to use a tool called objcopy which will take all sections from the executable file and create a binary file which doesn’t have any file format. This binary file can be loaded into the memory and executed or can be written in to the ROM at a particular address (specific to the architecture) which will be executed by cpu when power is applied to the board.
Assume that based on the U-boot configuration all files are compiled and object files are created. U-boot makefile uses the following linker script (specific to architecture) to build an executable file.
File: cpu/arm920t/u-boot.lds
32 OUTPUT_FORMAT(“elf32-littlearm”, “elf32-littlearm”, “elf32-littlearm”)
33 OUTPUT_ARCH(arm)
34 ENTRY(_start)
35 SECTIONS
36
37. = 0x00000000;
38
39. = ALIGN(4);
40.text:
41
42 cpu/arm920t/start.o (.text)
43 *(.text)
44
4546. = ALIGN(4);
47.rodata: *(SORT_BY_ALIGNMENT(SORT_BY_NAME(.rodata*)))
48
49. = ALIGN(4);
50.data: *(.data)
51
52. = ALIGN(4);
53.got: *(.got)
54
55. =.;
56 __u_boot_cmd_start =.;
57.u_boot_cmd: *(.u_boot_cmd)
58 __u_boot_cmd_end =.;
59
60. = ALIGN(4);
61 __bss_start =.;
62.bss (NOLOAD): *(.bss). = ALIGN(4);
63 _end =.;
64
OUTPUT_FORMAT in line #32 specify the file format of the executable file. Here the executable file format is elf32 and endianness is little endian. OUTPUT_ARCH in line # 33 specify the architecture on which this code runs. ENTRY in line #34 specifies the start function(entry point) of u-boot program. Here the entry point is _start.
SECTIONS in line #35 defines how different sections are mapped in the executable file. Loader uses the addresses specified in this section to load different section of the program into the memory.
‘.’ in the line #37 specifies the start address where the following sections should be loaded. In this case start address is 0x00000000. After this in line #39 the memory is aligned by 4 bytes and the.text section follows in the line #40.
40.text:
41
42 cpu/arm920t/start.o (.text)
43 *(.text)
44
At the ‘.’ position (0x00000000) the code in the cpu/arm920t/start.o is mapped and follows the code that is there in.text sections of all other object (.o) files. cpu/arm920t/start.o contains the _start() function(in assembly language) which is entry point of this program.
Now the ‘.’ will be at 0x00000000 + sizeof (.text). Again memory is aligned by 4 bytes and.rodata section follows in line #47.
. = ALIGN(4);
47.rodata: *(SORT_BY_ALIGNMENT(SORT_BY_NAME(.rodata*)))
.rodata sections from all objects files are mapped at this address. Follows the.data and.git sections.
49. = ALIGN(4);
50.data: *(.data)
51
52. = ALIGN(4);
53.got: *(.got)
Each U-boot command is an object of type ‘cmd_tbl_t’ which contains command name, help string and function pointer to be executed when this command is run. All these command objects are placed in the memory sequentially. Each of this command object is built into an U-boot defined section called.u_boot_cmd in the object file. These all.u_boot_cmd sections are placed in the memory after the above sections(.data and.git).
. =.;
56 __u_boot_cmd_start =.;
57.u_boot_cmd: *(.u_boot_cmd)
58 __u_boot_cmd_end =.;
__u_boot_cmd_start contains the start of the commands objects and __u_boot_cmd_end contains the end of the command objects.
And next follows the.bss (uninitialized global variables) sections.
60. = ALIGN(4);
61 __bss_start =.;
62.bss (NOLOAD): *(.bss). = ALIGN(4);
63 _end =.;
__bss_start points to the.bss start address and _end contains the end of the all sections.
Using this linker script linker will generate an executable file called u-boot. Objcopy tool is used to generate a binary file from the u-boot executable file.
u-boot.bin: u-boot
$(OBJCOPY) $OBJCFLAGS -O binary $