参考文献

搭建环境

  • 使用ubuntu-20.04.5-live-server-arm64.iso镜像文件,使用虚拟机环境搭建Ubuntu20.04,在其基础上编译linux kernel

安装编译内核需要的依赖

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
root@holelin:~# sudo apt install git build-essential kernel-package fakeroot libncurses5-dev libssl-dev ccache flex bison libelf-dev
Reading package lists... Done
Building dependency tree
Reading state information... Done
bison is already the newest version (2:3.5.1+dfsg-1).
ccache is already the newest version (3.7.7-1).
fakeroot is already the newest version (1.24-1).
flex is already the newest version (2.6.4-6.2).
kernel-package is already the newest version (13.018+nmu2).
build-essential is already the newest version (12.8ubuntu1.1).
git is already the newest version (1:2.25.1-1ubuntu3.13).
libelf-dev is already the newest version (0.176-1.1ubuntu0.1).
libncurses5-dev is already the newest version (6.2-0ubuntu2.1).
libssl-dev is already the newest version (1.1.1f-1ubuntu2.23).
0 upgraded, 0 newly installed, 0 to remove and 193 not upgraded.
root@holelin:~# sudo apt install gcc-x86-64-linux-gnu g++-x86-64-linux-gnu
Reading package lists... Done
Building dependency tree
Reading state information... Done
g++-x86-64-linux-gnu is already the newest version (4:9.3.0-1ubuntu2).
gcc-x86-64-linux-gnu is already the newest version (4:9.3.0-1ubuntu2).
0 upgraded, 0 newly installed, 0 to remove and 193 not upgraded.
root@holelin:~#
  • sudo apt install gcc-x86-64-linux-gnu g++-x86-64-linux-gnu安装适用于x86架构的gccg++编译器

下载linux内核代码

1
2
# cd /root
# git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

内核编译配置项设置

  • 进入源码目录,输入make ARCH=x86_64 menuconfig配置Linux内核的选项
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
# cd linux
# make ARCH=x86_64 menuconfig

# 下面选项如果没有选上的,选上(点击空格键),然后 save 保存设置,退出 exit。
##################################################################
General setup --->
[*] Initial RAM filesystem and RAM disk (initramfs/initrd) support

Device Drivers --->
[*] Block devices --->
<*> RAM block device support
(65536) Default RAM disk size (kbytes)

Processor type and features --->
[*] Randomize the address of the kernel image (KASLR)

Kernel hacking --->
Compile-time checks and compiler options --->
[*] Compile the kernel with debug info
[*] Provide GDB scripts for kernel debugging

Device Drivers -->
Network device support -->
<*> Universal TUN/TAP device driver support

[*] Networking support -->
Networking options -->
<*> 802.1d Ethernet Bridging

img

img

  • 经过上述操作后,会产生.config文件

开始编译内核

1
2
3
4
5
root@holelin:~/linux# make ARCH=x86_64 CROSS_COMPILE=x86_64-linux-gnu- -j8
mkdir -p /root/linux/tools/objtool && make O=/root/linux subdir=tools/objtool --no-print-directory -C objtool
INSTALL libsubcmd_headers
CALL scripts/checksyscalls.sh
Kernel: arch/x86/boot/bzImage is ready (#2)
  • 注意: 此处因为之前编译过,就直接成功产生了结果文件arch/x86/boot/bzImage,执行上述命令后,还需要一些配置,可以通过按Enter键来使用默认值

  • 最终会产生两个文件

    • 编译过程会产生一个映像文件arch/x86/boot/bzImage
    • Linux源代码目录中会产生一个名为vmlinux的对象文件
  • 成功产生上述两个文件,则表明已成功编译Linux内核 🎉

运行内核

  • 使用qemu来运行内核

    1
    # sudo apt install qemu qemu-system
  • 使用qemu运行

    1
    # qemu-system-x86_64 -no-kvm -kernel arch/x86/boot/bzImage -hda /dev/zero -append "root=/dev/zero console=ttyS0" -serial stdio -display none
    • 提示无法缺少文件系统

img

使用buildroot用于生成文件系统映像

  • 访问https://buildroot.org/download.html,下载并解压缩buildroot-2024.08.1.tar.gz文件

    1
    2
    3
    4
    5
    6
    7
    # cd buildroot-2024.08.1/
    # sudo make menuconfig

    # 下面选项如果没有选上的,选上(点击空格键),然后 save 保存设置,退出 exit。
    ##################################################################
    Target Options → Target Architecture → x86_64
    Filesystem images → ext2/3/4 root file system → ext4
  • 配置完成后,使用make ARCH=x86_64 -j8进行编译.这个会花费一段时间

    1
    # make ARCH=x86_64 -j8
  • 编译成功后会在buildroot-2024.08.1/output/images/下产生文件系统镜像

    1
    2
    3
    4
    5
    6
    7
    root@holelin:~/buildroot-2024.08.1# ll output/images/
    total 19012
    drwxr-xr-x 2 root root 4096 Nov 4 03:26 ./
    drwxr-xr-x 6 root root 4096 Nov 4 03:26 ../
    -rw-r--r-- 1 root root 62914560 Nov 4 06:24 rootfs.ext2
    lrwxrwxrwx 1 root root 11 Nov 4 03:26 rootfs.ext4 -> rootfs.ext2
    -rw-r--r-- 1 root root 5795840 Nov 4 03:26 rootfs.tar
  • 进入linux kernel目录下,重新使用qemu启动内核

    1
    2
    # cd /root/linux
    # sudo qemu-system-x86_64 -s -S -kernel arch/x86/boot/bzImage -boot c -m 2049M -hda /root/buildroot-2024.08.1/output/images/rootfs.ext4 -append "root=/dev/sda rw console=ttyS0,115200 acpi=off nokaslr" -serial stdio -display none
    • 输入root直接登录

img

debug linux kernel on mac m1

  • 因为本人笔记本为MBP M1,无法直接使用gdb来调试,只能使用XCode带有的lldb来调试

  • linux kernel目录以及buildroot-2024.08.1目录复制到Mac上,与此同时同样需要安装qemu,在Mac上进入,即在我本机上目录结构为

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    68
    69
    70
    71
    # cd ~/Downloads/linux-debug
    # tree -L 2
    .
    ├── buildroot-2024.08.1
    │   ├── CHANGES
    │   ├── COPYING
    │   ├── Config.in
    │   ├── Config.in.legacy
    │   ├── DEVELOPERS
    │   ├── Makefile
    │   ├── Makefile.legacy
    │   ├── README
    │   ├── arch
    │   ├── board
    │   ├── boot
    │   ├── configs
    │   ├── dl
    │   ├── docs
    │   ├── fs
    │   ├── linux
    │   ├── output
    │   ├── package
    │   ├── support
    │   ├── system
    │   ├── toolchain
    │   └── utils
    ├── buildroot-2024.08.1.tar.gz
    ├── linux
    │   ├── COPYING
    │   ├── CREDITS
    │   ├── Documentation
    │   ├── Kbuild
    │   ├── Kconfig
    │   ├── LICENSES
    │   ├── MAINTAINERS
    │   ├── Makefile
    │   ├── Module.symvers
    │   ├── README
    │   ├── System.map
    │   ├── arch
    │   ├── block
    │   ├── built-in.a
    │   ├── certs
    │   ├── crypto
    │   ├── drivers
    │   ├── fs
    │   ├── include
    │   ├── init
    │   ├── io_uring
    │   ├── ipc
    │   ├── kernel
    │   ├── kernel_test
    │   ├── lib
    │   ├── mm
    │   ├── modules.builtin
    │   ├── modules.builtin.modinfo
    │   ├── modules.order
    │   ├── net
    │   ├── rust
    │   ├── samples
    │   ├── scripts
    │   ├── security
    │   ├── sound
    │   ├── tools
    │   ├── usr
    │   ├── virt
    │   ├── vmlinux
    │   ├── vmlinux-gdb.py -> /root/linux/scripts/gdb/vmlinux-gdb.py
    │   ├── vmlinux.a
    │   └── vmlinux.o
    └── linux.tar.gz
  • 启动内核

    1
    2
    # cd linux
    # qemu-system-x86_64 -s -S -kernel arch/x86/boot/bzImage -boot c -m 2049M -hda /Users/holelin/Downloads/linux-debug/buildroot-2024.08.1/output/images/rootfs.ext4 -append "root=/dev/sda rw console=ttyS0,115200 acpi=off nokaslr" -serial stdio -display none
  • 使用lldb调试,新启动一个终端

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    # lldb --arch x86_64 /Users/holelin/Downloads/linux-debug/linux/vmlinux

    (lldb) target create --arch=x86_64 "/Users/holelin/Downloads/linux-debug/linux/vmlinux"
    Current executable set to '/Users/holelin/Downloads/linux-debug/linux/vmlinux' (x86_64).
    (lldb) settings set target.source-map /root/linux /Users/holelin/Downloads/linux-debug/linux
    (lldb) gdb-remote 1234
    Process 1 stopped
    * thread #1, stop reason = signal SIGTRAP
    frame #0: 0x000000000000fff0 vmlinux`exception_stacks + 16368
    vmlinux`exception_stacks:
    -> 0xfff0 <+16368>: addb %al, (%rax)
    0xfff2 <+16370>: addb %al, (%rax)
    0xfff4 <+16372>: addb %al, (%rax)
    0xfff6 <+16374>: addb %al, (%rax)
    (lldb) b start_kernel
    Breakpoint 1: where = vmlinux`start_kernel + 39 at main.c:908:2, address = 0xffffffff832669b7
    (lldb) b tcp_v4_connect
    Breakpoint 2: where = vmlinux`tcp_v4_connect + 34 [inlined] sock_net at sock.h:655:9, address = 0xffffffff81c93322
    (lldb) c
    Process 1 resuming
    Process 1 stopped
    * thread #1, stop reason = breakpoint 1.1
    frame #0: 0xffffffff832669b7 vmlinux`start_kernel at main.c:908:2
    905 char *command_line;
    906 char *after_dashes;
    907
    -> 908 set_task_stack_end_magic(&init_task);
    909 smp_setup_processor_id();
    910 debug_objects_early_init();
    911 init_vmlinux_build_id();
    • settings set target.source-map /root/linux /Users/holelin/Downloads/linux-debug/linux: 设置源码路径映射

    • gdb-remote 1234: 连接服务

    • b start_kernelb tcp_v4_connect 设置两个断点

    • c 跳过断点

    • b tcp_v4_connect 断点可以通过telnet ip port来触发

      img