How to Debug Crash Linux for ARM


It’s known that there are many tools to debug crash Linux for X86 architecture, such as kdump, LKCD, etc. Although many debugging tools claim that they could support ARM architecture, they are unstable. Is there a reliable method for ARM SOC? Yes, this blog will show a stable method for debugging ARM Linux. There is a premise that you could dump the memory snapshot by some assistive hardware tools when crash happen. This premise is easy for ARM SOC company.

This method for debugging ARM Linux is to compose a crash image and analyze the crash kernel with the aid of Crash Utility.

Now I will explain this method by step.
1.       Build Linux Kernel for ARM SOC. Please make sure that the feature kexec and debug info are enable.
·         Boot options  ---> 
·         [*] Kexec system call (EXPERIMENTAL)
·         [*] Export atags in procfs (NEW)
·         Kernel hacking  --->
·         [*] Compile the kernel with debug info
2.       Modify kernel source code for kernel/kexec.c
  12 --- a/kernel/kexec.c
  13 +++ b/kernel/kexec.c
  14 @@ -1089,14 +1089,16 @@ void crash_kexec(struct pt_regs *regs)
  15          * sufficient.  But since I reuse the memory...
  16          */
  17         if (mutex_trylock(&kexec_mutex)) {
  18 -               if (kexec_crash_image) {
  19 +//             if (kexec_crash_image) {
  20                         struct pt_regs fixed_regs;
  21  
  22                         crash_setup_regs(&fixed_regs, regs);
  23                         crash_save_vmcoreinfo();
  24                         machine_crash_shutdown(&fixed_regs);
  25 -                       machine_kexec(kexec_crash_image);
  26 -               }
  27 +                       flush_cache_all();
  28 +//                     machine_kexec(kexec_crash_image);
  29 +//             }
  30                 mutex_unlock(&kexec_mutex);
  31         }
3.       Build Linux Kernel.
4.       After the Linux Kernel run and the console shows, please output two physical address.
a.    cpu_notes_paddr: $cat sys/devices/system/cpu/cpu0/crash_notes
b.   vmcore_notes_paddr: $cat /sys/kernel/vmcoreinfo
5.       When crash happens, please use some assistive hardware method to dump the memory snapshot.
6.       Compose a available image for Crash Utility.
        This image is ELF format. There are some program headers. If the memory is flat sequence , two program headers are enough. One program header is for cpu_notes and vmcore_notes, the other is for memory snapshot. I have write this which could be referenced. The code could be found at https://github.com/tek-life/dump-crash-tool.
7.       After step 6 is completed, new file ‘newvmcore’ could be got. Then we could use Crash Utility to analyze the crash Linux Kernel.
$./crash vmlinux newvmcore
‘vmlinux’ is the kernel image with debug information. ‘newvmcore’ is got from step 6.

Following text is the Crash Utility output.

$ ./crash examples/vmlinux examples/vmcore

crash 6.1.5
Copyright (C) 2002-2013  Red Hat, Inc.
Copyright (C) 2004, 2005, 2006, 2010  IBM Corporation
Copyright (C) 1999-2006  Hewlett-Packard Co
Copyright (C) 2005, 2006, 2011, 2012  Fujitsu Limited
Copyright (C) 2006, 2007  VA Linux Systems Japan K.K.
Copyright (C) 2005, 2011  NEC Corporation
Copyright (C) 1999, 2002, 2007  Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions.  Enter "help copying" to see the conditions.
This program has absolutely no warranty.  Enter "help warranty" for details.

GNU gdb (GDB) 7.3.1
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "--host=i686-pc-linux-gnu --target=arm-linux-gnueabi"...

      KERNEL: examples/vmlinux
    DUMPFILE: examples/vmcore
        CPUS: 1
        DATE: Sat Jan  1 10:29:39 2000
      UPTIME: 00:00:51
LOAD AVERAGE: 0.08, 0.03, 0.01
       TASKS: 22
    NODENAME: Linux
     RELEASE: 3.4.5-00527-g320e261-dirty
     VERSION: #506 Fri May 10 15:57:51 CST 2013
     MACHINE: armv7l  (unknown Mhz)
      MEMORY: 128 MB
       PANIC: "[   51.520000] Internal error: Oops: 817 [#1] ARM" (check log for details)
         PID: 297
     COMMAND: "sh"
        TASK: c7870900  [THREAD_INFO: c793a000]
         CPU: 0
       STATE: TASK_RUNNING (PANIC)

crash>


评论

此博客中的热门博文

提交了30次才AC ---【附】POJ 2488解题报告

n个进程共享m个资源得死锁问题证明