1

I have made a custom board using Tegra TK1 ARM processor. I have compiled uboot and kernel for it and flashed into it. Now when it starts booting, sometimes it boots successfully while sometimes kernel crashes. Everytime it crashes at different address and start dumping. Every crash occur after the kernel mounts filesystem. It never crashes before mounting filesystem. I have also tested RAM using tool provided by NVIDIA and also tested RAM using stress utility, both tests complete successfully. So it seems there is no issue with RAM. Do anyone have any idea, I am sharing some of dumps here

[   13.044646] Internal error: Oops - undefined instruction: 0 [#1] PREEMPT SMP ARM
[   13.044654] Modules linked in:
[   13.044668] CPU: 0 PID: 40 Comm: khubd Tainted: G        W    3.10.40 #60
[   13.044674] task: ed53dac0 ti: ed554000 task.ti: ed554000
[   13.044692] PC is at hub_thread+0x64/0x174
[   13.044698] LR is at hub_thread+0x144/0x174
[   13.044704] pc : [<c04c00a0>]    lr : [<c04c0180>]    psr: 60030113
[   13.044704] sp : ed555f38  ip : 00000000  fp : 00000000
[   13.044706] r10: 00000000  r9 : 00000000  r8 : c008da64
[   13.044709] r7 : c0cad044  r6 : c0d310c8  r5 : ed554000  r4 : c0cad03c
[   13.044711] r3 : 40200040  r2 : ed53dac0  r1 : 20030113  r0 : c0cad044
[   13.044716] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment kernel
[   13.044719] Control: 10c5387d  Table: acc6006a  DAC: 00000015
[   13.044723] 
[   13.044723] PC: 0xc04c0020:
[   13.044724] 0020  c0cf6e52 c0cf6daa c0cf6e22 c0cf6dc2 c0cf6e0a c0cf6db0 c0cf6e10 e92d41f0
[   13.044738] 0040  e24dd018 e52de004 e8bd4000 e30d403c e1a0300d e3c35d7f e34c40ca e30160c8
[   13.044748] 0060  e30d8a64 e2847008 e3c5503f e34c60d3 e34c8008 ebefd422 ebfffd79 e595200c
[   13.044759] 0080  e5941000 e592300c e1510004 e3833101 e582300c 0a00001d e595200c e592300c
[   13.044770] 00a0  ed53edcc ed555f88 00000000 00000000 e3c33101 e582300c f57ff05f e5963000
[   13.044781] 00c0  e3530000 1a00000e ebef324e e3500000 0affffec e5943000 e1530004 1affffe9
[   13.044792] 00e0  e3130004 1a000026 e3a00000 e28dd018 e8bd81f0 e595000c ebefd39b e3500000
[   13.044803] 0100  0affffec e3a00000 ebefd3b2 eaffffe9 ebef3238 e3500000 1affffde e595200c

.

[   12.020636] tegra-udc tegra-udc.0: USB cable/charger disconnected
[   12.020996] otg state changed: SUSPEND --> HOST
[   12.021715] tegra USB phy - inst[0] platform info:
[   12.021719] port_otg: yes
[   12.021722] has_hostpc: yes
[   12.021725] phy_interface: USB_PHY_INTF_UTMI
[   12.021728] op_mode: TEGRA_USB_OPMODE_HOST
[   12.021731] qc2_voltage: 0
[   12.021734] vbus_gpio: -1
[   12.021739] hot_plug: disabled
[   12.021743] remote_wakeup: enabled
[   12.060761] mc-err: [mcerr] (ppcs) csw_ppcsahbslvw: EMEM decode error on PDE or PTE entry
[   12.060770] mc-err: [mcerr]   status = 0x6003203c; addr = 0x7c0c9880[   12.060779] mc-err: [mcerr]   ss
[   12.062219] mc-err: [mcerr] (ppcs) csw_ppcsahbslvw: EMEM decode error on PDE or PTE entry
[   12.062226] mc-err: [mcerr]   status = 0x6003203c; addr = 0x7c0c9880[   12.062231] mc-err: [mcerr]   ss
[   12.065313] mc-err: [mcerr] (ppcs) csw_ppcsahbslvw: EMEM decode error on PDE or PTE entry
[   12.065320] mc-err: [mcerr]   status = 0x6003203c; addr = 0x7c0c9880[   12.065326] mc-err: [mcerr]   ss
[   12.065913] mc-err: [mcerr] (ppcs) csw_ppcsahbslvw: EMEM decode error on PDE or PTE entry
[   12.065919] mc-err: [mcerr]   status = 0x6003203c; addr = 0x7c0c9880[   12.065925] mc-err: [mcerr]   ss
[   12.071274] Unable to handle kernel paging request at virtual address 00043884
[   12.071280] pgd = ec534000
[   12.071284] [00043884] *pgd=ac4b0831, *pte=00000000, *ppte=00000000
[   12.071305] Internal error: Oops: 17 [#1] PREEMPT SMP ARM
[   12.071311] Modules linked in:
[   12.071325] CPU: 1 PID: 262 Comm: nvfb.sh Tainted: G        W    3.10.40 #60
[   12.071331] task: eca0ea80 ti: ec4a2000 task.ti: ec4a2000
[   12.071351] PC is at part_round_stats_single+0x58/0x7c
[   12.071356] LR is at 0x1
[   12.071364] pc : [<c029a24c>]    lr : [<00000001>]    psr: 00000113
[   12.071364] sp : ec4a3c30  ip : fffb9b22  fp : c01bce5c
[   12.071368] r10: ece39e34  r9 : 00000003  r8 : ed06c400
[   12.071373] r7 : 00000001  r6 : fffb9b27  r5 : 00000001  r4 : 0000010a
[   12.071377] r3 : 00043860  r2 : fffb9b27  r1 : ed0a4c40  r0 : 0125d000
[   12.071386] Flags: nzcv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment kernel
[   12.071392] Control: 10c5387d  Table: ac53406a  DAC: 00000015
[   12.071398] 
[   12.071398] PC: 0xc029a1cc:
[   12.071402] a1cc  e3a06000 eafffff5 c0d6f168 e52de004 e8bd4000 e590305c e5930214 e3500000
[   12.071429] a1ec  128000d0 e12fff1e e92d4070 e52de004 e8bd4000 e591c1b0 e15c0002 08bd8070
[   12.071456] a20c  e59131b4 e591e1b8 e173000e 0a000012 e30a44e8 e59131bc e34c40bd e591e1b4
[   12.071482] a22c  e7940100 e042c00c e59151b8 e0833000 e5934024 e042c00c e59151b8 e0833000
[   12.071506] a24c  e5934024 e08ee005 e02c4e9c e583c024 e59131bc e080000c e5830020 e58121b0
[   12.071532] a26c  e8bd8070 e52de004 e8bd4000 e5903020 e1a01000 e590001c e3c33702 e5813020
[   12.071558] a28c  e590305c e3530000 012fff1e e12fff13 e52de004 e8bd4000 e5903070 e3530000
[   12.071585] a2ac  0a000000 e12fff13 e1a00003 e12fff1e e52de004 e8bd4000 e1a0300d e307136

.

[   11.059198] systemd-udevd[148]: starting version 215                         
[   11.072135] systemd[1]: Mounting /mnt/ramdisk...                             
[   11.086343] systemd[1]: Starting Syslog Socket.                              
[   11.097780] systemd[1]: Listening on Syslog Socket.                          
[   11.105728] systemd[1]: Starting Sockets.                                    
[   11.115019] systemd[1]: Reached target Sockets.                              
[   11.122357] systemd[1]: Starting Journal Service...                          
[   11.140562] systemd[1]: Started Journal Service.                             
[   11.551550] systemd-journald[158]: Received request to flush runtime journal1
[   11.862564] Tegra cpuquiet clusterswitch enabled                             
[   11.882266] *                                                                
[   11.898914] -[   11.936666] otg state changed: PERIPHERAL --> SUSPEND        
[   11.936836] tegra-udc tegra-udc.0: USB cable/charger disconnected            
[   11.937227] otg state changed: SUSPEND --> HOST                              
[   11.938022] tegra USB phy - inst[0] platform info:                           
[   11.938029] port_otg: yes                                                    
[   11.938036] has_hostpc: yes                                                  
[   11.938042] phy_interface: USB_PHY_INTF_UTMI                                 
[   11.938048] op_mode: TEGRA_USB_OPMODE_HOST                                   
[   11.938055] qc2_voltage: 0                                                   
[   11.938061] vbus_gpio: -1                                                    
[   11.938068] hot_plug: disabled                                               
[   11.938074] remote_wakeup: enabled                                           
[   11.993244] *[   12.147798] as3722-adc-extcon as3722-adc-extcon.2: USB-Host d
[   12.281049] Internal error: Oops - undefined instruction: 0 [#1] PREEMPT SMPM

[   12.292012] Modules linked in:                                               
[   12.298540] CPU: 0 PID: 309 Comm: cpufreq-set Tainted: G        W    3.10.400
[   12.309143] task: ec953ac0 ti: ec582000 task.ti: ec582000                    
[   12.316158] PC is at 0xec583d40                                              
[   12.320913] LR is at tegra_update_cpu_speed+0x128/0x2f0                      
[   12.327804] pc : [<ec583d40>]    lr : [<c004b1bc>]    psr: 80000013          
[   12.327804] sp : ec583d08  ip : 00000000  fp : ed6bdd80                      
[   12.342746] r10: c1df5568  r9 : 00000000  r8 : 001f845c                      
[   12.349718] r7 : c004b6e0  r6 : 001f845c  r5 : 000a9ec0  r4 : 00000000       
[   12.358037] r3 : 00000004  r2 : 00000000  r1 : c1dfd37a  r0 : 00000000       
[   12.366367] Flags: Nzcv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
[   12.375367] Control: 10c5387d  Table: ac8e006a  DAC: 00000015                
[   12.383008]                                                                  
[   12.383008] PC: 0xec583cc0:                                                  
[   12.390900] 3cc0  00000000 c1dfd37a 00000000 00000004 00000000 000a9ec0 001f0
[   12.403102] 3ce0  001f845c 00000000 c1df5568 ed6bdd80 00000000 ec583d08 c0040
[   12.415515] 3d00  80000013 ffffffff 001f845c 00000001 c0cee138 c004b758 c0d00
[   12.428162] 3d20  ed6bdd80 c004bd3c ec583d30 00000002 00000014 001f845c 00004
[   12.440884] 3d40  001f845c edaf2ac0 001f845c c05a73a0 00000003 ed6bdd80 00001
[   12.453901] 3d60  c0ba1568 00000004 00000064 edaf2ac0 c0bd9cbc ed6bdd80 00008
[   12.467055] 3d80  00000000 c05aff80 00000000 c1e10568 edaf2b80 c0b9fa78 c0bd0
[   12.480321] 3da0  00bae062 00000000 c0bd9cbc c1df55d8 edaf2ac0 ed6bdd80 edaf8
[   12.493747]                                                                  
[   12.493747] LR: 0xc004b13c:                                                  
[   12.502907] b13c  0a000060 e596300c e3530002 0a000009 e30632b0 e3a01002 e34c0
[   12.516329] b15c  ebffa872 e2508000 1a00004f e59d0008 e3a03002 e586300c e59d0
[   12.529856] b17c  e1530000 0a00003a e1500003 8a000051 e30b3760 e3e02000 e34c0
[   12.543566] b19c  e3094cbc e5935000 e5282010 e34c40bd ea000003 e3a01000 e1a0f
[   12.557385] b1bc  e59d2000 e2822001 e3a01004 e1a00005 eb098793 e5943000 e58d0
[   12.571207] b1dc  8afffff3 e59d3008 e3a01ffa e5960024 e0010391 ebff75a9 e2500
[   12.585024] b1fc  028d8010 05282010 0a000004 ea000053 e3a01001 e1a00008 eb150
[   12.598840] b21c  e2822001 e3a01004 e1a00005 eb09

Thanks

  • Try with an other kernel. To me, it seems an hardware problem [or check new firmware]. Change memory, check cables, check power, check if other machines will have the same problem with such image. – Giacomo Catenazzi Feb 13 at 11:43
  • i have tested on 3 boards and getting the same problem. and for the hardware, what interfaces should i check. as for the ram and flash, i tested different tests and they are passing it. – faiz07 Feb 13 at 12:23
  • The fact this is intermittent hints that it is hardware not software. The fact it fails right after mount hints at whatever holds the file system (an SD card?). First thing would be to swap the SD card. – Philip Couling Feb 13 at 12:36
  • we are using eMMC for storing filesystem. but kernel and uboot is also there. and we never get problem in uboot. – faiz07 Feb 13 at 12:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.