[PROBLEM] btrfs: mmcblk0p2 checksum verify failed

[ErickP]

Hi,
I was using Raspbmc and switched to Xbian 1.0 Beta 2 last night. I dumped the .img file with dd to the SD and booted from it.
For about an hour all goes well, then I try to perform any operations on the SD card (update, install, mv, cp something) and the the SD card is remounted read only.

Terminal

[ 13.471196] smsc95xx 1-1.1:1.0 eth0: link up, 100Mbps, full-duplex, lpa 0x5DE1
[ 17.106256] init: failsafe main process (454) killed by TERM signal
[ 17.470330] udevd[732]: starting version 175
[ 20.247073] mousedev: PS/2 mouse device common for all mice
[ 21.446950] device label xbian-beta2 devid 1 transid 47 /dev/mmcblk0p2
[ 36.133875] init: wait-for-state (xbmcxbmc-loaded) main process (780) killed by TERM signal
[ 43.505730] init: wait-for-state (zram-swapxbmc-done) main process (427) killed by TERM signal
[ 43.575642] init: xbmc-failed-start main process (459) killed by TERM signal
[ 45.075484] zram: module is from the staging directory, the quality is unknown, you have been warned.
[ 45.102104] zram: Created 4 device(s) ...
[ 45.615636] Adding 63484k swap on /dev/zram0. Priority:20 extents:1 across:63484k SS
[ 46.486324] Adding 251388k swap on /dev/mmcblk0p3. Priority:0 extents:1 across:251388k SS
[ 46.993206] device label xbian-beta2 devid 1 transid 48 /dev/mmcblk0p2
[ 48.279325] device label xbian-beta2 devid 1 transid 48 /dev/mmcblk0p2
[ 1301.112974] device label xbian-beta2 devid 1 transid 74 /dev/mmcblk0p2
[ 1370.701022] device label xbian-beta2 devid 1 transid 77 /dev/mmcblk0p2
[ 1371.517619] device label xbian-beta2 devid 1 transid 78 /dev/mmcblk0p2
[ 1372.946116] device label xbian-beta2 devid 1 transid 79 /dev/mmcblk0p2
[ 1593.320701] btrfs bad tree block start 225039243880169532 73302016
[ 1593.321970] btrfs: mmcblk0p2 checksum verify failed on 73285632 wanted A6F31876 found 110AA0E8 level 0
[ 1593.322051] ------------[ cut here ]------------
[ 1593.322073] WARNING: at fs/btrfs/super.c:254 0xc0154710()
[ 1593.322084] btrfs: Transaction aborted (error -5)
[ 1593.322093] Modules linked in: zram© frandom joydev mousedev rpcsec_gss_krb5
[ 1593.322129] CPU: 0 PID: 7938 Comm: dpkg Tainted: G C 3.10.12+ #61
[ 1593.322138] Backtrace:
[ 1593.322149] Function entered at [<c0011300>] from [<c0011490>]
[ 1593.322158] r6:c0154710 r5:00000009 r4:c7ebfad8 r3:e8fc311a
[ 1593.322173] Function entered at [<c0011478>] from [<c03f7c88>]
[ 1593.322184] Function entered at [<c03f7c68>] from [<c001d5ec>]
[ 1593.322192] Function entered at [<c001d598>] from [<c001d6ac>]
[ 1593.322200] r8:c0405b50 r7:00001586 r6:ce752400 r5:cf4bb000 r4:fffffffb
[ 1593.322200] r3:00000009
[ 1593.322220] Function entered at [<c001d674>] from [<c0154710>]
[ 1593.322227] r3:fffffffb r2:c0443274
[ 1593.322238] Function entered at [<c015466c>] from [<c0169820>]
[ 1593.322245] r8:cf4bb000 r7:cf60daa0 r6:fffffffb r5:00000000 r4:00000000
[ 1593.322262] Function entered at [<c016913c>] from [<c016dd78>]
[ 1593.322272] Function entered at [<c016d4bc>] from [<c0172824>]
[ 1593.322280] Function entered at [<c0172770>] from [<c018659c>]
[ 1593.322289] Function entered at [<c01864d0>] from [<c01867cc>]
[ 1593.322297] Function entered at [<c01867b8>] from [<c019723c>]
[ 1593.322307] Function entered at [<c01971d4>] from [<c00b85e8>]
[ 1593.322367] Function entered at [<c00b855c>] from [<c00bb378>]
[ 1593.322378] r6:c7ebff60 r5:c3d1b6e8 r4:c7ebfee0
[ 1593.322393] Function entered at [<c00babb8>] from [<c00bb7e8>]
[ 1593.322402] Function entered at [<c00bb734>] from [<c00bbe84>]
[ 1593.322410] Function entered at [<c00bbe50>] from [<c00ae134>]
[ 1593.322418] r7:ffffff9c r6:0000000a r5:ce54e000 r4:000200c1
[ 1593.322467] Function entered at [<c00ae048>] from [<c00ae1e8>]
[ 1593.322479] Function entered at [<c00ae1c4>] from [<c000e320>]
[ 1593.322488] ---[ end trace ec721da10c684c5e ]---
[ 1593.322503] BTRFS error (device mmcblk0p2) in __btrfs_free_extent:5510: errno=-5 IO failure
[ 1593.322520] BTRFS info (device mmcblk0p2): forced readonly
[ 1593.322540] BTRFS debug (device mmcblk0p2): run_one_delayed_ref returned -5
[ 1593.322562] BTRFS error (device mmcblk0p2) in btrfs_run_delayed_refs:2630: errno=-5 IO failure
[ 1594.525340] init: Failed to write to log file /var/log/upstart/xbmc.log
xbian@xbian /var/log $

Output from mount command:

Terminal

xbian@xbian /var/log $ mount
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,noatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,noatime)
udev on /dev type devtmpfs (rw,relatime,size=256k,nr_inodes=31069,mode=755)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620)
tmpfs on /run type tmpfs (rw,nosuid,noexec,noatime,size=25416k,mode=755)
/dev/mmcblk0p2 on / type btrfs (ro,relatime,compress=lzo,ssd,space_cache,autodefrag)
/dev/mmcblk0p1 on /boot type vfat (rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,​errors=remount-ro)
/dev/mmcblk0p2 on /home type btrfs (ro,noatime,compress=lzo,ssd,space_cache,autodefrag)
/dev/mmcblk0p2 on /lib/modules type btrfs (ro,noatime,compress=lzo,ssd,space_cache,autodefrag)
/dev/mmcblk0p2 on /xbmc-backup type btrfs (ro,noatime,compress=lzo,ssd,space_cache,autodefrag)

Being somehow new to btrfs can you please let me know how to FSCK/CHECK my SD? check the extents or scan for bad blocks? what i found interesting is that i took the SD out of the pi, reformatted it under windows and ran an extremely long scan for bad sectors. There is a possibility that the SD has gone wrong but as of this moment I only have a 2GB usb around to test.

Thanks for your comments.

[mk01]

do you have anything connected in to the RPI ports without external power adapter (other as the usb mouse)?

seeing the boot time you don't have it even clocked higher than the default.

normally OC problem or memory or similar would just report lots of checksum errors, doing RO remount at the end. but you also have two unfinished IO operations aborted with error. this could mean SD card failing.

btrfs currently doesn't implement classical fsck, but … ok I'm not right anymore. checked "btrfs" man page currently and there are more options:
1) btrfs check [options] <device>
Check an unmounted btrfs filesystem.

- so you could drop to recovery shell (initramfs) and do the check on unmounted fs. you can invoke recovery shell by holding left shift key during initramfs stage, or by putting "rescue" into cmdline.txt

2) btrfs scrub start [-BdqrR] [-c ioprio_class -n ioprio_classdata] <path>|<device>
Start a new scrub
btrfs scrub cancel <path>|<device>
Cancel a running scrub
btrfs scrub resume [-BdqrR] [-c ioprio_class -n ioprio_classdata] <path>|<device>
Resume previously canceled or interrupted scrub
btrfs scrub status [-dR] <path>|<device>
Show status of running or finished scrub

- scrub is supposed for online run, should recheck all data against existing checksums.

this kind of errors are very rare since XBian went for btrfs (few month back). maybe I remember three or four. I experienced it once when clocked my RPI to 980,500,500 (normally running 920,420,500).

report back please.

[ErickP]

(3rd Dec, 2013 03:16 AM)mk01 Wrote:  do you have anything connected in to the RPI ports without external power adapter (other as the usb mouse)?

seeing the boot time you don't have it even clocked higher than the default.

normally OC problem or memory or similar would just report lots of checksum errors, doing RO remount at the end. but you also have two unfinished IO operations aborted with error. this could mean SD card failing.

btrfs currently doesn't implement classical fsck, but … ok I'm not right anymore. checked "btrfs" man page currently and there are more options:
1) btrfs check [options] <device>
Check an unmounted btrfs filesystem.

- so you could drop to recovery shell (initramfs) and do the check on unmounted fs. you can invoke recovery shell by holding left shift key during initramfs stage, or by putting "rescue" into cmdline.txt

2) btrfs scrub start [-BdqrR] [-c ioprio_class -n ioprio_classdata] <path>|<device>
Start a new scrub
btrfs scrub cancel <path>|<device>
Cancel a running scrub
btrfs scrub resume [-BdqrR] [-c ioprio_class -n ioprio_classdata] <path>|<device>
Resume previously canceled or interrupted scrub
btrfs scrub status [-dR] <path>|<device>
Show status of running or finished scrub

- scrub is supposed for online run, should recheck all data against existing checksums.

this kind of errors are very rare since XBian went for btrfs (few month back). maybe I remember three or four. I experienced it once when clocked my RPI to 980,500,500 (normally running 920,420,500).

report back please.

One thing to note: As I was reading how to boot from USB, my /boot/cmdline.txt contains:

Quote: mod_scsi.scan=sync

instead of

Quote: mod_scsi.scan=async

the SD is supposed to be "async" right?

---

(3rd Dec, 2013 03:54 AM)ErickP Wrote:  

(3rd Dec, 2013 03:16 AM)mk01 Wrote:  do you have anything connected in to the RPI ports without external power adapter (other as the usb mouse)?

seeing the boot time you don't have it even clocked higher than the default.

normally OC problem or memory or similar would just report lots of checksum errors, doing RO remount at the end. but you also have two unfinished IO operations aborted with error. this could mean SD card failing.

btrfs currently doesn't implement classical fsck, but … ok I'm not right anymore. checked "btrfs" man page currently and there are more options:
1) btrfs check [options] <device>
Check an unmounted btrfs filesystem.

- so you could drop to recovery shell (initramfs) and do the check on unmounted fs. you can invoke recovery shell by holding left shift key during initramfs stage, or by putting "rescue" into cmdline.txt

2) btrfs scrub start [-BdqrR] [-c ioprio_class -n ioprio_classdata] <path>|<device>
Start a new scrub
btrfs scrub cancel <path>|<device>
Cancel a running scrub
btrfs scrub resume [-BdqrR] [-c ioprio_class -n ioprio_classdata] <path>|<device>
Resume previously canceled or interrupted scrub
btrfs scrub status [-dR] <path>|<device>
Show status of running or finished scrub

- scrub is supposed for online run, should recheck all data against existing checksums.

this kind of errors are very rare since XBian went for btrfs (few month back). maybe I remember three or four. I experienced it once when clocked my RPI to 980,500,500 (normally running 920,420,500).

report back please.

One thing to note: As I was reading how to boot from USB, my /boot/cmdline.txt contains:

Quote: mod_scsi.scan=sync

instead of

Quote: mod_scsi.scan=async

the SD is supposed to be "async" right?

Sorry forgot to mention that the pi is powered by an external 2amps usb hub. and my usb hard disk is 1tb in size, also connected to the external usb hub.

[mk01]

(3rd Dec, 2013 03:54 AM)ErickP Wrote:  Sorry forgot to mention that the pi is powered by an external 2amps usb hub. and my usb hard disk is 1tb in size, also connected to the external usb hub.

mod_scsi sync/async is related to usb bus scanning, not to operations. sync mode will simply cause kernel loading to wait for devices, async will not.

if you want to try USB stick don't follow USB how-tos related to XBian's prior B2. just flash img file to usb stick and change cmdline.txt root=/dev/sda2.