[SOLVED] After updating yesterday...

[Peiote]

Software:
Not sure about the Software, since I updated yesterday.

Hardware:
Device type and model: Raspberry Pi 3
Power supply rating: 1,8 Amp
SD card size and make/type: 16 GB
Network (Ethernet or wireless): Ethernet
Connected devices (TV, USB, network storage, ...):


Yesterday I updated, I've been many days working fine without updating.
After the update, yesterday it worked aparently well.
Today I couldn't start kodi, so I reboot... and now the problem:

Cause and solution?

[Nachteule]

Ok, bad news for you, your root fs seems to be damaged completely.

No idea what happened, maybe sd card corrupt or broken.

IMO there is only one solution: reinstall system. Hopefully you have backup, if not - your bad.

For the future (I'm sure it never happpens again) make image backup from time to time so you can restore your system completely (see xbian-config -> xbian copier)

[Peiote]

I think I have backups.

Do you think this could be a hardware damage of the microSD Card?
In such case it could happen many times :m

[Nachteule]

Hard do say. I got similar issue one time also at my i.MX6/Hummingbird board. Damaged sd card was my first thought, but image restored (if have one) and never happened again for months.

[duno]

(7th May, 2016 11:38 PM)Nachteule Wrote:  No idea what happened, maybe sd card corrupt or broken.

I do not think so, i've got the same problem

[BTRFS: superblock checksum mismatch]

After an update, on monday, xbian will not boot anymore.

Today i checked the sd-card in my computer and can't mount the btrfs-partition anymore.

Code:

dmesg | grep -i BTRF
...
[    5.395139] Btrfs loaded
[   14.818661] BTRFS: device label xbian devid 1 transid 82234 /dev/sdc2
[  177.585323] BTRFS: superblock checksum mismatch
[  177.590994] BTRFS: open_ctree failed

Code:

sudo btrfs-show-super /dev/sdc2

Code:

superblock: bytenr=65536, device=/dev/sdc2
---------------------------------------------------------
csum            0x323e98ec [DON'T MATCH]
bytenr            65536
flags            0x1
            ( WRITTEN )
magic            _BHRfS_M [match]
...

Code:

sudo btrfsck -s 2 /dev/sdc2
using SB copy 2, bytenr 274877906944
ERROR: superblock checksum mismatch
Superblock bytenr is larger than device size
Couldn't open file system

Code:

sudo btrfsck -s 1 /dev/sdc2
using SB copy 1, bytenr 67108864
ERROR: superblock checksum mismatch
couldn't open because of unsupported option features (10).
Couldn't open file system

Any chance to recover the system?

[Nachteule]

Don't know, I'm really not BTRFS expert for such internals

For me it looks like completely damaged file system. Hopefully you have backup of your system

[Nachteule]

@duno

can you remember which kernel version you were running?

[duno]

(11th Aug, 2016 03:48 AM)Nachteule Wrote:  @duno

can you remember which kernel version you were running?

Latest, 4.4.15+ and update was to 4.4.16+

Code:

btrfs-show-super -a /dev/sdc2
superblock: bytenr=65536, device=/dev/sdc2
---------------------------------------------------------
csum            0x323e98ec [DON'T MATCH]
bytenr            65536
flags            0x1
            ( WRITTEN )
magic            _BHRfS_M [match]
fsid            2dcbf32f-7a95-41ff-bf52-65d5b4f547a5
label            xbian
generation        82234
root            33685504
sys_array_size        97
chunk_root_generation    49679
root_level        1
chunk_root        147456
chunk_root_level    0
log_root        0
log_root_transid    0
log_root_level        0
total_bytes        31111249920
bytes_used        5725069312
sectorsize        4096
nodesize        16384
leafsize        16384
stripesize        4096
root_dir        6
num_devices        1
compat_flags        0x0
compat_ro_flags        0x0
incompat_flags        0x179
            ( MIXED_BACKREF |
              COMPRESS_LZO |
              COMPRESS_LZOv2 |
              BIG_METADATA |
              EXTENDED_IREF |
              SKINNY_METADATA |
              unknown flag: 0x10 )
csum_type        0
csum_size        4
cache_generation    82233
uuid_tree_generation    82232
dev_item.uuid        70322777-7343-4491-ae65-295472f29d6d
dev_item.fsid        2dcbf32f-7a95-41ff-bf52-65d5b4f547a5 [match]
dev_item.type        0
dev_item.total_bytes    31111249920
dev_item.bytes_used    8896118784
dev_item.io_align    4096
dev_item.io_width    4096
dev_item.sector_size    4096
dev_item.devid        1
dev_item.dev_group    0
dev_item.seek_speed    0
dev_item.bandwidth    0
dev_item.generation    0

superblock: bytenr=67108864, device=/dev/sdc2
---------------------------------------------------------
csum            0x0f19fcb1 [match]
bytenr            67108864
flags            0x1
            ( WRITTEN )
magic            _BHRfS_M [match]
fsid            2dcbf32f-7a95-41ff-bf52-65d5b4f547a5
label            xbian
generation        82233
root            23609344
sys_array_size        97
chunk_root_generation    49679
root_level        1
chunk_root        147456
chunk_root_level    0
log_root        0
log_root_transid    0
log_root_level        0
total_bytes        31111249920
bytes_used        5725073408
sectorsize        4096
nodesize        16384
leafsize        16384
stripesize        4096
root_dir        6
num_devices        1
compat_flags        0x0
compat_ro_flags        0x0
incompat_flags        0x179
            ( MIXED_BACKREF |
              COMPRESS_LZO |
              COMPRESS_LZOv2 |
              BIG_METADATA |
              EXTENDED_IREF |
              SKINNY_METADATA |
              unknown flag: 0x10 )
csum_type        0
csum_size        4
cache_generation    82233
uuid_tree_generation    82232
dev_item.uuid        70322777-7343-4491-ae65-295472f29d6d
dev_item.fsid        2dcbf32f-7a95-41ff-bf52-65d5b4f547a5 [match]
dev_item.type        0
dev_item.total_bytes    31111249920
dev_item.bytes_used    8896118784
dev_item.io_align    4096
dev_item.io_width    4096
dev_item.sector_size    4096
dev_item.devid        1
dev_item.dev_group    0
dev_item.seek_speed    0
dev_item.bandwidth    0
dev_item.generation    0

got a match on

Code:

csum            0x0f19fcb1 [match]
bytenr            67108864
flags            0x1

Code:

btrfs-show-super -s 67108864 /dev/sdc2
superblock: bytenr=67108864, device=/dev/sdc2
---------------------------------------------------------
csum            0x0f19fcb1 [match]
bytenr            67108864
flags            0x1
            ( WRITTEN )
magic            _BHRfS_M [match]
fsid            2dcbf32f-7a95-41ff-bf52-65d5b4f547a5
label            xbian
generation        82233
root            23609344
sys_array_size        97
chunk_root_generation    49679
root_level        1
chunk_root        147456
chunk_root_level    0
log_root        0
log_root_transid    0
log_root_level        0
total_bytes        31111249920
bytes_used        5725073408
sectorsize        4096
nodesize        16384
leafsize        16384
stripesize        4096
root_dir        6
num_devices        1
compat_flags        0x0
compat_ro_flags        0x0
incompat_flags        0x179
            ( MIXED_BACKREF |
              COMPRESS_LZO |
              COMPRESS_LZOv2 |
              BIG_METADATA |
              EXTENDED_IREF |
              SKINNY_METADATA |
              unknown flag: 0x10 )
csum_type        0
csum_size        4
cache_generation    82233
uuid_tree_generation    82232
dev_item.uuid        70322777-7343-4491-ae65-295472f29d6d
dev_item.fsid        2dcbf32f-7a95-41ff-bf52-65d5b4f547a5 [match]
dev_item.type        0
dev_item.total_bytes    31111249920
dev_item.bytes_used    8896118784
dev_item.io_align    4096
dev_item.io_width    4096
dev_item.sector_size    4096
dev_item.devid        1
dev_item.dev_group    0
dev_item.seek_speed    0
dev_item.bandwidth    0
dev_item.generation    0

But how to restore that?

Code:

bytenr            67108864

is normally "s 1" but

Code:

btrfsck -s 1 /dev/sdc2

or

Code:

btrfs check -s 1 /dev/sdc2

gives an

Code:

using SB copy 1, bytenr 67108864
ERROR: superblock checksum mismatch
couldn't open because of unsupported option features (10).
Couldn't open file system

I do not understand this btrfs-stuff

[Nachteule]

(11th Aug, 2016 05:44 AM)duno Wrote:  Latest, 4.4.15+ and update was to 4.4.16+

Ok, so no 4.6.x from devel repo. I'm using 4.6.5 and got strange BTRFS error today afternoon. But here it was not BTRFS culprit, it was mmc driver fault. That's why I was asking

Quote:I do not understand this btrfs-stuff

It seems that btrfs fsck is still very crappy.

[duno]

(11th Aug, 2016 06:37 AM)Nachteule Wrote:  It seems that btrfs fsck is still very crappy.

So why are you using btrfs?



When i remember correct "feature 10" was the lz4-compression for btrfs in the kernel.

I fixed that by building my own kernel with that support:

http://forum.xbian.org/thread-3642-post-32401.html

(4.4.14-btrfs01 #1 SMP PREEMPT Sat ...)

Yesterday i downloaded the latest image for xbian and made a dd to an usb-stick, because i thought it i also using the latest firmware to be able to boot the rpi3 via usb.

(it seems to be not ...)

I was wondering, that kernel 4.4.0-34-lowlatency in ubuntu-studio was also able to mount that btrfs-partitition.

Did you or ubuntu changed something with the btrfs-stuff?

btw. this is not the first time that xbian crashed with a btrfs error but the first time with an "nearly" productive system



Never got these problems on raspian (wheezy) on the PI-B nor on ubuntu (xenial) on the Pi-3

[Nachteule]

(11th Aug, 2016 07:49 AM)duno Wrote:  So why are you using btrfs?

IMO it has more pro's than con's

BTW, I never had btrfs data corruption under normal condition [1], and I'm using btrfs on nearly all my systems

Quote:When i remember correct "feature 10" was the lz4-compression for btrfs in the kernel.

Don't know

Quote:I fixed that by building my own kernel with that support:

http://forum.xbian.org/thread-3642-post-32401.html

(4.4.14-btrfs01 #1 SMP PREEMPT Sat ...)

Yes, I remember this thread

Quote:Yesterday i downloaded the latest image for xbian and made a dd to an usb-stick, because i thought it i also using the latest firmware to be able to boot the rpi3 via usb.

???

Quote:I was wondering, that kernel 4.4.0-34-lowlatency in ubuntu-studio was also able to mount that btrfs-partitition.

Did you or ubuntu changed something with the btrfs-stuff?

OC, no change. This is normal, image is build with lzo compression, compression is set to lz4 in /boot/cmdline.txt only
So, if you do not like to use lz4, you can change this before you boot that image the first time

Quote:btw. this is not the first time that xbian crashed with a btrfs error but the first time with an "nearly" productive system

Seems there is going something wrong with your system. Maybe you have to change sd-card. When I got first RPi2, I struggled a long time with heavy system instability. After I changed everything I could change (I used 3 identical brand new sd-cards from samsung) I bought another one (from samsung too, but latest EVO model), and after this all problems were gone.

Quote:Never got these problems on raspian (wheezy) on the PI-B nor on ubuntu (xenial) on the Pi-3

...

[1]
-I got btrfs fs crash some years ago on one of my systems I was undervolting CPU and this did not like btrfs.
-I got another btrfs crash about one year ago on my notebook because of too aggressive power saving settings and too old and buggy kernel (was 3.14)

That's all, never had critcal btrfs errors on my 5 XBian devices (2 of them are used for developing)

[f1vefour]

(11th Aug, 2016 08:37 AM)Nachteule Wrote:  

Quote:Yesterday i downloaded the latest image for xbian and made a dd to an usb-stick, because i thought it i also using the latest firmware to be able to boot the rpi3 via usb.

???

He was saying the latest image isn't using the latest firmware, else USB booting would be possible.

[Nachteule]

(11th Aug, 2016 12:51 PM)f1vefour Wrote:  

(11th Aug, 2016 08:37 AM)Nachteule Wrote:  

Quote:Yesterday i downloaded the latest image for xbian and made a dd to an usb-stick, because i thought it i also using the latest firmware to be able to boot the rpi3 via usb.

???

He was saying the latest image isn't using the latest firmware, else USB booting would be possible.

Sure, I understood what he was writing, but I don't see any relationship to this thread here. Booting from USB should work in all cases (ok, never needed and never tried, but very well described here)

That's why I was confused ...

[duno]

(11th Aug, 2016 10:54 PM)Nachteule Wrote:  ... , but I don't see any relationship to this thread here. Booting from USB ...

Bootable USB-Stick for the RPI3 to repair RPI2 broken btrfs-Partition

I thought, that the broken btrfs-partition, on the RPI2, is an Xbian-based problem.

Therfore my idea was, to install Xbian on an USB-Stick, booting Xbian from that USB-Stick under an RPI3 and trying to fix the problem on the SD-Card ...

Maybe my idea was wrong and the second partition, on the SD-Kard for the RPI2, is "toasted".

I am not sure about that after the "match".

Is there or isn't there a chance to restore the matching Superblock?

[Nachteule]

(14th Aug, 2016 01:52 AM)duno Wrote:  Therfore my idea was, to install Xbian on an USB-Stick, booting Xbian from that USB-Stick under an RPI3 and trying to fix the problem on the SD-Card ..

Why soooo complicated?

Why not installing fresh XBian and put your broken sd-card into an usb adapter (I'm sure you have one ) and plug it into one USB port of your RPi?
Ok, you need a second sd-card to do that ...

Quote:Is there or isn't there a chance to restore the matching Superblock?

To answer a question with a question: does it make sense to spend hours and hours for repairing that damaged fs? Does it not make more sense installing XBian from scratch and after finishing with configuration making an image backup?

I wouldn't do that (but I know it is easy to say, I'm having 2 different types of backup (image and rsync copy), I would never come into this situation...)