btrfs and lz4

[mk01]

BTRFS with LZ4 compression

I patched kernel/btrfs to support LZ4 compression (as addition to default LZO and ZLIB).
kernel package is at

http://xbian.brantje.com/others/xbian-package-kernel1.3-6.11b-lz4.deb

despite the offset -lz4 package version is still matching the last kernel posted to devel repo. although the original patches are old and were developed for 2.xx kernels, it was possible to rebase it to actual kernels.

it is running for me with no issues for 48h, so I don't expect issues there. just one thing to consider before installing and changing "compress=lzo" to "compress=lz4".

it will "mark" the filesystem with additional "feature flag" making it impossible to mount the volume on kernel without this patch - that means RPI is fine, problem would be moving the SD card to another linux with BTRFS with expectation to mount it (after patching the kernel no problem again - tried with ubuntu-saucy kernel 3.10.11.62 - patch is at xbian-patches repo, in kernel directory).

awaiting some reports as I would really consider making this as default.

I expect the actual LZ4 implementation being a bit old, maybe it would be possible to match it to actual sources. will check that.

[CurlyMo]

Benchmarks / Differences in user experience?

[mk01]

don't have (and don't plan benchs as general ones are as always on google) but lz4 was highly demanded few years back by zfs community and running apt-get install on XBian reminds more to ext4 times again.

speaking general facts, compress ratio is not suffering much (although our target was more "bandwidth" related than space saving), resource allocation goes minimal (lz4 by default works with minimal buffers - it works only with 8K internal buffer!) and decompression speed is more comparable to flat reading than the usual decompression speeds.

(when I cloned LZO -> LZ4, rootfs occupancy changed from 1.5GB to 1.6GB, cloning time was ~3/4, Gotham loads 2s faster, apt-get update is 28s vs 34s. so no revolution but definitely evolution for free. and RPI is more responsive if fully loaded by disk operations).

most importantly the patches to btrfs are really non-invasive to btrfs itself, it is clean add-on, not rewrite.

and as I wanted to try to match the archaic LZ4 to actual - LZ4 is since 3.11 part of Linus's tree. so maybe the patch at the end will be really minimal. will look at it right now.

---

from total 18K patch, 17K is lz4_compress and lz4_decompress function. the rest 1K is just .h LZ4 defs and new BTRFS flag.

---

https://code.google.com/p/lz4/

[Perkabalo]

Excuse me for my noobyness, but how exactly would I install this?

What I''ve done:

# wget http://xbian.brantje.com/others/xbian-package-kernel1.3-6.11b-lz4.deb

# dpkg -i xbian-package-kernel1.3-6.11b-lz4.deb

changed cmdline.txt compress=lzo to compress=lz4

reboot.

But I guess there's something more to it, cause I end up in recovery mode, and have to switch back to compress=lzo to be able to boot.

[mk01]

yes, that's what was expected (wget & dpkg)


stupid me. wrong file. replaced with correct now.
(so wget & dpkg & cmdline edit & reboot).

thanks
mk

[narotic]

Just a curiousity, but wouldn't it be better to turn off compression for certain paths? For instance, /home/xbian/.xbmc/userdata/Thumbnails and /var/cache/apt/archives contain almost exclusively compressed data and compression at btrfs level will in that case only add overhead/latency.

[CurlyMo]

Tested and booted just fine.

Code:

[    1.992668] zswap: using lz4 compressor
[    4.899731] btrfs: use lz4 compression

[Smultie]

(6th Apr, 2014 05:32 AM)CurlyMo Wrote:  Tested and booted just fine.

Code:

[    1.992668] zswap: using lz4 compressor
[    4.899731] btrfs: use lz4 compression

What are your observations? Any user-noticeable improvements?

[CurlyMo]

You got me.

I installed it on my test raspberry pi. So the only thing i can say is that it didn't wreck my installation as in it booted fine after changing and still works. Nothing more.

[mk01]

@CurlyMo

that means you got it all right )

just two notes to that:

1) there is somewhere my post answering some discussions about btrfs & ext4 & fs2 & any other actual fs.
- majority of user (even myself) is not able to "feel" difference there in daily usage
(for anyone willing to try just re-clone XBian with xbian-config by specifying filesystem type in front of block device name - for instance:
fs2:/dev/sda2 will clone to sda2 and use fs2 filesystem, ext4:/dev/sda2 will use ext4 - XBian is completely universal in system setting files - no other changes are needed - only cmdline.txt of course)

2) Curly just still consider that lz4 is effective for newly written data only. if you don't re-clone by using lz4 as compression at destination, all the data currently being read are stored with lzo compression.
- if you have enough free space you can just use btrfs send | receive to another snap/subvolume like this

Code:

btrfs sub snap -r .....root/@ ....root/@ro
btrfs send .....root/@ro | btrfs receive .....root/@newlz4
btrfs sub delete ....root/@ro

and then rename /@ and /@newlz4 or just boot from @newlz4. but you know that drills.

@Curly there is one more optimisation which was introduced to btrfs, it's called tiny_metadata. metadata are stored into 2x(3x?) smaller blocks what is speeding up metadata operations specially on "recursive" CoW operations (which are almost all in case snapshots are present - and taking my words to the extreme - altering a file property can theoretically end up with writing lots of MB of data).

I made the tiny metadata as default when creating new clones (lz4 not yet as the kernel wasn't made public).

@Smultie
once again to avoid later confusion - there is nothing like "Magic stick". But there are scenarios you will see a difference. for instance apt-get installing 10 packages.

@all
in case you will try to re-clone to clean lz4 XBian copy, it is needed to manually edit /usr/sbin/btrfs-auto-snapshot line #603 and change "lzo" to "lz4". in next issue of xbian-package-config-shell autodetection for lz4 support will be already present.

@@all
last info - in case someone is using "btrfs-tools" for any kind of low level filesystem operations (btrfstune, btrfscheck etc), there is "btrfs-tools-xbian3" in devel repo.
it was needed to patch the tools for recognising new feature flags on filesystem. anyhow for standard use (running XBian or even cloning it is not needed).

---

@narotic

yes, definitely you are right. unfortunately btrfs until today doesn't support some "flags" per subvolume. compression is one of them. although this is planned for introduction - maybe kernel 3.14.

there is a possibility to use "chattr" and flag "+c", but in that case the logic is inverted, so we have to turn compression off for filesystem in general and set +c for files (subdirs) we allow to compress. the other way around would be preferred in my case as it is in case of +C (nocow which is used already on /home/xbian/.xbmc/userdata/Databases).

but at least all those flags are inherited so we can consider one-time manual .img preparation with cp or rsync by creating the toplevel hierarchy first and flagging them as needed.

on the other hand I'm not sure if benefits will outbalance the time spent - generally individual non-compresable files are not causing measurable difference (or even /var/cache/apt/archives). until files are read & written always with their full content, we are fine.
real troubles are caused by small block read & writes with SEEK inside one file. and this is again ./Databases. as for instance to get 4k DB data (could be one row from DB), filesystem needs to read & decompress for instance 128K or even 256K. later, on write of changed 4k back, this 256K needs to be read again, modified by 4K, recompressed as 256K and written again as 256K.
((+ imagine here CoW nature where on a write all complementary data needs to be rewritten to new area before it is considered as written & commited)).

so yes, let's say there are some areas worth looking at, but most important message again. we do discuss here less than 10(maybe 5)% of total operations.
overall "BIG PICTURE" looks and feels as always differently.

----
I suppose if someone will take the time and effort in that areas - it will be welcomed and merged.

[CurlyMo]

Doesn't rebalance compress all data?

[CurlyMo]

The latest staging updates hang when installed over the lz4 kernel

---

Restoring a kernel backup doesn't work

[Chris83]

Hi!

I have a question regarding mounting of btrfs filesystem with lz4 compression.

I'm running Xbian RC2 Gotham on the RPi B+. Connected to this is a 2TB USB drive.
I cloned to USB according to http://wiki.xbian.org/doku.php/installation_usb
and afterwards moved my content over to this USB via a Samba share.
Now I wanted to have a faster way to copy stuff around this USB drive and my PC (moving via samba is sooo slow). Hence I installed Linux Mint 17 and hoped I could just mount the USB there and copy stuff from/to other drives directly. Unfortunately I can't get Linux Mint to mount the USB drive.

From what I could see the cloning via xbian-config is creating a btrfs file system with lz4 compression by default.
I basically used:
mount -t btrfs -o subvol=root/@ /dev/sde1 /mnt

and got two errors:
1. cannot mount because of unsupported optional features (10)
2. btrfs: open_ctree failed

I then realized that Linux Mint by default uses compress=zlib.

I tried:
mount -t btrfs -o subvol=root/@,compress=lz4 /dev/sde1 /mnt

and the first error disappeared. However, I couldn't get the second error to disappear (I also tried all the other -o options I could see Xbian using for the mount).
I then saw that Xbian uses a newer Kernel and btrfs version than what comes with Linux Mint v2.
I updated the Kernel on Linux to a 3.16.X and also updated the version for btrfs (now also on 3.16).
I also added zswap.enabled=1 zswap.compressor=lz4 to the grub config.
The error is still there. btrfs check or any of the other btrfs tools won't work, because they can't receive an option to use lz4 compression.

The USB drive mounts under Xbian perfectly - so I don't think the filesystem is broken.
Any other advice on how I can mount the drive under Linux Mint and avoid the "btrfs: open_ctree failed" error?

[f1vefour]

Edit: Don't do this.

According to the BTRFS wiki there exists a problem on the drive and the btrfs-zero-log tool should be used to attempt a repair.

https://btrfs.wiki.kernel.org/index.php/Btrfs-zero-log

[CurlyMo]

With all respect, don't follow the advice of f1vefour! It can actually wreck the filesystem totally.

The problem is that the kernel you are using doesn't have support for the lz4 feature in BTRFS:

Code:

#define BTRFS_FEATURE_INCOMPAT_SUPP            \
    (BTRFS_FEATURE_INCOMPAT_MIXED_BACKREF |        \
     BTRFS_FEATURE_INCOMPAT_DEFAULT_SUBVOL |    \
     BTRFS_FEATURE_INCOMPAT_MIXED_GROUPS |        \
     BTRFS_FEATURE_INCOMPAT_BIG_METADATA |        \
     BTRFS_FEATURE_INCOMPAT_COMPRESS_LZO |        \
     BTRFS_FEATURE_INCOMPAT_RAID56 |        \
     BTRFS_FEATURE_INCOMPAT_EXTENDED_IREF |        \
     BTRFS_FEATURE_INCOMPAT_SKINNY_METADATA |    \
     BTRFS_FEATURE_INCOMPAT_COMPRESS_LZ4 |        \
     BTRFS_FEATURE_INCOMPAT_NO_HOLES)

The 10th feature in the list. You need to a patched kernel which linux Mint apparently doesn't provide.