btrfs and lz4

[mk01]

@CurlyMo

that means you got it all right )

just two notes to that:

1) there is somewhere my post answering some discussions about btrfs & ext4 & fs2 & any other actual fs.
- majority of user (even myself) is not able to "feel" difference there in daily usage
(for anyone willing to try just re-clone XBian with xbian-config by specifying filesystem type in front of block device name - for instance:
fs2:/dev/sda2 will clone to sda2 and use fs2 filesystem, ext4:/dev/sda2 will use ext4 - XBian is completely universal in system setting files - no other changes are needed - only cmdline.txt of course)

2) Curly just still consider that lz4 is effective for newly written data only. if you don't re-clone by using lz4 as compression at destination, all the data currently being read are stored with lzo compression.
- if you have enough free space you can just use btrfs send | receive to another snap/subvolume like this

Code:

btrfs sub snap -r .....root/@ ....root/@ro
btrfs send .....root/@ro | btrfs receive .....root/@newlz4
btrfs sub delete ....root/@ro

and then rename /@ and /@newlz4 or just boot from @newlz4. but you know that drills.

@Curly there is one more optimisation which was introduced to btrfs, it's called tiny_metadata. metadata are stored into 2x(3x?) smaller blocks what is speeding up metadata operations specially on "recursive" CoW operations (which are almost all in case snapshots are present - and taking my words to the extreme - altering a file property can theoretically end up with writing lots of MB of data).

I made the tiny metadata as default when creating new clones (lz4 not yet as the kernel wasn't made public).

@Smultie
once again to avoid later confusion - there is nothing like "Magic stick". But there are scenarios you will see a difference. for instance apt-get installing 10 packages.

@all
in case you will try to re-clone to clean lz4 XBian copy, it is needed to manually edit /usr/sbin/btrfs-auto-snapshot line #603 and change "lzo" to "lz4". in next issue of xbian-package-config-shell autodetection for lz4 support will be already present.

@@all
last info - in case someone is using "btrfs-tools" for any kind of low level filesystem operations (btrfstune, btrfscheck etc), there is "btrfs-tools-xbian3" in devel repo.
it was needed to patch the tools for recognising new feature flags on filesystem. anyhow for standard use (running XBian or even cloning it is not needed).

---

@narotic

yes, definitely you are right. unfortunately btrfs until today doesn't support some "flags" per subvolume. compression is one of them. although this is planned for introduction - maybe kernel 3.14.

there is a possibility to use "chattr" and flag "+c", but in that case the logic is inverted, so we have to turn compression off for filesystem in general and set +c for files (subdirs) we allow to compress. the other way around would be preferred in my case as it is in case of +C (nocow which is used already on /home/xbian/.xbmc/userdata/Databases).

but at least all those flags are inherited so we can consider one-time manual .img preparation with cp or rsync by creating the toplevel hierarchy first and flagging them as needed.

on the other hand I'm not sure if benefits will outbalance the time spent - generally individual non-compresable files are not causing measurable difference (or even /var/cache/apt/archives). until files are read & written always with their full content, we are fine.
real troubles are caused by small block read & writes with SEEK inside one file. and this is again ./Databases. as for instance to get 4k DB data (could be one row from DB), filesystem needs to read & decompress for instance 128K or even 256K. later, on write of changed 4k back, this 256K needs to be read again, modified by 4K, recompressed as 256K and written again as 256K.
((+ imagine here CoW nature where on a write all complementary data needs to be rewritten to new area before it is considered as written & commited)).

so yes, let's say there are some areas worth looking at, but most important message again. we do discuss here less than 10(maybe 5)% of total operations.
overall "BIG PICTURE" looks and feels as always differently.

----
I suppose if someone will take the time and effort in that areas - it will be welcomed and merged.