[PROBLEM] BTRFS snapshots. Exclude paths

[xbfgf]

Running into a full filesystem often as I'm downloading files straight to my SD on a Raspberry.

Traced it to the regular snapshots running from cron.daily and cron.weekly, which keep a snapshot copy of the files even after they are deleted or moved. Deleting or moving the files doesn't free up space in that case.

Any ideas on how to exclude specific paths from the btrfs-auto-snapshot?
Is it a better idea to create a new btrfs subvolume, point all downloads there, and then change the cron entries to be specific (rather than the current //) and not include that new subvolume?

[CurlyMo]

No, not possible.

[mk01]

@xbfgf

it is as you proposed. in that way btrfs is like zfs. so snapshot operation starts and ends with sub-volume (as target for snapshot operations).

so create new subvolume and change temporarily // to "root home". or just root. it is up you (btrfs-auto-snapshots will take to consideration all listed filesystem - the names needs to be as last parameters).

I will put --exclude parameter to btrfs-auto-snapshot and update the cronntabs to run

Code:

btrfs-auto-snapshot -l daily -k $KEEPDAYS -v --exclude="$EXCLUDESUB" //

where EXCLUDESUB is new parameter in

/etc/default/xbian-snap:

Code:

# exclude those subvolumes by default for btrfs-auto-snapshot operations (put a list of names separated by spaces)
EXCLUDESUB=""

it was always considered as planned feature specially because XBian on its own provides "downloader" package. so once we test & deploy this general changes, downloader package should be updated to create new subvolume and use it as target for the DL programs.

I remember that you are not very first refreshing this problem - but until now all the other users probably migrated to USB HDD installs anyhow (to big disks so they postponed this discussion for few months ).
but sure it needs to be an implemented option. I will ask you bit later for testing the new btrfs-auto-snapshot ...

[f1vefour]

That sounds excellent mk01, I would like to see an exclude option.

[mk01]

preliminary support in 2.1.11 of -config-shell package. all is as proposed earlier. only the list in default/xbian-snap is comma separated, not space separated

cron jobs updated

and core support at btrfs-auto-snapshot wrapper takes --exclude or -x

it can be easily tested with

Code:

btrfs-auto-snapshot list -x root,home,data //

[f1vefour]

Is this in staging?

[mk01]

as it was not changing existing code I put directly to stable

[f1vefour]

Okay, thanks a lot mk01. I will test it out later and report any issues here.

[xbfgf]

Seemed to be working fine for me (I configured the exclusion of "/media"), until I had an unrelated issue and had to rollback to a previous "root" snapshot. I was not expecting that by doing it I would lose everything under /media.

I now list snapshots and find this one:
root/@btrfs-auto-snap_apt-run-2015-02-21-1837_rollback/media
which I was hoping could get me /media back, but it comes back with:

btrfs-auto-snapshot rollback root/@btrfs-auto-snap_apt-run-2015-02-21-1837_rollback/media
Error: Source name does not exists!


Any ideas?

[f1vefour]

If you still are running xbian what does this show:

Terminal

btrfs subvolume list '/'

[xbfgf]

Media was a subvolume. After restoring "root" snapshot, "/media" was still there, but empty and I was unable to do anything inside it (even a touch as root came back with permission denied).
I therefore ended up deleting the subvolume (which did work) and recreating it.

I guess that kind of invalidates the results now, but here goes:
(Note I'd taken a couple of the snapshots myself manually)

root@xbian ~ # btrfs subvolume list '/'
ID 257 gen 26534 top level 5 path root
ID 258 gen 25407 top level 257 path root/@btrfs-auto-snap_apt-run-2015-02-21-1837_rollback
ID 259 gen 26531 top level 5 path modules
ID 260 gen 26529 top level 259 path modules/@
ID 261 gen 26528 top level 5 path home
ID 262 gen 26560 top level 261 path home/@
ID 266 gen 26525 top level 5 path data
ID 267 gen 26523 top level 266 path data/@
ID 315 gen 25289 top level 258 path root/@btrfs-auto-snap_apt-run-2015-02-21-1837_rollback/media
ID 1050 gen 20777 top level 257 path root/@btrfs-auto-snap_apt-run-2015-02-14-1338
ID 1068 gen 22774 top level 266 path data/@btrfs-auto-snap_weekly-2015-02-16-0919
ID 1070 gen 22777 top level 261 path home/@btrfs-auto-snap_weekly-2015-02-16-0919
ID 1072 gen 22780 top level 259 path modules/@btrfs-auto-snap_weekly-2015-02-16-0919
ID 1074 gen 22783 top level 257 path root/@btrfs-auto-snap_weekly-2015-02-16-0919
ID 1076 gen 22916 top level 257 path root/@btrfs-auto-snap_apt-run-2015-02-16-1556
ID 1118 gen 26556 top level 257 path root/@
ID 1129 gen 25402 top level 258 path root/@btrfs-auto-snap_apt-run-2015-02-21-1837_rollback/root/before_rollback_22_02_2015
ID 1130 gen 25429 top level 1118 path var/log/after_rollback_22_02_2015
ID 1131 gen 25453 top level 266 path data/@btrfs-auto-snap_daily-2015-02-22-1803
ID 1133 gen 25456 top level 261 path home/@btrfs-auto-snap_daily-2015-02-22-1803
ID 1135 gen 25459 top level 259 path modules/@btrfs-auto-snap_daily-2015-02-22-1803
ID 1137 gen 25462 top level 257 path root/@btrfs-auto-snap_daily-2015-02-22-1803
ID 1139 gen 25814 top level 266 path data/@btrfs-auto-snap_daily-2015-02-23-0950
ID 1141 gen 25817 top level 261 path home/@btrfs-auto-snap_daily-2015-02-23-0950
ID 1143 gen 25820 top level 259 path modules/@btrfs-auto-snap_daily-2015-02-23-0950
ID 1145 gen 25823 top level 257 path root/@btrfs-auto-snap_daily-2015-02-23-0950
ID 1147 gen 25827 top level 266 path data/@btrfs-auto-snap_weekly-2015-02-23-0950
ID 1149 gen 25830 top level 261 path home/@btrfs-auto-snap_weekly-2015-02-23-0950
ID 1151 gen 25833 top level 259 path modules/@btrfs-auto-snap_weekly-2015-02-23-0950
ID 1153 gen 25836 top level 257 path root/@btrfs-auto-snap_weekly-2015-02-23-0950
ID 1155 gen 26411 top level 1118 path media
ID 1156 gen 26524 top level 266 path data/@btrfs-auto-snap_daily-2015-02-25-0902
ID 1157 gen 26524 top level 266 path data/@last_good_known
ID 1158 gen 26527 top level 261 path home/@btrfs-auto-snap_daily-2015-02-25-0902
ID 1159 gen 26527 top level 261 path home/@last_good_known
ID 1160 gen 26530 top level 259 path modules/@btrfs-auto-snap_daily-2015-02-25-0902
ID 1161 gen 26530 top level 259 path modules/@last_good_known
ID 1162 gen 26533 top level 257 path root/@btrfs-auto-snap_daily-2015-02-25-0902
ID 1163 gen 26533 top level 257 path root/@last_good_known

[xbfgf]

Hi, reproduced issue again.

Did a root snapshot rollback.
After that "/media" still showed content.
Rebooted, "/media" is there but has no content and cannot "touch it":

Terminal

root@xbian /media # touch 1
touch: cannot touch `1': Permission denied
root@xbian /media # ls -alrt
total 0
drwxr-xr-x 1 root root 172 Feb 23 17:42 ..
drwxr-xr-x 1 root root 0 Feb 27 13:00 .

[xbfgf]

Any ideas? Happened once again. Is this just the way it's meant to work?

[CurlyMo]

Yes

[xbfgf]

(24th Sep, 2015 03:24 AM)CurlyMo Wrote:  Yes

Ok, now I'm finding that every time I reboot, files inside my /media folder get deleted, except for two files dated back in March. This is different from my previous issue in that the actual "/media" folder remains, but it is practically empty.

Had created it with

Terminal

btrfs subvolume create /media

Any ideas? Might it be restoring some old snapshot on each reboot?