[SOLVED] duplicate/clone an xbian installation

[rikardo1979]

(2nd Jun, 2013 06:27 AM)effemmeffe Wrote:  thanks. so i have to conclude there is no safe way to backup all my sd from inside the system, the only way would be remove the sd, put it in a card reader and dd it on an external system. not very friendly...

isn't the XBMC Backup add-on good enough for you ?

[effemmeffe]

afaik XBMC Backup only does backup of XBMC configuration. If I install xbian and then I personalize it for my needs I'd like to have a copy of all the system to restore. I thought that dd'ing the whole sd would be the fastest way: in case of sd corruption I could take the sd card, put it in the card reader on my mac and dd on it the image backup. 10 minutes up and running.
am i wrong?
i'm open to learn some way to have that.

[rikardo1979]

(2nd Jun, 2013 07:05 AM)effemmeffe Wrote:  afaik XBMC Backup only does backup of XBMC configuration. If I install xbian and then I personalize it for my needs I'd like to have a copy of all the system to restore. I thought that dd'ing the whole sd would be the fastest way: in case of sd corruption I could take the sd card, put it in the card reader on my mac and dd on it the image backup. 10 minutes up and running.
am i wrong?
i'm open to learn some way to have that.

yes sure this would be the best and easiest way to do so.
So what stopping you to do it this way? I see you trying to do it the automated way but to be honest I dont see a point to complicate the life with this. How many backups do you really need ? I dont do it personally as I do the strange testing things all the time but for average user once a week should be just enough, so it should not be that much hassle just shut it down, plug the card to computer and do the backup

[effemmeffe]

(2nd Jun, 2013 07:11 AM)rikardo1979 Wrote:  So what stopping you to do it this way? I see you trying to do it the automated way but to be honest I dont see a point to complicate the life with this. How many backups do you really need ?

just one, as for all the backups: the one before any unexpected problem occurred.
that's why i tried to do it on a day base, you can never tell when the next power outage will occour, and it' sure will be the next time you're copying that big file on the card...

[mk01]

from beta1 this should not be an issue anymore, thanks to CoW (copy on write) after such situation you will have old version of the file, or new version of the file (new (or changed) data are written to a new location as you would make a copy - only after 'new' copy is successfully written the old one is trashed).

but this doesn't mean a disaster can't happen. together with this snapshots are coming to xbian and rootfs split into system and home. this will allow you to restore your xbian to a previous point in time - and separately the system and xbmc settings (/home dir).

so if system goes wrong, you can revert just system without touching your current XBMC customizations / DBs / add ons etc. and you will be allowed to create snapshots (checkpoints) before each update / change, so if any package / update breaks something you will be able to on-click restore in time without the husle of reinstalling / installing / repairing anything.

(and for those wanting backup on separate media, all the snapshots are possible to send to different drives, similar as let's say create image with dd, but the snapshots are atomic, each part of FS is exactly the same as you would turn off RPi and do a copy offline).

btw; if you are putting so much importance into the actual RPi setup / files, transform RPi to a thin-client. move the filesystem to a mirrored NAS (or similar solution) and boot the RPi directly from network. I'm using it like this, you just needs the small VFAT partition to load network and rootfs is mounted from network directly via NFS.

[kas_27_es]

At last, today I've tried to duplicate my SD using DD command...

It didn't work.

I'm using this:

Terminal

sudo dd if=/dev/sdb of=xbian.back.img

Apparently this instruction works fine, but when I try to do the duplicate with:

Terminal

sudo dd if=xbian.back.img of=/dev/sdb

New SD doesn't work... I'm frustrated...

Xbian version: xbian beta 1.1
Linux version: xubuntu 12.04

Any help?

Thanks...

[kraleksandr]

@kas_27_es
try this:

Code:

dd if=/dev/sdb1 of=first.img
dd if=/dev/sdb2 of=second.img

Then create 2 partitions on ur second SDcard (first - 34M Fat32) and

Code:

dd if=first.img of=/dev/sdb1
dd if=second.img of=/dev/sdb2

---

How exactly it doesnt work?

[kas_27_es]

(29th Jul, 2013 05:15 AM)kraleksandr Wrote:  @kas_27_es
try this:

Code:

dd if=/dev/sdb1 of=first.img
dd if=/dev/sdb2 of=second.img

Then create 2 partitions on ur second SDcard (first - 34M Fat32) and

Code:

dd if=first.img of=/dev/sdb1
dd if=second.img of=/dev/sdb2

---

How exactly it doesnt work?

First: thanks for your answer.

Second: when I plugged it in my rpi I only saw a black screen with th command prompt and a message at the top of the screen saying something about it's no possible to mount something...
Sorry I know I'm not so much clear, but I didn't take note about it. I've given up...

Third: Have you tested this solution? Does it work? I trust you, but I'd like to hear good news... As I told before, I'm a little bit depressed with this question.

Thanks again.

[Fred]

Is the second SD card at least as big as the first?
Do the commands (sudo dd ...) when you make the image and dd the image to the new SD card return succes messages?
If you do a clean install on the second SD card will it work? (maybe this SD card is not compatibel with the RPi).

[zilexa]

I can confirm, doing an offline backup using DD on Mac OSX or using USBIT on Windows works perfectly.
The only problem is that SD cards have different sizes. Not just 4, 8, 16GB. I have installed 4 RPis all with ICIDU 20MB/s class 10 4GB. But mine is 3.97GB, another one is 4.01GB, another 4.04GB and finally 3.99GB.
So the image of 4.04GB won't fit on the other ones. I use 3.97GB myself so I use mine as the original, fits on all others.

edit: after reading Fred's post and link I will definitely try rsync. Although creating a backup on the device I want to backup just sounds wrong to me (especially after reading mk01's comment above that post "after one run, run it one more time to copy changes from time during the copy and you are ready.")

But it would result in a smaller image (I think) that would fit on any other 4GB SD card and also, less of a hassle shutting down my RPi and putting the SD in a laptop.. Im lazy.

[mk01]

(29th Jul, 2013 10:50 PM)zilexa Wrote:  edit: after reading Fred's post and link I will definitely try rsync. Although creating a backup on the device I want to backup just sounds wrong to me (especially after reading mk01's comment above that post "after one run, run it one more time to copy changes from time during the copy and you are ready.")

the question "why I'm making backup", "for what purpose" must be answered before we choose a strategy.

using snapshots is a excellent way to freeze system / data state at specific point (for example before system upgrade / update). it takes seconds, rollback seconds again. imagine you would sector copy 64GB card each time you run apt-get.

rsync can provide something like incremental backups to extern media, providing backup again media failure - much more efficient as dd. and again, using dd on mounted filesystem can never lead to a atomic and 100% healty copy. it could not on ext2/3/4 and definitely can't on btrfs.

if you are lazy (i'm lazy too), use the shift key "hack" to drop down to initramfs before the rootfs is mounted. that way you can use even dd on btrfs and the copy will work.

or, it's worth try remouting the rootfs as RO, with:

"mount -o remount,ro /"

and do the dd copy as always.

taking into account we are on btrfs, use the native send/recv commands to sends snapshot data from one btrfs/subvolume to another. it's atomic, can be done on RW mounted and used filesystem, will copy only the data (as rsync) and natively supports increments (differences between snapshots). it can be used to local media as well as via network to another system with btrfs support making instant duplicate ready to be used on the other system.

[Karen Pollard]

(2nd Jun, 2013 08:07 AM)mk01 Wrote:  btw; if you are putting so much importance into the actual RPi setup / files, transform RPi to a thin-client. move the filesystem to a mirrored NAS (or similar solution) and boot the RPi directly from network. I'm using it like this, you just needs the small VFAT partition to load network and rootfs is mounted from network directly via NFS.

Hi there MK01

How would one go about such a setup. It seems vastly preferable to relying on sdcards for the working filesystem. Could you give an overview of what's involved ?

Also, you may be able to shed some light on this quandary I have from THIS THREAD.

I have compiled XBMC Gotham on an xbian pi with the latest xbmc-pvr-addons (v.1.8.1). It has solved ALL of my problems with mythtv and there were lots of issues (EPG stalling, slow EPG load, FF, RW, HD channels not working (BBC especially)).

Now, I'd like to share an image of this setup with the world for others in the same quandary (with explicit warnings that it effectively means you can't use xbian-updates etc in the future or it will no doubt overwrite the compiled gotham version etc etc....

But the image file is 16GB. I have the image file on a linux box and am struggling to safely shrink it. I need to know how to safely know how much it can be shrunk by ?

Thanks for any help you can give with either of the above questions and BIG THANKS for all you do on xbian on pi !!! It's my favoured distro !

kp

[belese]

I'm not the developer of this, but here's a copy of a PM from mk01,
i didn't test myself the size params, but should be what you want :

Quote:sudo xbian-config xbiancopy start source dest label size

where:
1- source is source partition - normally this is /dev/root (actually booted root), but can by used to copy others as well (btrfs only)
2- dest is destination partition - like /dev/sda2 etc. if "dest" is prefixed with "file:", then file will be created

not required:
3- label - by default is "xbian-copy" used
4- size - this makes sense only for destination as file. by default it is size of source partition. but sometimes you have 16gb sdcard, but 2gb used, so could be specified.

exemple:
xbian-config xbiancopy start /dev/root /dev/sda1
xbian-config xbiancopy start /dev/root file:/mnt/path/to/file.img

if file is used, btrfs-auto-snapshot script is copying also /boot and adapts /boot/cmdline.txt so direct flash of that img is possible.

ps : will work only on beta2 test
and command will return directly,
to know the status progress you can call
sudo xbian-config xbiancopy status (if i remember return 0 when finished)

[Exnor]

(10th May, 2013 06:03 AM)captainsammitch Wrote:  Apologies if this thread is somehow an infraction against the forum procedures or out of format or in the wrong forum, but I checked the FAQ and other threads and haven't found an answer thus far. I work at a store in a modest-sized chain of computer stores and was recently handed the assignment (well, I kind of volunteered) to build a looping video kiosk for product promos, tech demos, and the like. I used the most recent release on an RPi v2 and have it configured exactly how I want it - plug it in, it boots into XBMC, shuffles the videos in a playlist and loops them continuously with no interaction needed. It's completely self-contained and everything is on the SD card.

Our store manager and I hit on the idea at the exact same time that if you could somehow clone that SD card with everything on it and ready to go, then you could have an identical kiosk in every store in the chain. Unfortunately, everything I've tried thus far to duplicate my existing installation has fallen flat. The problem is that the partition XBian is using for the root filesystem won't show up in any of the OSes to which I have access - I've tried XP, 7, 8, OSX 10.6, Mint 14 KDE, and even Paragon, DriveImageXML, and Clonezilla show that partition as unallocated space or don't pick it up at all. The only way I can add/remove videos or edit the contents of my userdata folder and subfolders is by piping in for FTP via SSH using WinSCP; I use PuTTY to config everything.

So what I'm wondering is, can I somehow clone the card on which XBian and our videos are kept so that I can pop the duplicate into another RPi and have the exact same thing happen that happens with our display unit when I come in in the morning? Or am I just going to have to do a separate install procedure on each card and manually drag-and-drop the media and any userdata and configs via the FTP program? I have a feeling the first option would be quite a bit quicker and make my manager a lot more likely to pitch this to home office. Any advice would be appreciated.

Dont know if this will help you, but i always clone my SDcard with xbian to my 2nd Pi with Norton Ghost 8.0... its old soft but runs well on win7 & 8...

No issue so far.. in fact i keep an image of a stable setup on backup just in case i mess up any of the Pi's

[IriDium]

Beta 2 - latest release has the option and it works.

1. Backup home directory and then copy it off the device (Gets removed on reboot)
2. Full system backup to IMG file. (Needs external device). Just flash to new SD card and you are back where you backed up.

With automatic backups, snapshots and IMG creation, nobody on Beta 2 should suffer a catastrophe no return situation. (I KNOW I will eat those words :cool but it shouldn't if the setup is correct.