[Smultie]

(3rd Jul, 2013 12:04 AM)niftydude Wrote:  Hey mk,
I resized the btrfs partition, added a 256 mb linux-swap partition at the end of the sd card, edited /etc/fstab to mount that swap partition, and edited /etc/init/zram-swap.conf so that it doesn't create of mount /var/swapfile.

Thanks for letting me know about the resize on reboot thing - it would've taking me ages to figure out what was going on!

This seems to be working quite nicely!
I haven't seen the slowdowns and hangs that I was getting with swap set up the other way.

Also - using pvr to stream high def 1080p video channels from my mythtv server is now working!
This would also hang with swap the other way.

Note that I am just using the original beta for this - I haven't installed any of your other hotfixes yet.

I really think we need to make this change!

I'll let it run overnight and try it again in the morning to see if anything bad happens, but I think we are good.

I think you just found out why Beta 1 feels slower than Alpha 5! Great job!

[alfredo]

(2nd Jul, 2013 05:52 PM)mk01 Wrote:  you told very little about how you tried this. downloaded image of beta1, copied /usr and /var to the other card, changed /etc/fstab and rebooted?

Sorry for being too vague, you're right. Essentially what you described is what I did. First I ensured I could run the system with /usr and /var on the same filesystem (that is, vanilla beta1 works on my rpi). Then, I copied /usr and /var to the usb stick, modified /etc/fstab, emptied the content of such directories on the sdcard, and rebooted.

This is the setup that worked for me on alpha5. Ther I had 2 ext3 partitions on the usb stick, one for /usr and one for /var, and accordingly a slightly different /etc/fstab. Now I have a single partition on the usb stick with two subvolumes.

(2nd Jul, 2013 05:52 PM)mk01 Wrote:  if so, try without "splash" on cmdline. at the end of initramfs stage, there is splash re-run from /rootfs (real root partition as specified on cmdline), what can be problem, because your /usr is not within this /rootfs. so splash won't get know about rootfs change and will stay on screen for ever.

so disable it and tell how it's going.

I did it (removed the "splash" option from cmdline.txt in /boot), and this is the output I get (Please note due to overscan I cannot read the beginning of the output!, some words are guessed...):

Code:

Checking for keyboard...
Mounting root as: mount -t btrfs -o [...] /dev/mmcblk0p2 /rootfs
[...]ing root

I can't read/guess the beginning of the third line... sorry!

After this, the system "hangs" (no activity on any led). My guess is some file in /usr/bin is not being found, but no error is reported at this stage.

As a side note: When we sort this booting issue out, once I delete the content from /usr and /var on the sdcard, I guess the @safe snapshot will still be keeping the sdcard space busy. Would it be safe to delete them?

Thanks for the support!

[anonimo200]

Good evening from Spain. New issue here: after adding mk01's repo, installing xbian-update and upgrading, the pi doesn't shut down (neither from gui nor from ssh) I've read it can be a Xbmc 12.2 problem but I'm not sure if it would affect power off when "shutdown -h now" from ssh...
And it's strange. When I did a reboot or shutdown in ssh the connection stopped. Now the terminal keeps open (althought I can't write anything)

[mk01]

(3rd Jul, 2013 12:50 AM)raspberry_pd Wrote:  

(2nd Jul, 2013 06:38 PM)mk01 Wrote:  .ts is most probably MPEG1 so yes (assumption)

Do you mean MPEG2?

of course, typo

---

(3rd Jul, 2013 12:04 AM)niftydude Wrote:  I'll let it run overnight and try it again in the morning to see if anything bad happens, but I think we are good.

ok, I will add param to cmdline.txt "partswap", which will, in the case of rootfstype=btrfs, create 256mb swap partition as last partition on the drive. and the param will be default after tests.

---

(3rd Jul, 2013 03:30 AM)alfredo Wrote:  

Code:

Checking for keyboard...
Mounting root as: mount -t btrfs -o [...] /dev/mmcblk0p2 /rootfs
[...]ing root

I can't read/guess the beginning of the third line... sorry!

After this, the system "hangs" (no activity on any led). My guess is some file in /usr/bin is not being found, but no error is reported at this stage.

As a side note: When we sort this booting issue out, once I delete the content from /usr and /var on the sdcard, I guess the @safe snapshot will still be keeping the sdcard space busy. Would it be safe to delete them?

Thanks for the support!

it is "Moving root". last action before switch_root to /dev/mmcblk0p2. then, /sbin/init is started and /sbin/mountall as first binary (with no any library dependency outside /lib).

so technically it should work. can I see your /etc/fstab? then, remove "quiet" parameter from cmdline.txt and be sure to check, that /dev/sdxy partition appear BEFORE "Moving root". if not, change in cmdline.txt scsi_mod.scan=async to sync. this should tell the kernel to run scsi (usb drives) scan not asynchronously. if even this not helps, edit /etc/init/mountall.conf, change --timeout=1 to --timeout=50 (it is seconds). or just hold left shift on keyboard during boot (after cursor first appear on screen - what you will not see because overscan). ok, reboot, hold left shift on keyboard (attached directly to RPI), you should get recovery shell from within initramfs. then wait until /dev/sdxx appear, exit. it will continue boot process.

if even this not helps, I don't know.

btw: mountall is doing .log file in /run/mountall.log. you can change mountall.conf file to provide screen output by uncommenting #console output line and remove the redirect to /run/ ....

also, you can boot with cmdline option init=/bin/bash, which will (after Moving root) instead of /sbin/init start /bin/bash and provide console. and check what is going on.

anyhow you should be able to diagnose with the above info. still, you can send the /etc/fstab

and the snapshot stuff is like you say. and yes, feel free to delete @safe. but then no possibility to revert to stock files / confs - but you are aware of it.

[alfredo]

(3rd Jul, 2013 04:22 AM)mk01 Wrote:  can I see your /etc/fstab?

I have quite an inconvenient setup (detailed at the bottom for completeness), and I can only access /boot when the sdcard is not mounted on the rpi. That said, each of the two lines I added to /etc/fstab looks like:

Code:

LABEL=xbian-usb-btrfs /var btrfs [copied/pasted options from root, except "subvol=var"] 0 0

I have formatted the usb stick with btrfs, giving the appropriate label (xbian-usb-btrfs), and created the two subvolumes, "var" and "usr".

(3rd Jul, 2013 04:22 AM)mk01 Wrote:  then, remove "quiet" parameter from cmdline.txt and be sure to check, that /dev/sdxy partition appear BEFORE "Moving root". if not, change in cmdline.txt scsi_mod.scan=async to sync. this should tell the kernel to run scsi (usb drives) scan not asynchronously.

I did both, in order, and the output didn't change in any case. Removing quite did not add any line to the output (is this expected?).

Right now, I cannot continue experimenting with mountall.conf nor the keyboard, due to my constrained setup. I will try a fresh re-install to access files under /etc, and will report here my findings. But I hope at least my (pseudo-)fstab can give you some hints if I'm doing anything wrong.

(A note about my setup, and induced constraints: I only have a rpi, and a macbook. So direct usb-keyboard access to the rpi is not doable. Moreover, macosx only reads the vfat /boot partition, and the sdcard reader of the mac is not available to virtual machines (e.g. to let a linux VM mount the btrfs) ...it's SO frustrating!)

[mk01]

(3rd Jul, 2013 03:39 AM)anonimo200 Wrote:  Good evening from Spain. New issue here: after adding mk01's repo, installing xbian-update and upgrading, the pi doesn't shut down (neither from gui nor from ssh) I've read it can be a Xbmc 12.2 problem but I'm not sure if it would affect power off when "shutdown -h now" from ssh...
And it's strange. When I did a reboot or shutdown in ssh the connection stopped. Now the terminal keeps open (althought I can't write anything)

yes, I found one small issue, if you have /etc/init/avahi-daemon and /etc/init/dbus, do

"insserv -r /etc/init.d/avahi-daemon; insserv -r /etc/init.d/dbus"

or apt-get update; apt-get upgrade. it's fixed in the package.

mk

---

(3rd Jul, 2013 07:24 AM)alfredo Wrote:  I did both, in order, and the output didn't change in any case. Removing quite did not add any line to the output (is this expected?).

(A note about my setup, and induced constraints: I only have a rpi, and a macbook. So direct usb-keyboard access to the rpi is not doable. Moreover, macosx only reads the vfat /boot partition, and the sdcard reader of the mac is not available to virtual machines (e.g. to let a linux VM mount the btrfs) ...it's SO frustrating!)

- remove loglevel=0 as well

- indeed I use usb sticks / sd cards in VM under mac with no issues. but I have parallels. 70$. worth the money. but usb keyboard is cheaper

---

if you use Virtualbox, it should be possible to dd the whole disk (or in mac with disk utils). create img and mount it as hdd to virtualbox session. if not

use loop devices with offset parameter.

losetup /dev/loop0 /path_to_dd_file --offset 1048576
losetup /dev/loop1 /path_to_dd_file --offset 36700160

so loop0 will be mmcblk0p1
and loop1 partition2

---

(3rd Jul, 2013 12:04 AM)niftydude Wrote:  Hey mk,
I resized the btrfs partition, added a 256 mb linux-swap partition at the end of the sd card, edited /etc/fstab to mount that swap partition, and edited /etc/init/zram-swap.conf so that it doesn't create of mount /var/swapfile.

on git is zram-swap which uses swap partition(s) if found, loop is fallback.

can you test ? feel free to adapt.

[mk01]

(3rd Jul, 2013 03:39 AM)anonimo200 Wrote:  Good evening from Spain. New issue here: after adding mk01's repo, installing xbian-update and upgrading, the pi doesn't shut down (neither from gui nor from ssh) I've read it can be a Xbmc 12.2 problem but I'm not sure if it would affect power off when "shutdown -h now" from ssh...

btw: shutdown -P now

[niftydude]

Hi mk,
i've tried out that zram-swap script from git and rebooted. Nice. Seems to be working OK:

xbian@xbian ~ $ free
total used free shared buffers cached
Mem: 125112 100740 24372 0 8 22200
-/+ buffers/cache: 78532 46580
Swap: 393208 16784 376424

xbian@xbian ~ $ sudo swapon -sa
[sudo] password for xbian:
Filename Type Size Used Priority
/dev/mmcblk0p3 partition 262140 0 -1
/dev/zram0 partition 131068 16784 20


I feel like this is still a little bit slower than alpha 5, but better than beta 1 was. I'll let it run and see how it goes.

[mk01]

(3rd Jul, 2013 11:59 AM)niftydude Wrote:  [sudo] password for xbian:
Filename Type Size Used Priority
/dev/mmcblk0p3 partition 262140 0 -1
/dev/zram0 partition 131068 16784 20
[/font][/size]

I feel like this is still a little bit slower than alpha 5, but better than beta 1 was. I'll let it run and see how it goes.

you can experiment with disabling zram (zram is doing zlib compression on memory blocks up to its size) - or change it's size (it's in /etc/default/zram-swap), you can disable CoW for sqlite files (rename Database dir, mkdir Database, chattr +C Database, cp all files from old to new) - no idea if you use them.

btw: don't forget, that alpha5 had different resolution settings for gui. but at the end is strange, how people differently are measuring (feeling) "speed" and "time". there is another group, which sees the beta1 as fast

in beta1 until RC there is boot/start time logger (logged uptime at various steps during booting until full xbmc). check file /run/uptime_start.log.

---

(3rd Jul, 2013 11:59 AM)niftydude Wrote:  Filename Type Size Used Priority
/dev/mmcblk0p3 partition 262140 0 -1
/dev/zram0 partition 131068 16784 20

the 16mb usage is just after start, or after at least 2h usage ?

it looks like I will put 384mb ram to gfx, so for system stays 128mb. it's like the A model then - 128/128?

[mk01]

(1st Jul, 2013 12:18 PM)Pawlisko Wrote:  

(1st Jul, 2013 06:55 AM)f1vefour Wrote:  It appears masquerade has not been included in the current kernel.

In that case - xBian developers - PLEASE include it in next kernel.

done, kernel on github

[niftydude]

(3rd Jul, 2013 12:16 PM)mk01 Wrote:  you can experiment with disabling zram (zram is doing zlib compression on memory blocks up to its size) - or change it's size (it's in /etc/default/zram-swap)
...
but at the end is strange, how people differently are measuring (feeling) "speed" and "time". there is another group, which sees the beta1 as fast

Ah I hadn't realised. This could explain why mine is feeling slow. I have one of the early rpis, which only has 256mb ram. With the current split of 128 gfx mem, this means I have only 128 mb system ram, and zram is being run on ALL of that, so memory access is slow.

But the people with newer rpis have 512mb ram, so only a third of their ram is being used by zram, so I can see why they would have fast performance. This might be why different people are having such different experiences!

I think I'll make zram much smaller or maybe disable it completely for a bit and see how performance is.

(3rd Jul, 2013 12:16 PM)mk01 Wrote:  you can disable CoW for sqlite files (rename Database dir, mkdir Database, chattr +C Database, cp all files from old to new) - no idea if you use them.

Good idea - when I did my tv-show scan, it crashed after it reached shows starting with J :-) maybe partly because of this.

(3rd Jul, 2013 12:16 PM)mk01 Wrote:  in beta1 until RC there is boot/start time logger (logged uptime at various steps during booting until full xbmc). check file /run/uptime_start.log.

at initramfs/init end: 6.62 0.58
at xbmc-preload: 8.63 0.58
at xbmc full load: 33.15 0.58

But booting isn't slow for me, beta1 definitely boots up much faster than the alphas!

(3rd Jul, 2013 12:16 PM)mk01 Wrote:  

---

(3rd Jul, 2013 11:59 AM)niftydude Wrote:  Filename Type Size Used Priority
/dev/mmcblk0p3 partition 262140 0 -1
/dev/zram0 partition 131068 16784 20

the 16mb usage is just after start, or after at least 2h usage ?

This was after about an hour - but I ran a high def mkv video file for about 20 mins.

After about a day, swapon -sa looks like this:


/dev/mmcblk0p3 partition 262140 1196 -1
/dev/zram0 partition 131068 128296 20


and the system grinds to a halt. I think I'll definitely disable zram.

[alfredo]

A short update about my booting issue: removing loglevel=0 I finally get some interesting output. With "scsi_mod.scan=sync" I see that the USB stick is recognized as sda, along with its partition sda1, *before* "Mounting root".

After "Mounting root", only one more line appears:

Code:

init: mount all main process (234) terminated with status 127

I really need now to debug why mountall fails (or increase its timeout).

I also gave Parallels a try, but again it fails with the embedded sd card reader. I think I'll buy an external USB card reader, and make my life easier ;-)

As usual I'll get back as soon as I get more info, but wanted to report early on current findings. Thanks a lot for your support... and patience with my issue!

[mk01]

(4th Jul, 2013 02:11 PM)niftydude Wrote:  Ah I hadn't realised. This could explain why mine is feeling slow. I have one of the early rpis, which only has 256mb ram. With the current split of 128 gfx mem, this means I have only 128 mb system ram, and zram is being run on ALL of that, so memory access is slow.

of course. I really don't know what blackout I had keeping default 128, probably forgot again, that the other 128 is video memory.

so as I said, I'm doing the development now with 384 video mem, what keeps 128mb ram as those with model A. and guess what. even the default 128 is BIGGER as free mem (124) so the very first time I was installing debs I created few second before, I got swap error, then then second, then kernel tried kill processes and then freeze. of course this is not the way.

in the update which is already been posted, default is still there, and is taken, but only if it is 1/3 of memory available to kernel (so total - video). otherwise, is it 1/3.

anyhow, did you try the swappart parameter? it resizes root, creates 256 mb swap at the end of the device. zram-swap - if finds ANY swap partition (even on other usb disks), uses them instead of file.

---

(4th Jul, 2013 05:29 PM)alfredo Wrote:  As usual I'll get back as soon as I get more info, but wanted to report early on current findings. Thanks a lot for your support... and patience with my issue!

put "debug" in cmdline.txt. you should see all the output, including mountall and upstart.

second option is put "rescue_late" in cmdline, it will drop you to shell after "Moving root" and before switch_root. root is under /rootfs folder, /dev, /proc, /run and /sys moved already. so the best is right go to "chroot /rootfs". practically you are at your root (of course without /var and /usr mounted).

third option is to put "init=/bin/bash" into cmdline, it will drop you after switch root. / is your root. if you manage to mount /var and /usr manually, you can put "exec /sbin/init" and upstart (and immediately mount all) will be launched.

in option 2 and 3 you can ifup your lan, you can fully access filesystem and do necessary changes to fstab or /etc/init/mountall.conf.

NOTE: during switch_root, udevd is killed and re-launched, even with udevadm trigger --action add, than means, your disk will have to take again some time to reappear (/dev is recreated for devices not already in use). parameter for mountall is --dev-wait-time=value (in seconds).

[niftydude]

(7th Jul, 2013 12:39 PM)mk01 Wrote:  anyhow, did you try the swappart parameter?

Oh yeah - forgot to mention - yep - tried this, it worked fine for me.

[puckMan]

(8th Jul, 2013 03:14 PM)niftydude Wrote:  

(7th Jul, 2013 12:39 PM)mk01 Wrote:  anyhow, did you try the swappart parameter?

Oh yeah - forgot to mention - yep - tried this, it worked fine for me.

can you please tell me how to try that out?