Idea: Making best use of 512mb Pi with tmpfs - Printable Version

Forum

[IDEA] Making best use of 512mb Pi with tmpfs - Printable Version

+- Forum (http://forum.xbian.org)
+-- Forum: Software (/forum-6.html)
+--- Forum: Testing & Experimental (/forum-21.html)
+--- Thread: [IDEA] Making best use of 512mb Pi with tmpfs (/thread-764.html)

Making best use of 512mb Pi with tmpfs - JayBlanc - 9th Apr, 2013 08:06 AM

THIS IS DEPRICATED, DO NOT TRY IT.

This will no longer have a beneficial effect with future changes to memory handling and swap space in the next version. I recommend reversing the changes if you made them.

---------------------
Having a recent Rev.B I've modified my xbian set up to make a bit better use of the memory available.

First, I've mounted /tmp as a tmpfs. The defaults are fine because half of hardware memory as the top limit appears okay for the intended use. (Deliberately leaving /var/tmp alone as that may have large files written to it.)

Add the following line to /etc/fstab

Code:

tmpfs   /tmp         tmpfs   nodev,nosuid    0    0

Stopping XBMC, I symlink ~/.xbmc/temp to /tmp/, ~/.xbmc/userdata/addon_data/cache and ~/.xbmc/userdata/addon_data/script.common.plugin.cache to /tmp/, then reboot.

Terminal

sudo /etc/init.d/xbmc stop
rm /home/xbian/.xbmc/temp
ln -s /tmp /home/xbian/.xbmc/temp
rm /home/xbian/.xbmc/userdata/addon_data/cache
ln -s /tmp /home/xbian/.xbmc/userdata/addon_data/cache
rm /home/xbian/.xbmc/userdata/addon_data/script.common.plugin.cache
ln -s /tmp /home/xbian/.xbmc/userdata/addon_data/script.common.plugin.cache

Then create cron.daily script that reaps old files from /tmp/ to try and prevent overflow on a device that is rarely rebooted.

Code:

#!/bin/sh

#/etc/cron.daily/reaptmp

find /tmp/ -type f -mtime +3 -exec rm {} \;

Remember to run chmod a+r on the file so it will run every day. Then reboot.

Benefit: Noticeable responsiveness improvements in addons that create and process temporary files. Particularly ones that process downloaded rss/xml files, or download subtitle files.

Drawbacks: Caches cleared on reboot. But reboots are hopefully uncommon for XBian.

If anything fills /tmp/ with data over ~190mb, it's going to fail. I've yet to see this happen, and I don't know of any xbmc addon that would do this. If you know of a addon that uses creates temporary files over ~190mb, you might want to ask the addon's creator to store these files in a cache in it's addon_data directory. .xbmc/temp/ and common_cache appear to be intended for small temporary files only. tmpfs can be expanded beyond free memory, because it uses swap, but it can never be set above total hardware memory due to memory allocation handling.

There is the minor possibility that this could cause an issue due to file name collisions with something else. I haven't seen this happen on xbian, but it might be worth making a init script that properly creates a unique directory name in /tmp/ and creates the symlinks on boot.

Making best use of 512mb Pi with tmpfs - Koenkk - 9th Apr, 2013 08:52 AM

good idea, but like you already said, we need to make 100% this causes no failures. Therefore it would be nice if people can test this and post their results here.

RE: Making best use of 512mb Pi with tmpfs - CurlyMo - 9th Apr, 2013 06:15 PM

I would wait for all the things Matus is implementing because he is already working on better ram usage due to different tmpfs solutions.

RE: Making best use of 512mb Pi with tmpfs - JayBlanc - 13th Apr, 2013 05:19 AM

This seems to conflict with the zram improvements that are coming up, and needs to be reverted if you want to take advantage of them without the system bogging down. With zram swap loopback, I'm noticing a great improvement in responsiveness.

RE: Making best use of 512mb Pi with tmpfs - mk01 - 15th Apr, 2013 02:03 PM

despite the changes which are comming, I still like the idea of putting xbmc ./temp and some other caching files into memory.

debian is finally implementing standard linux features (8 years old!!!) like tmpfs shm on it etc - as standard feature.

@JayBlanc specially for this purpose I would choose /run/shm, not /tmp and /tmp I would leave solid. It's ok for you if you knows you system, but general many things can go wrong. /tmp is defined in posix with no limit in size. linux is changing /tmp from permanent to clean on reboot storage, but still no size limit.

feel free to play with /etc/default/tmpfs file, everything is managed via init according to those params.

and if you use mount --bind instead of delete / create symlinks, it will be completely transparent

(fstab records)
/run/shm /home/xbian/.xbmc/temp none bind 0 0
/run/shm /home/xbian/.xbmc/userdata/addon_data/script.common.plugin.cache none bind 0 0

RE: Making best use of 512mb Pi with tmpfs - namtih - 16th Apr, 2013 02:53 AM

Perhaps another idea would be vmtouch:
http://forum.xbian.org/thread-193-post-1783.html#pid1783

I added it to my startup script combined with some paths/files like

Code:

#!/bin/sh

#Preload Files

#touch /home/xbian/.xbmc/userdata/Database/zzstart_`date +%F_%T`

vmtouch -t /home/xbian/.xbmc/userdata/Database

vmtouch -t /home/xbian/.xbmc/userdata/addon_data/script.common.plugin.cache

vmtouch -t /home/xbian/.xbmc/addons/plugin.video.youtube

vmtouch -t /home/xbian/.xbmc/addons/plugin.video.dokumonster

vmtouch -t /home/xbian/.xbmc/addons/plugin.video.imdb.trailers

vmtouch -t /home/xbian/.xbmc/addons/plugin.video.the.trailers

#vmtouch -t /home/xbian/.xbmc/addons/plugin.video.hdtrailers_net-master

vmtouch -t /home/xbian/.xbmc/addons/script.globalsearch

vmtouch -t /usr/local/share/xbmc

vmtouch -t /usr/local/lib/xbmc

vmtouch -t /usr/lib/arm-linux-gnueabihf/gconv/gconv-modules.cache

#vmtouch -t /usr/share/pyload

vmtouch -t /usr/share/python

vmtouch -t /usr/bin/python2.7

vmtouch -t /usr/lib/python2.7

#vmtouch -t /home/xbian/.xbmc/addons/skin.realaskadark/fonts

#vmtouch -t /home/xbian/.xbmc/addons/skin.realaskadark/media/overlays

#vmtouch -t /home/xbian/.xbmc/addons/skin.realaskadark/media/views

vmtouch -t /home/xbian/.xbmc/addons/skin.refocus.big/fonts

vmtouch -t /home/xbian/.xbmc/addons/skin.refocus.big/media/overlays

vmtouch -t /home/xbian/.xbmc/addons/skin.refocus.big/media/views

vmtouch -t /usr/local/share/downloaddaemon

vmtouch -t /usr/local/bin/DownloadDaemon

vmtouch -t /usr/sbin/openvpn

vmtouch -t /etc/downloaddaemon

vmtouch -t /etc/openvpn/ovpn

vmtouch -t /lib/arm-linux-gnueabihf

vmtouch -t /bin/ntfs-3g

#touch /run/zzstop_`date +%F_%T`

exit 0

RE: Making best use of 512mb Pi with tmpfs - JayBlanc - 21st Jun, 2013 07:18 AM

Just a note that putting .xbmc/temp into a tmpfs absolutely breaks things in rar playback, because rar files get copied there.

RE: Making best use of 512mb Pi with tmpfs - mk01 - 23rd Jun, 2013 01:22 AM

@JayBlanc, you don't have the expansion memory module installed, to provide 8GB of ram? (it is for B model only).

Re: RE: Making best use of 512mb Pi with tmpfs - f1vefour - 23rd Jun, 2013 05:44 AM

(23rd Jun, 2013 01:22 AM)mk01 Wrote: @JayBlanc, you don't have the expansion memory module installed, to provide 8GB of ram? (it is for B model only).

You have a link for this?