Threaded Mode | Linear Mode

nertskull · 2nd Oct, 2013, 09:51 PM

(2nd Oct, 2013 11:38 AM)mk01 Wrote: and to clear my confusion now, with MB/s you mean MegaBits (Mb) per sec, not megabytes (MB/Mbyte), right? as RPI can hardly do over 15MB/s. 120MB would be quite much even for standard PC on 1Gbit network.

So I'm pretty sure its actually MB as in MegaByte. Here is the output from "dd"

Code:

xbian@xbian ~ $ dd if=/media/Aspartic_Acid/media/video/Movies/ProcessedBD/The_Lord_Of_The_Rings_The_Fellowship_Of_The_Ring_2001_extended_Disc_1.mkv of=/dev/null bs=1M cou

nt=150

150+0 records in

150+0 records out

157286400 bytes (157 MB) copied, 14.924 s, 10.5 MB/s

I'm not great at this stuff, but even if I calculate by hand the 157286400 bytes in 14.924 s then you end up with ~10 megabytes/s

So unless dd is outputting the wrong message it should be megabytes.

And here is the dd output on raspbmc for comparison

Code:

pi@raspbmc:~$ dd if=/media/Aspartic_Acid/media/video/Movies/ProcessedBD/The_Lord_Of_The_Rings_The_Fellowship_Of_The_Ring_2001_extended_Disc_1.mkv of=/dev/null bs=1M count

=150

150+0 records in

150+0 records out

157286400 bytes (157 MB) copied, 1.29605 s, 121 MB/s

pi@raspbmc:~$ dd if=/media/Aspartic_Acid/media/video/Movies/ProcessedBD/The_Lord_Of_The_Rings_The_Fellowship_Of_The_Ring_2001_extended_Disc_1.mkv of=/dev/null bs=1M count

=150

150+0 records in

150+0 records out

157286400 bytes (157 MB) copied, 1.28328 s, 123 MB/s

pi@raspbmc:~$ dd if=/media/Aspartic_Acid/media/video/Movies/ProcessedBD/The_Lord_Of_The_Rings_The_Fellowship_Of_The_Ring_2001_extended_Disc_1.mkv of=/dev/null bs=1M count

=150

150+0 records in

150+0 records out

157286400 bytes (157 MB) copied, 1.29872 s, 121 MB/s

pi@raspbmc:~$ dd if=/media/Aspartic_Acid/media/video/Movies/ProcessedBD/The_Lord_Of_The_Rings_The_Fellowship_Of_The_Ring_2001_extended_Disc_1.mkv of=/dev/null bs=1M count=150

150+0 records in

150+0 records out

157286400 bytes (157 MB) copied, 2.05953 s, 76.4 MB/s

You're right though. I never noticed till now. But I thought the specs for the Pi were on 10/100 Internet. So I'm not sure how on raspbmc its pulling that fast. This is confusing to me.

Quote:So if then recalculate 10.5Mb you report for XBian that's not even 1.5MB/s and with this speed I'm surprised FullHD DTS movie even plays. Can you post your fstab?

There will be something broke as If I try hard (stop XBMC etc) I can get 12.5MB/s from my NFS4 box so even if I admit not everyone running 3com network etc, even on china stuff the difference can not be 1:10. Simply told I would accept to hear 10, 9, maybe even 7. Considering it is only NAS (so not ultra CPU power). But not 1.5. Don't worry, we find it

Here is my fstab

Code:

#

#

#

10.23.24.137:/media/Aspartic_Acid /media/Aspartic_Acid nfs _netdev,defaults,user,auto,noatime,intr 0 0

/dev/mmcblk0p1         /boot           vfat            rw        0       1

/dev/root             /home                   xbian   subvol=home/@,noatime        0       0

/dev/root             /lib/modules            xbian   subvol=modules/@,noatime        0       0

none            /run/shm                        tmpfs                   noauto                  0       0

none            /run/user                       tmpfs                   noauto                  0       0

none            /sys/kernel/debug               debugfs                 noauto                  0       0

none            /sys/kernel/security            securityfs              noauto                  0       0

Which is interesting, because when I put it in, I know I put the NFS mount at the bottom. Don't know how it got moved to the top. I don't remember every getting back into the fstab since that first time.

I'm wondering, does the pi cache a video locally? So even though I'm asking it to pull from the NFS, maybe its really pulling from a local cache to get that fast on raspbmc? Because I can't get my head around those numbers. The specification clearly say its 100Mb max. My dd command shows upwards of 800Mb. I don't get it.

**IriDium** · 3rd Oct, 2013, 12:49 AM

I've just done a Test on Beta 1.2 vs Raspbmc - albeit via USB HDD and not NFS.

Same hardware - same RPi, same file.

Xbian ~ 18MB/s
Raspbmc ~ 15MB/s

If I'm correct - USB2 max speed is ~60 MB/s, and the N/W 10/100 is max 100mbits or ~ 12MB/s.

**rikardo1979** · 3rd Oct, 2013, 02:27 AM

so after reading the inputs from all of you I start to think if there is maybe and some firmware/kernel mismatch or something like that.I dont see that actual XBMC or XBian causing this issue.
But as I said, its just my feeling

nertskull · 3rd Oct, 2013, 07:42 AM

(3rd Oct, 2013 12:49 AM)IriDium Wrote: I've just done a Test on Beta 1.2 vs Raspbmc - albeit via USB HDD and not NFS.

Same hardware - same RPi, same file.

Xbian ~ 18MB/s
Raspbmc ~ 15MB/s

If I'm correct - USB2 max speed is ~60 MB/s, and the N/W 10/100 is max 100mbits or ~ 12MB/s.

Allright, I agree. It doesn't make any sense to see raspberry pi speeds that fast. Its gotta be a caching issue. Because I've run it again, and I get more normal results the first time, then something faster.

Code:

pi@raspbmc:~$ dd if=/media/Aspartic_Acid/media/video/Movies/ProcessedBD/The_Lord_Of_The_Rings_The_Fellowship_Of_The_Ring_2001_extended_Disc_1.mkv of=/dev/null bs=1M count=150

150+0 records in

150+0 records out

157286400 bytes (157 MB) copied, 22.3734 s, 7.0 MB/s

pi@raspbmc:~$ dd if=/media/Aspartic_Acid/media/video/Movies/ProcessedBD/The_Lord_Of_The_Rings_The_Fellowship_Of_The_Ring_2001_extended_Disc_1.mkv of=/dev/null bs=1M count=150

150+0 records in

150+0 records out

157286400 bytes (157 MB) copied, 1.32309 s, 119 MB/s

So 7MB/s the first time. I've tried it with different files that haven't been played at all. And they also show reasonable speeds in the 7-8 MB/s neighborhood.

So I take back that its probably an NFS issue. The NFS seems to be fine both on raspbmc and Xbian.

(3rd Oct, 2013 02:27 AM)rikardo1979 Wrote: so after reading the inputs from all of you I start to think if there is maybe and some firmware/kernel mismatch or something like that.I dont see that actual XBMC or XBian causing this issue.
But as I said, its just my feeling

I don't know how to do anything about that.

But I'm working on finding a way to get a minute or two of my video saved. I'll post once I've got that figured out as a sample for people.

Thanks

mk01 · 3rd Oct, 2013, 08:14 AM

(2nd Oct, 2013 09:51 PM)nertskull Wrote:
(2nd Oct, 2013 11:38 AM)mk01 Wrote: and to clear my confusion now, with MB/s you mean MegaBits (Mb) per sec, not megabytes (MB/Mbyte), right? as RPI can hardly do over 15MB/s. 120MB would be quite much even for standard PC on 1Gbit network.

So I'm pretty sure its actually MB as in MegaByte. Here is the output from "dd"

Code:

xbian@xbian ~ $ dd if=/media/Aspartic_Acid/media/video/Movies/ProcessedBD/The_Lord_Of_The_Rings_The_Fellowship_Of_The_Ring_2001_extended_Disc_1.mkv of=/dev/null bs=1M cou nt=150 150+0 records in 150+0 records out 157286400 bytes (157 MB) copied, 14.924 s, 10.5 MB/s

10.5 is ok and nice. but before you posted this:

nertskull Wrote:So that is interesting. With raspbmc doing the file transfer over NFS I ran a bunch of times and average about 120 MB/s.

that's why I was wondering. now with the two runs with dd and posting whole output it makes sense.

(2nd Oct, 2013 09:51 PM)nertskull Wrote: Which is interesting, because when I put it in, I know I put the NFS mount at the bottom. Don't know how it got moved to the top. I don't remember every getting back into the fstab since that first time.

I'm wondering, does the pi cache a video locally? So even though I'm asking it to pull from the NFS, maybe its really pulling from a local cache to get that fast on raspbmc? Because I can't get my head around those numbers. The specification clearly say its 100Mb max. My dd command shows upwards of 800Mb. I don't get it.

xbian-update scripts is doing that reordering. I hope you don't mind. As using mountall and upstart the order actually does not matter there.

PI is not caching videos locally, even XBMC does not (version < 13). only in case of internet sources, there is cache in memory.

as you now specified that the 120MB/s was from raspbmc I remembered that yes, RaspBMC is caching NFS mounts locally - what is quite killing the SD cards but ok. Their choice.

Taking into account that you have higher NFS transfer speeds with XBian than with RaspBMC then there two options now. Your NAS is not providing sustainable speed over the period while watching movies - and that's why RaspBMC caching is helping there - or it's like Rikardo said.

How big are the BRRips you have? What is the video and audio nitrate?

mk01 · 3rd Oct, 2013, 09:43 AM

I now tried mounting my media storage with caching enabled and got this results:

Code:

root@xbian02:~# dd if=/mnt/media/movies/file.mkv of=/dev/null bs=1M count=150

150+0 records in

150+0 records out

157286400 bytes (157 MB) copied, 17.0417 s, 9.2 MB/s

root@xbian02:~# dd if=/mnt/media/movies/file.mkv of=/dev/null bs=1M count=150

150+0 records in

150+0 records out

157286400 bytes (157 MB) copied, 0.890908 s, 177 MB/s

if you would like to test such setup follow this:

Code:

sudo -i

apt-get install cachefilesd

btrfs-auto-snapshot createvol fscache

into /etc/fstab put following line:

Code:

/dev/root               /var/cache/fscache     xbian   subvol=fscache,noatime        0       0

and change your NFS mount in /etc/fstab by adding "fsc" as mount option so you get:

Code:

10.23.24.137:/media/Aspartic_Acid /media/Aspartic_Acid nfs _netdev,defaults,user,auto,noatime,intr,fsc 0 0

change /etc/default/cachefilesd and un-comment #RUN=yes
reboot

@IriDium

haven't you had this one problem with your NAS as well when we tried to hack it to support NFS exports?

nertskull Wrote:In regards to this, I just found this out. I thought I should maybe try with the xbmc nfs settings. But I actually can't. So maybe that plays into this. But I can find my nfs share with xbmc. I can see the shared drives. But xbmc won't let me actually open those shared drives.

nertskull · 3rd Oct, 2013, 09:55 AM

Allright, I don't know what people use to upload files. So I'm just going to put it on dropbox. I probably won't leave it on for too long, since I don't know what the legalities are. I'll probably take it down in a day or so.

I've tried it on my Xbian and it still skips. So hopefully it does for your systems as well. Thanks.

File:

edited out link to file

mk01 · 3rd Oct, 2013, 10:05 AM

2 minutes and I have it. Ok, got the file. I will test tomorrow.

How you did do the sample? Share the info please as it could be useful for others (myself included) as well.

nertskull · 3rd Oct, 2013, 10:08 AM

(3rd Oct, 2013 08:14 AM)mk01 Wrote: 10.5 is ok and nice. but before you posted this:

xbian-update scripts is doing that reordering. I hope you don't mind. As using mountall and upstart the order actually does not matter there.

Nope thats fine, didn't know it reordered it. But now I know it does and I won't worry about it.

Quote:PI is not caching videos locally, even XBMC does not (version < 13). only in case of internet sources, there is cache in memory.

as you now specified that the 120MB/s was from raspbmc I remembered that yes, RaspBMC is caching NFS mounts locally - what is quite killing the SD cards but ok. Their choice.

Good to know, another reason I can add to the list of why I would like to get xbian working.

Quote:Taking into account that you have higher NFS transfer speeds with XBian than with RaspBMC then there two options now. Your NAS is not providing sustainable speed over the period while watching movies - and that's why RaspBMC caching is helping there - or it's like Rikardo said.

How big are the BRRips you have? What is the video and audio nitrate?

So the BD Rip of the LOTR I have is pretty big. Its split over two discs (extended version) and each one is around 12 GiB and my dts audio track is 1536 kb/s (according to VLC)

But I've tried other smaller ones. For example I have a Muppets From Space HD video also with DTS that stutters. It is only 720. But the same 1536 kb/s bitrate. The file size is only 6 GiB though.

I hope that made sense. I'm not super familiar with mkv files and bitrates and such things yet. [/quote]

(3rd Oct, 2013 10:05 AM)mk01 Wrote: 2 minutes and I have it. Ok, got the file. I will test tomorrow.

How you did do the sample? Share the info please as it could be useful for others (myself included) as well.

So to make the sample I used the mkvtoolnix package. I use that to create all my mkv files and its pretty versatile.

To make this specific one. I used the MKV Merge GUI. In the "Global" options tab there is an option for splitting. And I just split it at the 2 hour mark. The movie is like 2 hours and 3 minutes. So the 2nd file was just the ending credits.

If people aren't familiar with it, the mkvtoolnix program is pretty awesome.

http://www.bunkus.org/videotools/mkvtoolnix/

Just load the video. Go to global tab. And select your splitting options. Then "Start muxing". Super simple. And its cross platform. I run linux, and have only tested it there, but I'm pretty sure it can run on windows and mac osx also

mk01 · 3rd Oct, 2013, 10:13 AM

(3rd Oct, 2013 10:08 AM)nertskull Wrote: So the BD Rip of the LOTR I have is pretty big. Its split over two discs (extended version) and each one is around 12 GiB and my dts audio track is 1536 kb/s (according to VLC)

But I've tried other smaller ones. For example I have a Muppets From Space HD video also with DTS that stutters. It is only 720. But the same 1536 kb/s bitrate. The file size is only 6 GiB though.

I hope that made sense. I'm not super familiar with mkv files and bitrates and such things yet.

no no this info is enough for me, just wanted to get idea. it is true I have not tried mkv 24gb in size but then when you can't play even 6gb it is more than below the size I tested (without passthrough). it was 15gb. I don't have bigger.

lets see tomorrow, in-between you can always try turning on the caching.

nertskull · 3rd Oct, 2013, 10:19 AM

(3rd Oct, 2013 10:13 AM)mk01 Wrote:
(3rd Oct, 2013 10:08 AM)nertskull Wrote: So the BD Rip of the LOTR I have is pretty big. Its split over two discs (extended version) and each one is around 12 GiB and my dts audio track is 1536 kb/s (according to VLC)

But I've tried other smaller ones. For example I have a Muppets From Space HD video also with DTS that stutters. It is only 720. But the same 1536 kb/s bitrate. The file size is only 6 GiB though.

I hope that made sense. I'm not super familiar with mkv files and bitrates and such things yet.

no no this info is enough for me, just wanted to get idea. it is true I have not tried mkv 24gb in size but then when you can't play even 6gb it is more than below the size I tested (without passthrough). it was 15gb. I don't have bigger.

lets see tomorrow, in-between you can always try turning on the caching.

yeah i'm going to mess with the caching a little bit tonight and tomorrow and will post how things go.

Thanks for all the continued help. I love communities like this.

Allright. I set up all the caching stuff as you described.

Unfortunately I still get the stuttering/skipping every 5-10 seconds as before. So I guess that didn't help.

But I do think I got caching to work. I can get the really fast speeds in xbian now. Just still get the stuttering.

Code:

xbian@xbian ~ $ dd if=/media/Aspartic_Acid/media/video/Movies/ProcessedBD/The_Lord_Of_The_Rings_The_Two_Towers_2002_extended_Disc_1.mkv of=/dev/null bs=1M count=100

100+0 records in

100+0 records out

104857600 bytes (105 MB) copied, 8.9534 s, 11.7 MB/s

xbian@xbian ~ $ dd if=/media/Aspartic_Acid/media/video/Movies/ProcessedBD/The_Lord_Of_The_Rings_The_Two_Towers_2002_extended_Disc_1.mkv of=/dev/null bs=1M count=100

100+0 records in

100+0 records out

104857600 bytes (105 MB) copied, 0.583944 s, 180 MB/s

**IriDium** · 3rd Oct, 2013, 11:45 PM

(3rd Oct, 2013 09:43 AM)mk01 Wrote: @IriDium
haven't you had this one problem with your NAS as well when we tried to hack it to support NFS exports?

@mk01 I don't remember a buffer problem. It was only that the NAS was R/O so it reset itself everytime you rebooted and exportfs didn't make a difference.
@nertskull Have you actually timed the transfers or just relying on what "dd" says?

nertskull · 4th Oct, 2013, 02:50 AM

I've not timed them using "time" but I can tonight. But they sync up in my head. I have been paying attention to that. When it says it takes 15s, it takes close to 15s. When it says it takes 0.5s, it really does finish almost instantaneous. So I don't doubt that the dd times are at least in the ballpark. You can tell the difference. But I can check tonight to make sure they are correct.

**IriDium** · 4th Oct, 2013, 03:05 AM

Probably no issue: If I "believe" in the output but don't check,then I am a fool. I like to know, that it's not a figment of ones imagination.

Just count from start to finish.

I'm either impressed with Raspbmc or very confused. The figures posted do not make sense. Either you have the best RPi manufactured with optimal Raspbmc and can achieve speeds that defy physics or something is wrong.

mk01 · 4th Oct, 2013, 03:30 AM

(3rd Oct, 2013 11:45 PM)IriDium Wrote:
(3rd Oct, 2013 09:43 AM)mk01 Wrote: @IriDium
haven't you had this one problem with your NAS as well when we tried to hack it to support NFS exports?
@mk01 I don't remember a buffer problem. It was only that the NAS was R/O so it reset itself everytime you rebooted and exportfs didn't make a difference.

no you didn't read what I quoted. I mean connect from XBMC via NFS. I'm not sure it was you (but remember discussed with someone) so that's why I asked.

It was like NAS + hacked NFS and XBMC has seen the export, but after opening them they were empty. Or not accessible.

(3rd Oct, 2013 11:45 PM)IriDium Wrote: @nertskull Have you actually timed the transfers or just relying on what "dd" says?

@IriDium,

dd is quite correct there, there is no reason not to believe it. and while using NFS caching, the timing is like it is.

memory in RPI can give you ~250MB/s so reading a file from local NFS cache (which is on SD) - but still in FS buffers - memory - will give you the file back with speed like this.

@IriDium

have you actually tried the sample? I'm just going to do.