[Part 2] Download torrent & newsserver package(s)

[namtih]

Regarding the NTFS issues, please try this idea first:
http://forum.xbian.org/thread-512-post-8826.html#pid8826

For me it's running fine since January.

[zilexa]

I read your tip before but I just couldn't find it anymore. Thanks! I just updated my fstab and did a reboot to be sure.

BIG edit.

with my NTFS raspberry, Transmission has troubles. I get this message regularly: "No data found! Ensure your drives are connected or use "Set Location". To re-download, remove the torrent and re-add it."

With the other raspberry, that has an EXT4 harddrive I never get this.. even with the mounting tip I get this error. so I will move to ext4 asap.

[Davem]

(16th Apr, 2013 09:38 PM)Enigmach Wrote:  I went through the same with a 1TB disk. Best way to go was indeed a simple Linux Live CD (I used Puppy Linux) and copied the stuff around. Resizing with GParted took waaaaay too long, and using ext2fsd plugin caused all sorts of file and folder permission issues.

Thanks for the tip. This was explaining the permission errors I was facing with Sickbeard and NZBget. If others are also facing permissions errors don't use Ext2Fsd as I suggested earlier.

For me the following steps were the solutions to solve the permission errors and get the download package to work:
- backup data to windows computer (for me NTFS)
- format external HDD to EXT4, using Gparted live cd (probably you could also directly use Puppy linux, this includes Gparted)
- copy the data from NTFS (in my case) to EXT4 using the puppy linux live cd
- change the permissions for external HDD as koenkk described here: https://github.com/xbianonpi/xbian/issues/248

Code:

sudo chmod -R 777 /media/*
sudo chown -R xbian:xbian /media/*

And now it's working! Thanks for the great package. Looking forward to also try the Flexget package zilexa is creating.

[zilexa]

Transmission is really a pain in the ass. After the new mount option Transmission doesnt give me the "set location" error anymore, but now I constantly get this error after a few MBs have been downloaded:

Code:

Error: File exists (/media/usb/TV Shows/Body of Proof/Season 3/Body of Proof S03E09 HDTV x264-LOL[ettv]/body.of.proof.309.hdtv-lol.mp4

I dont understand why this is happening! Its a new episode the file didnt exist before.

EDIT: I do NOT get this error when I manually add a torrent using the web interface.. only when Flexget adds a torrent this happens.

EDIT2: hopefully fixed. I made mistakes moving the /home/xbian/incomplete, /home/xbian/downloads folder to my USB drive. The symbolic links did exists but the actual folders did not exist on my USB drive. Not sure why some downloads did finish and others did not.

[salouma]

Hi,
In headphone i experiment issue when using my headphones acount for serching (beacause of limitation of musicbrainz.org

Search is not working on my raspberry pi (in my ubuntu desktop using local Headphones it works well)

It seems to be a issue with this package of headphones for Xbian

Thank you very much for XBian teem for providing such a great system !!!

[zilexa]

Transmission has the feature to run a script after a torrent has finished downloading. Would be great if we can use this feature to signal XBMC to update the library for just that one folder where the torrent has been downloaded (instead of a full update). I have found the command to update the video library but I do not know how to pass the folder data from Transmission via this script to XBMC:

Code:

wget -q "http://XBMCUSER:XBMCPASS@XBMCHOST:XBMCPORT/xbmcCmds/xbmcHttp?command=ExecBuiltIn&parameter=XBMC.updatelibrary(video,%LOCATION%)"

So %LOCATION% should simply be the folder location of the downloaded torrent... Unfortunately I cannot find any documentation on this. Is someone here already experimenting with this?

[Davem]

Maybe this can help. See the transmission part in combination with SickBeard
https://github.com/clinton-hall/nzbToMedia/wiki

[zilexa]

Thanks, I figured it out via old XBMC topics. HTTP API is deprecated so this is the JSON command to update a specific folder:

Terminal

curl -v -H "Accept: application/json" -H "Content-type: application/json" -d '{"id":5,"jsonrpc":"2.0","method":"VideoLibrary.Scan","params":{"directory":"/media/usb/TV Shows/"}}' http://192.168.1.1:8080/jsonrpc

or via browser(!):

Terminal

http://192.168.1.1:8080/jsonrpc?request={"jsonrpc": "2.0", "method": "VideoLibrary.Scan", "id": "5", "params": {"directory":"/media/usb/TV Shows/"}}

And to let Transmission pass the folder of the downloaded torrent, use this parameter as directory (untested): $TR_TORRENT_DIR

Since I am still very noobish, I have not figured out how to run the curl command or the url via ssh (or actually via an .sh script). "curl" is not available on XBian and I do not want to install it. Calling an url is ofcourse not working in SSH since you need a browser to process it. Transmission needs a script to run and it should be easy..

---
EDIT: Transmission does not run any .sh script it seems Transmission has no rights to run a script? How can I change this? all my script contains now (as a test) is: echo $TR_TORRENT_DIR/$TR_TORRENT_NAME >> media/usb/Downloads/autoconverted.log

[Lipeablaze]

Insert on the top of your .sh the follow:
#!/bin/sh
Save the file and run this command :
chmod +x nameOfScript.sh
Now the transmission should be able to run your script

[zilexa]

(fully edited this post)
Thanks! This script was executed successfully via transmission:

Code:

#!/bin/sh
echo $TR_TORRENT_DIR/$TR_TORRENT_NAME >> /media/usb/Downloads/autoconverted.log

Simply copying the location of the finished torrent to a logfile. So Transmission successfully executes this script.

Now I replace the command with the actual command and since XBian doesnt have curl I am using wget. I tested this command in SSH and it worked. So I expect it to work via the script as well:

Code:

#!/bin/sh
echo /usr/bin/wget --header='Content-Type:application/json' --post-data='{"jsonrpc": "2.0", "method": "VideoLibrary.Scan", "id": "5", "params": {"directory":"media/usb/TV Shows/"}}' "http://192.168.1.1:8080/jsonrpc"

But unfortunately nothing happens! I also tested without the /usr/bin part and both cases with and without ECHO but no difference why is this command working via SSH but not via the script?

[mk01]

Code:

#!/bin/sh

{ printf "get /jsonrpc?request={\"jsonrpc\": \"2.0\", \"method\": \"VideoLibrary.Scan\", \"id\": \"5\", \"params\": {\"directory\":\"/media/usb/TV Shows/\"}}\r\n"; sleep 2} | telnet 192.168.1.1 8080

[zilexa]

Wow, that looks complicated. Unfortunately it did not work 'out of the box'.

How can a command start with a "{"? and I dont understand the first part, printf, then "get" and then part of the required url. The rest I understand, but how will this make the call 192.168.1.1:8080/jsonrpc ?
Perhaps it didnt work because this is a "get", not a "post"?

[mk01]

)

it can work, because of shell expansion, expressions, redirects and standard outputs and inputs

try

Code:

{ echo "get /index.html\r\n"; sleep 1; } | telnet xbianip 8080

my output

Code:

(02:00) [media:~] amk$ { printf "get /index.html\r\n"; sleep 2; } | telnet xbian 8080
Trying 192.168.1.126...
Connected to xbian.private.
Escape character is '^]'.
<!DOCTYPE html>
<html>
  <head>
    <title>XBMC</title>
    <meta http-equiv="Content-Language" content="EN" />
    <meta http-equiv="Content-Type" content="UTF-8" />
    <!-- <link rel="search" href="provider.xml" type="application/opensearchdescription+xml" title="XBMC Library" /> -->
    <link rel="icon" href="favicon.ico" type="image/x-icon">
    <link rel="shortcut icon" href="favicon.ico" type="image/x-icon">
    <!-- <link href="/images/logo.png" rel="image_src" /> -->
    <meta name="robots" content="NOINDEX, NOFOLLOW">
    <link href="css/core.css?1.3.57" rel="stylesheet" type="text/css">
    <link href="css/ipad.css?1.0.5" rel="stylesheet" media="only screen and (max-device-width: 1024px)" type="text/css">
  </head>

if the command I sent you is not working, then it must be typo somewhere... I see probably where

after sleep 2 must be ";" ant speace, before } ... so I try again:

Code:

#!/bin/sh

{ printf "get /jsonrpc?request={\"jsonrpc\": \"2.0\", \"method\": \"VideoLibrary.Scan\", \"id\": \"5\", \"params\": {\"directory\":\"/media/usb/TV Shows/\"}}\r\n"; sleep 2; } | telnet 192.168.1.1 8080

---

and the point how it works is:

printf just takes text between " " and puts it on stdout
the command inside " " starting with get is nothing else as direct communication to http server. protocols GET and POST.
that part after "get " is exactly the same (should be), what you type into address bar of your browser. what we do, we don't send the command via browser, but via telnet.

telnet by default listens on stdin for commands. and we do it with

Code:

printf "text" | telnet

and now the interesting part - and we come to the { }. printf opens stdout, sends text "text" and !closes! output. what would make telnet to close as well and therefore terminate the connection. take telnet here like some kind of glasses to other side. so we need to wait few ms or s for the data to be transfered back.

so we use sleep. ";" is used to separate commands on one line. normally you do:

Code:

echo "1"; echo "2"; echo "3"

but if you do like this

Code:

echo "1" ; sleep 2 | telnet

wouldn't work, because this would send "1" on screen and then sleep's 2 output send to telnet

so you need something like buffer (accumulate) one buffer from more programs... and then send to telnet.

so you can use shells alternation ... or brace expansion is it called, i don't know. it just combines the commands (data) inside

try funny things like:

Code:

echo s,e{x,y}

[wietze314]

Hello all,

Thanks all for the good pointers in this thread. So far I have configured my xbian with xbmc to download stuff with sickbeard and transmission.

The problems I ran in to were:
- bad performance with 2tb harddrive with NTFS filesystem.
Solved this by reformatting it with ext4 (via puppy linux cd)

- repeated post processing by sickbeard:
this was caused by the fact that I had sickbeard watch my download directory and also keep the original files. So the files would be post processed infinitly. It made my XBMC very slow.
Files need to stay in the directory for transmission to seed them for a while.

To solve this I made my own transmission postprocessing script that copies the finished download to another directory for sickbeard to postprocess. In which sickbeard is allowed to delete the files.

- RAR files did not get processed by sickbeard.
For this I installed unrar-nonfree (http://raspberrypi.stackexchange.com/questions/3617/how-to-install-unrar-nonfree) and added a line to my transmission postprocessing script. The script is not very suffisticated but maybe someone finds it helpful:

Code:

cat /usr/local/etc/transmission/transmission-complete.sh
#!/usr/bin/env bash

LOGFILE="/usr/local/etc/transmission/transmission-complete.log"

function edate
{
  echo "`date '+%Y-%m-%d %H:%M:%S'`    $1" >> "$LOGFILE"
}

edate "Directory is $TR_TORRENT_DIR"
edate "Torrent ID is $TR_TORRENT_ID"
edate "Torrent Hash is $TR_TORRENT_HASH"
edate "Working on the new download $TR_TORRENT_DIR/$TR_TORRENT_NAME"


if [ -e "$TR_TORRENT_DIR/$TR_TORRENT_NAME" ]; then
cp -R "$TR_TORRENT_DIR/$TR_TORRENT_NAME" "/media/networkdrive/downloads/sickbeard-postprocessing" >> "$LOGFILE"
unrar x -y -p- -o+ "/media/networkdrive/downloads/sickbeard-postprocessing/$TR_TORRENT_NAME/*.rar" "/media/networkdrive/downloads/sickbeard-postprocessing/$TR_TORRENT_NAME/" >> "$LOGFILE"
fi

The thing I now want to do is have more influence on the sickbeard torrent searches. It seems to me that it does not really look at the torrent availability while searching. Since I dont have invites to those private trackers, I seem to be pulling in some torrents, that I manually won't choose.
Is there some easy way to influence the search behavior. For example:
- force it to take only torrents with more than x peers.
- prefer torrents with certain strings in the torrent name?

Thanks for the help!

[Kevf]

in the OP:

SABnzbd port 9092

but that's the port for nzbget.

edit:I have an external HDD. When I download something with nzbget, the file will be put in XBian/downloads
I used the commands in the OP, including the ls (although I don't fully understand the logica between the linking).

With this setting the file will be put on the sd card instead of on the hdd. Shouldn't the file be in the Xbian/downloads/downloads?

I tried setting the path in nzbget with /devices/usb0 (edit: changed it to /media/usb0, works!)
but that doesn't seem to work.