Forum
[IDEA] Dynamic priority process - Printable Version

+- Forum (http://forum.xbian.org)
+-- Forum: Software (/forum-6.html)
+--- Forum: Testing & Experimental (/forum-21.html)
+--- Thread: [IDEA] Dynamic priority process (/thread-1107.html)

Pages: 1 2

RE: Dynamic priority process - belese - 25th Jul, 2013 06:32 PM

Thanks Curly,
but it's not a client/server socket that i develop myself,
i don't write the server, just the client.
i've to connect to netlink socket, and used the netlink protocol.

otherwise, i know json, lot of api use it, xbmc upstart bridge use the json notification from xbmc.

RE: Dynamic priority process - belese - 26th Jul, 2013 11:17 PM

i've updated starting condition for ipconnection bridge.
rewriting it with netlink seems out of my knowledge, i've try, but really don't find how to do,
i've also writed a priority management job for transmission.

default priority can be set in
/etc/default/transmission

variable that can be set :
PHIGH='-5'
PNORMAL='0'
PLOW='10'
DPRIORITY=yes

PHIGH will be set when access by web
PNORMAL when xbmc is not used (xbmcplevel 0-1)
PLOW when xbmc is used
DPRIORITY activate dynamic priority or not

i'll try to rewrite the xbmc one to looks like this one, and other download ones after.

ps :
seems there's a bug in ipconnection bridge,
for transmission, it didn't detect end of connection now.

RE: Dynamic priority process - mk01 - 12th Aug, 2013 04:02 PM

(26th Jul, 2013 11:17 PM)belese Wrote:  ps :
seems there's a bug in ipconnection bridge,
for transmission, it didn't detect end of connection now.

belese, as always too late, but still maybe you use in future.

with standard netfilter it's easy as:
Code:
root@media:/var/log# iptables -L -n -v
Chain INPUT (policy ACCEPT 292 packets, 32127 bytes)
pkts bytes target     prot opt in     out     source               destination        

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target     prot opt in     out     source               destination        

Chain OUTPUT (policy ACCEPT 165 packets, 31081 bytes)
pkts bytes target     prot opt in     out     source               destination

normally those chains are meant for firewalling / routing, but we can downsize it's function to just count packets / bytes.

so if you want to monitor for instance port 22 (ssh), add a rule to ACCEPT
Code:
iptables -I INPUT -j ACCEPT -p tcp --dport 22

with result
Code:
root@media:/var/log# iptables -L -n -v
Chain INPUT (policy ACCEPT 473 packets, 50461 bytes)
pkts bytes target     prot opt in     out     source               destination        
   17  2053 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:22

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target     prot opt in     out     source               destination        

Chain OUTPUT (policy ACCEPT 289 packets, 49084 bytes)
pkts bytes target     prot opt in     out     source               destination

you can poll this numbers with grep or let it send to a file descriptor (opened by variant of your python script which listens to the xbmc events). from this point of view, it;s the same. you just use LOG policy instead of accept.

and to get rid of calculations in the bridge script, you can directly define priority targets directly in netfilter, what means according to current bandwidth (not only absolute pkt/bytes numbers) to MARK and then LOG.

so at the end you would have in input chain four rules, with target MARK, described not by port number but RATE, followed by one rule to LOG, which would on some transfer send a human readable text line MARKed already according your priority split.

so the bridge daemon would be just a loop with case and initctl emit.

RE: Dynamic priority process - belese - 13th Aug, 2013 01:35 AM

before writing daemon, i was looking in iptable and that's really what i need,

For polling, i always thinking that it's not a good solution (maybe i'm wrong)

and to react on event i don't find the way to run a script insteadl of log, accept,...) in iptables.
after search, i've read it's not possible,
the work around i've seen (and if i understand you, is what you explain),
is to use LOG in iptables, and the daemon read log file, and when specific log is writed, react on this.
this is what you explain, right?

fo netfilter, no sure iv'e understand, only thing i need is tcp (maybe udp) socket state changed.

my idea is :
socket 9091 is CONNECTED, i've a transmission http session, i can renice transmission lower.
socket 9091 is LISTEN, no transmission session, renice transmission higher.

so why bandwith could be usefull in this ?

RE: Dynamic priority process - mk01 - 13th Aug, 2013 02:36 AM

(13th Aug, 2013 01:35 AM)belese Wrote:  after search, i've read it's not possible,
the work around i've seen (and if i understand you, is what you explain),
is to use LOG in iptables, and the daemon read log file, and when specific log is writed, react on this.
this is what you explain, right?

fo netfilter, no sure iv'e understand, only thing i need is tcp (maybe udp) socket state changed.

yes, LOG to rsyslog and rsyslog to socket. no polling. LOG is writing to syslog via 514/tcp/udp so if you don't use it, you can bind directly on 514 and just wait - never tried parsing 514/syslog tcp communication.

but for this CONNECT / LISTEN Smile, just create rule to dup packet. one will go to original destination, second to you bind port Wink

iptables -A PREROUTING -t mangle -j TEE -m state --state ESTABLISHED -p tcp --dport XX --gateway 127.0.0.1

(will copy & forward packet to 127.0.0.1, which was part of established connection to your box on port XX)

based on actual rate / bandwidth you could change the priority in more steps... somehow I though you want this.
but when I'm thinking, the easy effective - whole daemon

Code:
conntrack -E -e NEW,DESTROY -p tcp --dport 22 | \
while read line; do  
   case $line in
     *NEW*)
        initctl emit -n NEW_CONNECTION_priority_high
        ;;
     *DESTROY*)
        initctl emit -n CLOSED_CONNECTION_EXPIRED_priority_low
        ;;
    esac
done


RE: Dynamic priority process - belese - 14th Aug, 2013 02:47 AM

Hi great, looks so simple.

but i've tried, and don't work
is possible that module are not enabled in kernel?
i've read this page
http://conntrack-tools.netfilter.org/manual.html
Quote:Verifying kernel support
To check that the modules are enabled in the kernel, run `conntrack -E' and generate traffic, you should see flow events reporting new connections and updates.
when i run conntrack -E, nothing appear, even if i generate traffic.

i'm on beta2,
kernel : Linux xbian 3.10.4+
and it is conntrack from raspian repository

RE: Dynamic priority process - mk01 - 14th Aug, 2013 03:37 AM

(14th Aug, 2013 02:47 AM)belese Wrote:  when i run conntrack -E, nothing appear, even if i generate traffic.

i'm on beta2,
kernel : Linux xbian 3.10.4+
and it is conntrack from raspian repository

nothing more is needed, conn track module should be loaded auto. just the conn track package is not installed by default (binary).

check if nf_conntrack_ipv4 is loaded. or straight go for "modprobe nf_conntrack_ipv4"
btw: this is tracking changes in connection table, not packet traffic (just for info)

RE: Dynamic priority process - belese - 14th Aug, 2013 05:19 AM

Thanks, have to load the module, work now.