PDA

Bekijk de volledige versie : SABnzbd with par2cmdline on WL500gP



DrChair
08-04-2007, 22:42
SABnzbd is a tool for downloading from usenet. After you feed it a nzb-file, it starts downloading, verifying/repairing using par2 and unpacking using unrar/unzip.

Prerequisites:
- You need to be using the OLEG optware repository
- You need to have a swap-file

The Howto:

Install the necessary packages:



ipkg update

ipkg install py-sabnzbd
or if you want to use python2.5
ipkg install py25-sabnzbd

ipkg install par2cmdline
ipkg install unrar
ipkg install unzip


The SABnzbd package creates the following directories:

download_dir = /opt/tmp/SABnzbd/tmp/
complete_dir = /opt/tmp/downloads/
nzb_backup_dir = /opt/tmp/SABnzbd/nzb/backup/
cache_dir = /opt/tmp/SABnzbd/cache/
log_dir = /opt/var/log/
dirscan_dir = /opt/tmp/SABnzbd/nzb/

If thats oke, you only need to fill in your newsservers in /opt/etc/SABnzbd.ini.
Otherwise, make new directories and change these aswell in the .ini.

The package also installs a startup-script, S70SABnzbd.

Finally, this package contains a few patches i made.
The first one fixes the amount of disk-free-space reported. The second one adds two new options: pause download during assembly and pause download during postprocessing. Both are enabled in the default ini-file the package installs.
(if you already have SABnzbd installed and are keeping your old ini-file, you might have to add these two options manually to use them.)

You can start SABnzbd, using the startup-script or a reboot.
The webinterface is at http://192.168.1.1:8080/sabnzbd/

Happy leeching.

Cr4sh3d
09-04-2007, 15:21
Hi . thx for the tutorial ,

But im running into to a few problems after the installation
1. is it didnt find celementtree i found out a fix for this was ipkg install py-celementtree instead of your ipkg install py-elementtree .
2. It still cannot find a few files like rar and unzip while installed with ipkg they seem not detected by Sabnzbd even the chrooted par2 are not found .:confused: :confused:

typing in console however give me these files further more my web-iface seems stuck its a white screen where a cherry is seen in browser tabpage the page itself is empty , only thing thats bugging there seems to be a port ?
I am not sure but i found my only error is it clames port is not free .:o

As u see im in need for some help ,maybe u can help me on my way .

Ps i had to use a slighty diffrent setup from your dir setup but i double checked and cant find any errors in that . :|

Thanks in advance

DrChair
09-04-2007, 21:47
1. is it didnt find celementtree i found out a fix for this was ipkg install py-celementtree instead of your ipkg install py-elementtree .


cElementtree is optional. Installing it might speed-up things, but it should work without.
More speed is a good thing though, so I'll add it to the howto.



2. It still cannot find a few files like rar and unzip while installed with ipkg they seem not detected by Sabnzbd even the chrooted par2 are not found .:confused: :confused:


In my case, it seems that SABnzbd can't find those files, if S70SABnzbd is started by the startup-script rc.unslung. If I manually start S70SABnzbd, it does find those files. I'll need to look into this.



typing in console however give me these files further more my web-iface seems stuck its a white screen where a cherry is seen in browser tabpage the page itself is empty , only thing thats bugging there seems to be a port ?
I am not sure but i found my only error is it clames port is not free .:o


are you using http://192.168.1.1:8080/sabnzbd/ ??
or perhaps you need to change the port-number in the ini-file.

Cr4sh3d
15-04-2007, 21:46
Any progress bro in solving that yet ?

DrChair
17-04-2007, 23:18
Any progress bro in solving that yet ?

nope... I tried several things, but everytime when I let it startup automaticly, it doesn't find par2, unrar and unzip. The strange thing is, that it does find yenc and celementtree.

for now, I just renamed S70SABnzbd to K70SABnzbd, so that it doesn't start during the boot, and start it manually afterwards.

Cr4sh3d
27-04-2007, 14:44
I found that when i tried the webface of sabnzbd it came back with an internal error . :mad:
Now you and me might found that but i doubt if all users will find it .
So if you please consider in adding this line in your tutorial.With your favorite editor , edit /opt/etc/SABnzbd.ini and look for this lines

## web file dir for (custom) Cheetah templates and the default.css file
web_dir = ""

and change them into

## web file dir for (custom) Cheetah templates and the default.css file
web_dir = "/opt/local/share/SABnzbd-0.2.5/templates/"

then your webface of sabnzbd will be working . Atleast mine does :)
Thanks DrChair for updating things .

sollie
25-05-2007, 17:57
Thanks, i have it running.

I have only one problem left. I use a WL700gE and those devices do not have an writable /.
I can only use my /opt path. So i cant use the yenc fix.

Does anybody has an idea.

Sollie.

DrChair
27-05-2007, 21:26
Thanks, i have it running.

I have only one problem left. I use a WL700gE and those devices do not have an writable /.
I can only use my /opt path. So i cant use the yenc fix.

Does anybody has an idea.

Sollie.

The WL500gP also doesn't have a writeable /. That's why i mount a folder that is writable over the exisiting /home, so that you get a writable /home.
(mount --bind /opt/tmp/home /home)

sollie
28-05-2007, 08:16
Yeah, ok. But i do not have an /home dir and i cant create it.

mount: mounting /opt/tmp/home on /home failed

Sollie.

rowi
29-05-2007, 07:23
Hi all,

Thanks for the very clear tutorial, always good for a relative newbie on linux like me.:D

I do have a question though, i'have it all working on an wl500gp with an ext2 formatted HDD, swap enabled, 32 Mb of Ram. When i run 'top' it says to only consume 35 % of memory and about 5% of CPU.

Still the system is not stable: it runs between one and 4-5 hours and then it dies. No real indication in the logs why...

Anyone with the same problems or is it running stable for you??

Regards

DrChair
31-05-2007, 22:17
Yeah, ok. But i do not have an /home dir and i cant create it.


Then, there are two options:
One: fix the python package
Two: make an yenc package

(or maybe three: I send you the compiled yenc (if i can find it on my router), you manually copy it in place and hope that it works)

DrChair
31-05-2007, 22:28
Still the system is not stable: it runs between one and 4-5 hours and then it dies. No real indication in the logs why...

Anyone with the same problems or is it running stable for you??


Well it is a heavy program, especially on memory usage. I'm using a 512 MB swap-file, but if I add more than 2 dvd-nzbs to the queue, after a while (especially during repair), the router runs out of memory, and kills the SABnzbd.py processes.

So most of the time, I add one nzb, wait till it's done, and add the next.

I'm planning on writing a script (cronjob), that watches the tmp-folder, and if the tmp-folder is empty (meaning the nzb is finished), it can move the next nzb into the nzb-import-dir.
But I'm short in time and shell-scripting isn't a daily routine for me. So if there are volunteers...

rowi
01-06-2007, 21:39
Well it is a heavy program, especially on memory usage. I'm using a 512 MB swap-file, but if I add more than 2 dvd-nzbs to the queue, after a while (especially during repair), the router runs out of memory, and kills the SABnzbd.py processes.

Ok same here, 512 swap and big ext2 partiotion for data. I know that we push these boxes to the limits with these applications but hey...thats the fun :D

I was also thinking of a cronjob to restart the proces but ofcourse the best would be not to have the system kill the proces. I might try to put some time in your idea of adding the nzb's one by one (maybe combined with restarting the system if needed).

By the way, instead of manually restarting SAB after reboot i just added the line
/opt/local/bin/SABnzbd.py -f /opt/etc/SABnzbd.ini -d at the bottom of my post-boot file, after every reboot SAB is automaticaly started.

DrChair
03-06-2007, 20:57
By the way, instead of manually restarting SAB after reboot i just added the line
/opt/local/bin/SABnzbd.py -f /opt/etc/SABnzbd.ini -d at the bottom of my post-boot file, after every reboot SAB is automaticaly started.

If I do that, it can't find par2 and unrar, during startup. Does it do find those in your setup??

rowi
13-06-2007, 19:57
If I do that, it can't find par2 and unrar, during startup. Does it do find those in your setup??

Hi DrChair,

Sorry for the late reply. Actually my system stays quite unstable. And you where right, par2 and unrar where not found in de SABnzbd.log with my addition in the post-boot. So alternatively i tried something different. With a cron system already installed (thanx to macsat for his tutorial) i added a new shell script in my cron.5mins. Now every 5 minutes it checks if a python process is running. If not i start SABnzbd with the startup file. If the process is running i do nothing.

After reboot cron is loaded and within 5 minutes SABnzbd is running automaticaly. Together with the queu saving this also solves most of the stability issues because after breakdown i get an automatic restart.

And moreover...with this option par2 and unrar are loaded correctly.

Underneath you find the shell file i use:
-save it as sabnzbd.sh
-place file somewhere in the cron folders (i used 5mins)
-and make it executable with a chmod 755.


#!/bin/sh
SERVICE='python'

if ps | grep -v grep | grep $SERVICE > /dev/null
then
echo "$SERVICE service running, everything is fine"
else
echo "$SERVICE was not running, restarting"
startSABnzbd
fi

Hope this works for you too.

raas
13-08-2007, 07:53
Hi,

I took a chance with your tutorial about Sabnzbd (great program).

I'm running into a problem: when I want to install buildroot, it gives an error, package not found.. please update.
I do an update, get al new definitions, but still, the package is not found..

second approach i took was to download the latest buildroot package from the 'feeds' page and install it..

How long does it have to take before the package is installed? it was installing for at least 30 mins. yesterday-evening before i got to bed. (haven't looked at it in the morning if it was ready)

raas
13-08-2007, 20:59
ok,

found out what was wrong when the ipkg did not found the package.
had to point ipkg.conf to the oleg-optware location.

did an update, and install buildroot.

it's running for more than an hour now.
- no response from another putty-session.
- wl500gp still responds to ping command from pc
- samba obviously not running anymore, shares are unreachable.
- webgui (oleg) also not available

Any Ideas what I might be doing wrong ?

Thanks in advance,

Raas

raas
14-08-2007, 21:48
ok,

So I made a little progess.
thinking that the buildroot package was installed (left it on for a long while, but incidentally shutted down my computer, so i lost my telnet session before I got to take a look at it.) I proceed with the tutorial.
all went fine. no problems.

only thing is, sab would not run.
I could only think off the buildroot package going wrong.
so I downloaded the latest package from the site http://nslu2-linux.dyoung-mirror.net/feeds/optware/oleg/cross/stable/buildroot_4.1.1-12_mipsel.ipk
copied it to the router, and installed it manually.. after 20 minutes or so. all installed succesfully (terminated).
rebooted the router. started my startup script but got the following error:


Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Traceback (most recent call last):
File "/opt/local/bin/SABnzbd.py", line 34, in ?
import logging
ImportError: No module named logging

What's going on?
I installed everything, not in the given order, but does that matter?

raas
15-08-2007, 22:20
DrChair and Rowi,

could you please show a list off all the packages you have installed?

I can't get it to work.

Here's mine:

[admin@NAS root]$ ipkg list_installed
buildroot - 4.1.1-12 - uClibc compilation toolchain
bzip2 - 1.0.4-1 - Very high-quality data compression program
cron - 4.1-6 - Standard vixie cron, with cron.d addition
ctorrent - 1.3.4-2 - CTorrent is a BitTorrent client written in the C programming language.
cups - 1.2.12-2 - Common Unix Printing System
e2fsprogs - 1.40.2-1 - Ext2 Filesystem Utilities
expat - 2.0.1-1 - XML Parser library
fontconfig - 2.3.2-5 - Font configuration library
freetype - 2.1.10-4 - Free truetype library
gconv-modules - 2.2.5-7 - Provides gconv modules missing from the firmware. These are used by glibc's iconv() implementation.
glib - 2.6.6-6 - The GLib library of C routines.
gnuplot - 4.2.0-5 - Command-line driven interactive data and function plotting utility
gnutls - 1.6.2-2 - GNU Transport Layer Security Library.
ipkg - 0.99.149-2 - lightweight package management system
libbt - 1.05-5 - a C library implementing the core BitTorrent protocol
libcurl - 7.16.4-1 - Curl is a command line tool for transferring files with URL syntax, supporting FTP, FTPS, HTTP, HTTPS, GOPHER, TELNET, DICT, FI
libdb - 4.2.52-3 - Berkeley DB Libraries
libevent - 1.3c-2 - libevent to implement an event loop
libgcrypt - 1.2.4-1 - GNU cryptography libray, needed by gnutls
libgd - 2.0.35-1 - An ANSI C library for the dynamic creation of images
libgpg-error - 1.4-1 - Error handling library for libgcrypt
libjpeg - 6b-2 - collection of jpeg tools
libpng - 1.2.18-1 - Portable Network Graphics Libraries
libtasn1 - 0.3.10-1 - ASN.1 structure parser library.
libtiff - 3.8.2-1 - Tag Image File Format Libraries
libuclibc++ - 0.2.2-4 - C++ standard library designed for use in embedded systems
logrotate - 3.7.1-4 - Rotates, compresses, removes and mails system log files.
mc - 4.6.1-3 - Midnight Commander File Manager
nano - 2.0.6-1 - A pico like editor
ncurses - 5.6-1 - NCurses libraries
nload - 0.6.0-2 - Nload is a console application which monitors network traffic and bandwidth usage in real time
openssl - 0.9.7m-2 - Openssl provides the ssl implementation in libraries libcrypto and libssl, and is needed by many other applications and librari
par2cmdline - 0.4-1 - tool to apply the data-recovery capability concepts of RAID-like systems to the posting & recovery of multi-part archives on Us
popt - 1.7-2 - A C library for parsing command line parameters.
py-celementtree - 1.0.5-20051216-4 - A toolkit that contains a number of light-weight components for working with XML (C implementation).
py-cheetah - 2.0rc7-2 - Cheetah - The Python-Powered Template Engine.
py-cherrypy - 2.2.1-2 - A pythonic, object-oriented web development framework.
py-elementtree - 1.2.6-20050316-3 - A toolkit that contains a number of light-weight components for working with XML.
python - 2.4.4-3 - This is a package that sets up the default python.
python24 - 2.4.4-5 - Python is an interpreted, interactive, object-oriented programming language.
readline - 5.2-2 - The GNU Readline library provides a set of functions for use by applications that allow users to edit command lines as they are
samba2 - 2.2.12-1 - Lightweight Samba suite provides file and print services to SMB/CIFS clients.
screen - 4.0.3-2 - A screen manager that supports multiple logins on single terminal
slang - 2.1.1-1 - S-Lang is a multi-platform library designed to allow a developer to create robust multi-platform software.
termcap - 1.3.1-2 - Terminal emulation library
torrent - 1.4-7 - a collection of scripts that processes torrent files
uclibc-opt - 0.9.28-12 - micro C library for embedded Linux systems
unrar - 3.6.8-2 - unrar is an application that can decompress files and archives created using the RAR compression scheme
unzip - 5.52-1 - A (de)compression library for the ZIP format
xinetd - 2.3.14-6 - Highly configurable, modular and secure inetd
zlib - 1.2.3-2 - zlib is a library implementing the 'deflate' compression system.

thanks

raas
20-08-2007, 09:59
ok..

started all over again.

got sabnzbd working.
however. It crashed when I inserted the 2nd nzb to the watch folder.
After reading I discovered this was common behaviour.

however
after a reboot the contents off my /tmp/harddisk/ is cleared. Files which i have stored there are deleted on reboot.
Will have to look into that.

but..

I got it working. So it must be me who had done something wrong the first time.

raas
20-08-2007, 22:00
Hi..

just wanted to let you know that all problems are sorted out now..

downloading from payserver (eweka) with 2 connections gives me max 564kb/s (4.5mbit/s+) and is constantly between 500-550kb/s on the longer period when downloading a dvd.
This is more than I had hoped for, reading other posts so I'm more than happy.

I have a 12mbit connection (bbned) so it's not maxing-out, but that doesn't mather.. very nice power-efficient download machine. :D

mistraller
29-08-2007, 19:35
How much time does it take to repair and unrar a complete DVD? On my P4 3Ghz it takes for about 20 minutes if I have to repair a lot. I tried nzbget but my router died after some minutes downloading.
(I have wl500gx 32MB/200Mhz)

raas
30-08-2007, 06:53
Hmm..

a bit hard to answer that quetion right now.


Each time I downloaded a single nzb (eg. 1 dvd) I never measured the times for unrar. (I hardly ever have to par 'cause all downloads are complete on newsserver).

Now...
my SABnzbd does not freeze-up anymore if I 'feed' it more than one nzb, so it downloads multiple collections.
However, since it downloads more than one collection it does not unrar it anymore.. (have to look into that. )

I now unrar the files 'over network' what takes a bit less than 20 minutes per DVD (~4.5gb) to mediacenter pc (p4 2.66)

Hope this helps.

Jeroen van Omme
01-09-2007, 08:00
Hi guys!

Have been trying to get my WL-HDD to run SABnzbd. Have installed all packages but running SABnzbd results in the following error in SABnzbd.error.log:


Traceback (most recent call last):
File "/opt/local/bin/SABnzbd.py", line 313, in ?
main()
File "/opt/local/bin/SABnzbd.py", line 208, in main
init_ok = sabnzbd.initialize(pause)
File "/opt/local/lib/python2.4/site-packages/sabnzbd/decorators.py", line 26, in newFunction
return f(*args, **kw)
File "/opt/local/lib/python2.4/site-packages/sabnzbd/__init__.py", line 325, in initialize
DOWNLOADER = Downloader(servers)
File "/opt/local/lib/python2.4/site-packages/sabnzbd/downloader.py", line 119, in __init__
host = servers[server]['host']
TypeError: string indices must be integers


Does anyone have suggestions?

PS. Thanks for the great write-up!

raas
02-09-2007, 12:11
Hi,

I personally think this has got something to do with the pyton package..
Try to remove the pyton package and reinstall the pytong package.

Good luck.

Raas

Jeroen van Omme
03-09-2007, 06:48
Raas,

Thanks for the tip.
I reinstalled all the packages and it seems to run.
PS shows me a few running processes of SABnzbd.

Now I'll have to figure out how to get it to download.
I have SABnzbd running on a WL-HDD connected to my WL500g router, so there's probably be some port forwarding to be done...

I seem to remember that there is also a webbased config for it?

Cheers for now!

Jeroen

raas
03-09-2007, 07:17
yep. webbased application.
look for portnumber in configfile in
/opt/etc/sabnzbd.ini

(:8080 or :8008)


Also,. configure the newsservers...


Good luck

Jeroen van Omme
09-09-2007, 13:38
Hi Raas,

looks like you're the person to talk to when it comes to installing SABnzbd, so I hope you can help me with this.

Reinstalled all packages and editted de ini file for SAB. When I run it, ps shows me three sab-processes. When I look into the logfile this is what I see:

2007-09-04 21:11:21,736::INFO::--------------------------------
2007-09-04 21:11:21,791::INFO::SABnzbd v0.2.5
2007-09-04 21:11:21,810::INFO::Initializing SABnzbd v0.2.5
2007-09-04 21:11:21,833:: DEBUG::FAIL_ON_CRC -> False
2007-09-04 21:11:21,853:: DEBUG::CREATE_GROUP_FOLDERS -> False
2007-09-04 21:11:21,874:: DEBUG:: DO_FILE_JOIN -> True
2007-09-04 21:11:21,894:: DEBUG:: DO_UNZIP -> True
2007-09-04 21:11:21,914:: DEBUG:: DO_UNRAR -> True
2007-09-04 21:11:21,934:: DEBUG:: DO_SAVE -> False
2007-09-04 21:11:21,955:: DEBUG::PAR_CLEANUP -> True
2007-09-04 21:11:21,974:: DEBUG::CLEANUP_LIST -> []
2007-09-04 21:11:21,995:: DEBUG::UMASK -> 493
2007-09-04 21:11:22,016:: DEBUG::SEND_GROUP -> False
2007-09-04 21:11:22,036:: DEBUG::CREATE_CAT_FOLDERS -> False
2007-09-04 21:11:22,055:: DEBUG::CREATE_CAT_SUB -> False
2007-09-04 21:11:22,141::INFO:: DOWNLOAD_DIR: /tmp/harddisk/SABnzbd/tmp
2007-09-04 21:11:22,169::INFO::COMPLETE_DIR: /tmp/harddisk/downloads
2007-09-04 21:11:22,213::INFO::NZB_BACKUP_DIR: /tmp/harddisk/SABnzbd/nzb/backup
2007-09-04 21:11:22,245::INFO::CACHE_DIR: /tmp/harddisk/SABnzbd/cache
2007-09-04 21:11:22,267::INFO::dirscan_dir: /tmp/harddisk/SABnzbd/nzb
2007-09-04 21:11:22,288::INFO::BANDWITH_LIMIT: 0.0
2007-09-04 21:11:22,312::INFO::schedlines: []
2007-09-04 21:11:22,333::INFO::dirscan_opts: 3
2007-09-04 21:11:22,354::INFO::top_only: True
2007-09-04 21:11:22,374::INFO::auto_sort: False
2007-09-04 21:11:22,394::INFO::[sabnzbd] Loading data for bytes.sab from /tmp/harddisk/SABnzbd/cache/bytes.sab
2007-09-04 21:11:22,479::INFO::[sabnzbd] /tmp/harddisk/SABnzbd/cache/bytes.sab missing
2007-09-04 21:11:22,501::INFO::[sabnzbd] Loading data for queue.sab from /tmp/harddisk/SABnzbd/cache/queue.sab
2007-09-04 21:11:22,522::INFO::[sabnzbd] /tmp/harddisk/SABnzbd/cache/queue.sab missing
2007-09-04 21:11:22,610::INFO::_yenc module... found!
2007-09-04 21:11:22,629::INFO::celementtree module... found!
2007-09-04 21:11:22,648::INFO:: par2 binary... found!
2007-09-04 21:11:22,666::INFO::rar binary... found!
2007-09-04 21:11:22,684::INFO::unzip binary... found!
2007-09-04 21:11:23,216::INFO::Starting SABnzbd v0.2.5
2007-09-04 21:11:23,322:: DEBUG::[sabnzbd] Starting postprocessor

The error-log contains nothing and the webpage can't be reached.

Any ideas?
cheers
Jeroen

mistraller
09-09-2007, 14:31
Somebody knows what this means? Looks like garbage to me:


Traceback (most recent call last):
File "/opt/local/bin/SABnzbd.py", line 313, in ?
main()
File "/opt/local/bin/SABnzbd.py", line 138, in main
if int(cfg['__version__']) < sabnzbd.__configversion__:
File "/opt/local/lib/python2.4/site-packages/sabnzbd/utils/configobj.py", line 375, in __getitem__
val = dict.__getitem__(self, key)
KeyError: '__version__'

After this message, nothing is running as far as I can see. I followed the toturial exactly how it's described. Only changed ip adres in ini file.

raas
09-09-2007, 17:28
@Jeroen,


hmm... strange.. my log file ends the same.. (starting postprocessor), and I have 3 running SAB processes too....

When you start SABnzbd with the file you have created to start up SAB, does it respond with a normal prompt? I mean, no error-messages there..

btw... all the credits for installing SAB goes to DrChair, he's the one that figured it all out.... /me, being a newsgroup junkie, wanted to get this to work..


@Mistraller..
Did you configure any newsservers in the sabnzbd.ini file?
Also... the python package might be inproperly installed..

mistraller
10-09-2007, 21:51
@Mistraller..
Did you configure any newsservers in the sabnzbd.ini file?
Also... the python package might be inproperly installed..
Fool me...My ini file was corrupt. Website is running now. Let's see if it's actually working.

mistraller
10-09-2007, 22:33
The looks are great! But how about this?

415 mistraller D 20M 363 8.6 68.1 SABnzbd.py
443 mistraller R 240 273 1.9 0.7 top
272 mistraller S 232 95 1.9 0.7 dropbear
4 mistraller SW 0 1 1.9 0.0 kswapd
3 mistraller SWN 0 1 1.3 0.0 ksoftirqd_CPU0
414 mistraller S 20M 363 0.0 68.1 SABnzbd.py
380 mistraller S 20M 363 0.0 68.1 SABnzbd.py
413 mistraller S 20M 363 0.0 68.1 SABnzbd.py
373 mistraller S 20M 363 0.0 68.1 SABnzbd.py
372 mistraller S 20M 363 0.0 68.1 SABnzbd.py
374 mistraller S 20M 363 0.0 68.1 SABnzbd.py
376 mistraller S 20M 363 0.0 68.1 SABnzbd.py
378 mistraller S 20M 363 0.0 68.1 SABnzbd.py
362 mistraller S 20M 1 0.0 68.1 SABnzbd.py
375 mistraller S 20M 363 0.0 68.1 SABnzbd.py
371 mistraller S 20M 363 0.0 68.1 SABnzbd.py
379 mistraller S 20M 363 0.0 68.1 SABnzbd.py
377 mistraller S 20M 363 0.0 68.1 SABnzbd.py
416 mistraller S 20M 363 0.0 68.1 SABnzbd.py
370 mistraller S 20M 363 0.0 68.1 SABnzbd.py
363 mistraller S 20M 362 0.0 68.1 SABnzbd.py
369 mistraller S 20M 363 0.0 68.1 SABnzbd.py
412 mistraller S 20M 363 0.0 68.1 SABnzbd.py

My box reacts very slow :D

I have 2 boxes, one is for testing purposes, and the other is monitoring my solar panels. I think I should stop SABnzbd before my solar script starts creating RRDtool images....is it possible to pause and/or kill sabnzbd from commandline? I want to run everything from one box, and keep my testing box only for testing.

I saw the scheduling options, but in winter, my solarpanels start some hours later, so it only has to pause after my solar system wakes up.

mistraller
12-09-2007, 10:51
Ran into the same issue; after some hours, the process gets killed. Don't know why. I will experiment with number of threads, but it looks like the same problem as I had with nzbget; newsleeching is just to much for this machine.
(I run on a wl500g deluxe)

raas
12-09-2007, 11:35
Until now I 'never' had SABnzbd crashing on my system.
Even not when I put 7 nzb's with dvds in the queue.. it downloaded all the files.. however.. did not unrar them .. but as I'm learning now in another post this seems a very commom problem in the past few months.

however.. when I reboot my asus and the post-boot starts SAB, SAB is going wild, I have as many processes as you have.
I've found out this has got something to do with SAB's queue.. (starting a download after reboot even though sab has already downloaded the nzb).

I now solve this by deleting the queue.sab file (find out the location in your ini file).. When the asus reboots then.. sab behaves nicely instead off taking up almost have the memory and most off the cpu...

Tonight I'm going to take a look at hellanzb (www.hellanzb.com) it seems to run on the asus also.. but I have no further info at the moment..

mistraller
13-09-2007, 21:23
I was looking into my dir's to see what my test asus did last night...

First of all, I set SABnzbd to use only one thread.

strange output, but it looks better than ever before!

My download dir contains:
All the rar files, although some incomplete (not exactly 50MB)
Only 2 par2 files

AND.....: A video_ts folder WITH some unrarred VOB files!

I think my router stopped unrarring because of the incomplete rar files, which couldn't be repaired because the other par2 files were missing. (They were in the nzb, I checked that.)

I finally have some monitoring too:


First of all, traffic (it passes my first asus which has traffic monitoring)

http://i46.tinypic.com/ngyjvp.jpg

My connection is 4Mbit max, normally resulting in ~400KB download from newsserver. So, 240 KB is not bad at all with only 1 thread.

And second:
CPU load on router2, which is actually doing all the work.
http://i45.tinypic.com/2jb9zjk.jpg
Wow: Heavy load! Other router doesn't come much higher as 0.6, only when creating RRD images.
It also shows how long it will take to get a complete download. (Some hours longer as my pc, but...my router is always on, and it cannot consume more power than 10W...)


But, I can conclude, there's nothing wrong with Unrarring! (In my case...) because I started over several times, on my test router, I'm sure I have a clean install with only the newest packages, and firmware.

Next time I will look for the sort order in the nzb file. Maybe I have to download pars first.

raas
13-09-2007, 21:42
Very nice to hear it's also working for you!...

Even better yours does unrar too !!

I use 2 threads connected to eweka.. resulting in 500-550kb/s steady download..

I think I will want to try only one thread.. maybe it will unrar then.. (not very likely 'cause unrar doesn't work in telnet also.. )
But when I first started using sab I only used 1 thread.. (hearing stories that more than 1 thread kills sab.)

One question though.

Does your sab support more than 1 nzb? i've heard stories about it crashing then, but mine works properly..

it's not as fast as a pc, but kb/s to watt ratio is way better !! haha.

mistraller
13-09-2007, 21:59
Very nice to hear it's also working for you!...

Even better yours does unrar too !!

I use 2 threads connected to eweka.. resulting in 500-550kb/s steady download..

I think I will want to try only one thread.. maybe it will unrar then.. (not very likely 'cause unrar doesn't work in telnet also.. )
But when I first started using sab I only used 1 thread.. (hearing stories that more than 1 thread kills sab.)

One question though.

Does your sab support more than 1 nzb? i've heard stories about it crashing then, but mine works properly..

it's not as fast as a pc, but kb/s to watt ratio is way better !! haha.

I don't now if it will work with more than one nzb. I could imagine an overload situation when it starts downloading the next collection, while it is still quickparring your completed download. The quickpar gets killled, and no unrar will start....but, if unrarring in commandline doesn't work either, it's another story.
I like the way you think...kb/s to watt ratio :D
As you know I use my primary router also as a logging device for my solarpanels. After I find out how I can kill sabnzbd if it's not ready yet when the sun comes up, I want to try running sab on the primary router. (Otherwise I have 2 routers running 24x7)
My primary router performs lots of other things, so the SABnzbd load is simply to heavy for the other programs. (especially RRDtool and Palantir need some cpu power, and my monitoring script want some...)

raas
14-09-2007, 07:39
I like the way you think...kb/s to watt ratio

I've already taken a look at your solar panels. very impressive.
(that's why I said it..)

.
You might want to try the bandwith throttle in sab perhaps that will lower the load.

.
about running multiple processes.
i'm downloading multiple nzb's with sab and multiple torrents with Transmission simultaniously. (totalling 600kb/s+) No problems there.
Also webserver with php/mysql/myphpadmin is running (now i have the thttpd troubles sorted out).(response from CMS < 1s)
Sure you can notice some slowing down on the machine, but overall it's running very good.

Going to take a look tonight if it did unrar with only 1 connection.

raas
14-09-2007, 22:47
hmmm..

even with one thread enabled it didn't unrar..

with one thread I reached about 430kb/s on the asus.. which is not bad.
But it's not maxing-out on the connection.
Eweka supports 500kb/s per connection (max 8.)
When I use 2 connections I get higher download speed (500-550kb/s), though one would think, 2 connections is creating some overhead which uses the cpu....

Anyway.. the story continues.. I have to find a way to unpar..
(just to inconvenient.. the media-center wants to play files on the asus.. however. they have to be unpacked using the pc... if it would unrar correctly the media-center is directly able to play the files stored on the asus)
...

Jester
15-09-2007, 19:59
@raas,

Hi,
I installed everthing according to DrChair's tutorial, and I have a similar problem when setting up Yenc:

$ python setup.py install
Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Traceback (most recent call last):
File "setup.py", line 29, in ?
from distutils.core import setup, Extension
ImportError: No module named distutils.core

What did you do to solve this?

raas
16-09-2007, 15:02
Hi,

Are you sure the install off the buildroot is ok? It takes a long time to install and it happened to me before that the install wasn't right.. started all over, then it worked for me..

ipkg install buildroot


..
if the install doesn't work properly, you might want to try to download the package manually, put it on the router's share, and install it manually in the way this threads ask you to install the par2commandline manually..


good luck

Jester
17-09-2007, 20:51
Thanks, it seems to work now.
It had to do with ipkg installation.
Ipkg was downloading from the wrong location, not Olegs optware.

raas
17-09-2007, 20:59
Glad to hear it works now.

Jeroen van Omme
18-09-2007, 21:00
Raas, Jester,

Nice to hear someone has sabnzbd running, perhaps you can help me along?
I posted a logfile in this thread some time ago. Later I did some changes and now the log file is longer:

2007-09-15 15:15:39,958::INFO::--------------------------------
2007-09-15 15:15:40,000::INFO::SABnzbd v0.2.5
2007-09-15 15:15:40,021::INFO::Initializing SABnzbd v0.2.5
2007-09-15 15:15:40,043::DEBUG::FAIL_ON_CRC -> False
2007-09-15 15:15:40,063::DEBUG::CREATE_GROUP_FOLDERS -> False
2007-09-15 15:15:40,084::DEBUG::DO_FILE_JOIN -> True
2007-09-15 15:15:40,104::DEBUG::DO_UNZIP -> True
2007-09-15 15:15:40,124::DEBUG::DO_UNRAR -> True
2007-09-15 15:15:40,145::DEBUG::DO_SAVE -> False
2007-09-15 15:15:40,165::DEBUG::PAR_CLEANUP -> True
2007-09-15 15:15:40,185::DEBUG::CLEANUP_LIST -> []
2007-09-15 15:15:40,206::DEBUG::UMASK -> 493
2007-09-15 15:15:40,226::DEBUG::SEND_GROUP -> False
2007-09-15 15:15:40,247::DEBUG::CREATE_CAT_FOLDERS -> False
2007-09-15 15:15:40,266::DEBUG::CREATE_CAT_SUB -> False
2007-09-15 15:15:40,355::INFO::DOWNLOAD_DIR: /tmp/harddisk/SABnzbd/tmp
2007-09-15 15:15:40,383::INFO::COMPLETE_DIR: /tmp/harddisk/downloads
2007-09-15 15:15:40,427::INFO::NZB_BACKUP_DIR: /tmp/harddisk/SABnzbd/nzb/backup
2007-09-15 15:15:40,458::INFO::CACHE_DIR: /tmp/harddisk/SABnzbd/cache
2007-09-15 15:15:40,481::INFO::dirscan_dir: /tmp/harddisk/SABnzbd/nzb
2007-09-15 15:15:40,502::INFO::BANDWITH_LIMIT: 0.0
2007-09-15 15:15:40,526::INFO::schedlines: []
2007-09-15 15:15:40,547::INFO::dirscan_opts: 0
2007-09-15 15:15:40,568::INFO::top_only: True
2007-09-15 15:15:40,588::INFO::auto_sort: False
2007-09-15 15:15:40,608::INFO::[sabnzbd] Loading data for bytes.sab from /tmp/harddisk/SABnzbd/cache/bytes.sab
2007-09-15 15:15:40,693::INFO::[sabnzbd] /tmp/harddisk/SABnzbd/cache/bytes.sab missing
2007-09-15 15:15:40,715::INFO::[sabnzbd] Loading data for queue.sab from /tmp/harddisk/SABnzbd/cache/queue.sab
2007-09-15 15:15:40,736::INFO::[sabnzbd] /tmp/harddisk/SABnzbd/cache/queue.sab missing
2007-09-15 15:15:40,817::INFO::_yenc module... found!
2007-09-15 15:15:40,835::INFO::celementtree module... found!
2007-09-15 15:15:40,861::INFO::par2 binary... found!
2007-09-15 15:15:40,880::INFO::rar binary... found!
2007-09-15 15:15:40,898::INFO::unzip binary... found!
2007-09-15 15:15:41,159::INFO::Starting SABnzbd v0.2.5
2007-09-15 15:15:41,294::DEBUG::[sabnzbd] Starting postprocessor
2007-09-15 15:15:41,676::DEBUG::[sabnzbd] Starting assembler
2007-09-15 15:15:41,705::DEBUG::[sabnzbd] Starting downloader
2007-09-15 15:15:41,735::DEBUG::[sabnzbd] Starting dirscanner

Those last 3 line weren't there the first time, so it looks like i'm making Progress. However, the webinterface still doesn't work. Any suggestions?
Also, placing a nzb in the directory doesn't seems to have any effect....

raas
19-09-2007, 10:40
Bit hard to say.

you're using the right port to connect to?
What kind off messaged does the browser respond with? 404?

the directories to download to/scan/work/etc are they correct? i saw you're pointing them to /tmp/harddisk/
Are you putting the nzb's in a /tmp/harddisk/*/* folder..
I have everything pointing to /opt/*/*/ because that's where my harddisk is mounted...

(I think the /tmp/harddisk/ thing is wrong.. something tells me/reminds me that that is not the place the harddisk is actually mounted to.)

If it still doesn't work, you also could use Hellanzb. works also
see the bottom of this thread. there's a small write-up on how to install it.
http://www.wl500g.info/showthread.php?t=10577

Good Luck

mistraller
19-09-2007, 23:45
I was looking further to my par problem, Sabnzbd should download more par if repair fails, but It doesn't do that. So in case of failed repair, it just starts unrar.
So, why isn't it downloading more par? Im not sure of this. Anyone saw this working? (more par download after failing repair?)

Jester
20-09-2007, 19:11
I can access SABnzbd via 192.168.1.1:8080/sabnzbd from a local PC.
When I try to access it remotely it does not work (error 404)
I used {my ISP IPaddress}:8080/sabnzbd.

I have enabled Web Access from WAN in the firewall setting of the WL500gP, port 8080. Do I have to enable other settings?
In the sabnzbd.ini file there a 'localhost'setting do I have to put my IP address here? What else?

I also do not have remote access to my WL500gP via telnet (putty). I get a time out. What could be the cause of that?

I do have remote access to the router setting page.

raas
20-09-2007, 21:55
could be different problems.

1) port 8080 incoming outside is not forwarded to port 8080 inside.
2) the remote location you use in order to visit the website might be blocking port 8080 (eg. corporate firewall)

mistraller
20-09-2007, 23:05
I can access SABnzbd via 192.168.1.1:8080/sabnzbd from a local PC.
When I try to access it remotely it does not work (error 404)
I used {my ISP IPaddress}:8080/sabnzbd.

I have enabled Web Access from WAN in the firewall setting of the WL500gP, port 8080. Do I have to enable other settings?
In the sabnzbd.ini file there a 'localhost'setting do I have to put my IP address here? What else?

I also do not have remote access to my WL500gP via telnet (putty). I get a time out. What could be the cause of that?

I do have remote access to the router setting page.

Look to other topics how to setup a webserver. Think this has nothing to do with Sabnzbd.

raas
21-09-2007, 07:36
No. if Sab is already running you do not have to setup a website for sab specifically, it's already there, it has it's own 'internet-server' built in..

you have to do some port forwarding.

something like.

ouside.ip.address:8080 to ip.address.of.asus:8080

I have it setup like this and my Sab is addressable from the outside.

like each application on the asus has it's on port, you can make it available through port forwarding.

mistraller
23-09-2007, 21:03
No. if Sab is already running you do not have to setup a website for sab specifically, it's already there, it has it's own 'internet-server' built in..

you have to do some port forwarding.

something like.

ouside.ip.address:8080 to ip.address.of.asus:8080

I have it setup like this and my Sab is addressable from the outside.

like each application on the asus has it's on port, you can make it available through port forwarding.
.....and so you are writing part of the "How to setup a webserver at home". Whatever webserver you run, you always need to setup NAT. As I said, it has nothing to do with SAB specifically.

mundu
28-09-2007, 19:28
Awesome!!

Works nice but with a little to much load!!

Add some problems because of the init. Forgot the version part and the newzbin stuff. (Not mentioned on the tutorial.:p)

thanks for the tutorial.:cool:

DrChair
03-10-2007, 16:19
I finally found out the reason why sabnzbd doesn't find unzip, unrar and par2cmdline when starting with a startup-script.

Sabnzbd searches for those in the path. But the path is set in profile, thus only present after a user logs on to the router.

Solution is to define the path in de startup-script, before sabnzbd is started.
I updated the first post with this new info.

Furthermore, i've made ipkg-packages for yenc and sabnzbd. If i manage to get them added into the feed, the installation tutorial will be only a few lines.

mundu
03-10-2007, 18:31
I can access SABnzbd via 192.168.1.1:8080/sabnzbd from a local PC.
When I try to access it remotely it does not work (error 404)
I used {my ISP IPaddress}:8080/sabnzbd.

I have enabled Web Access from WAN in the firewall setting of the WL500gP, port 8080. Do I have to enable other settings?
In the sabnzbd.ini file there a 'localhost'setting do I have to put my IP address here? What else?

I also do not have remote access to my WL500gP via telnet (putty). I get a time out. What could be the cause of that?

I do have remote access to the router setting page.

Had the same problem!
I had in my .ini file host = "192.168.1.1" and changed it to host= "" so it was defined to localhost! I donīt if this was problem and if you have it like this, but it seems to have solved my problem.

mistraller
03-10-2007, 22:54
Anyone saw the problem that more par files were not downloaded when needed?

ind.jones
26-10-2007, 18:51
Hi
reading throught various forum, i can't find from where you've got the packages !
Do i have to compile the source ?

If someone 've got the binary for my asus wl 500gp please
send me it at indy.jones.1@m6mail.fr
Thanks

raas
28-10-2007, 13:24
Hi,

SABnzbd can be downloaded from http://www.sourceforge.net/projects/sabnzbd

HellaNZB can be downloaded from http://www.hellanzb.com


No need to compile as it runs on python.. (Make sure you've got python installed.)

Also.. read this thread for installing SABnzbd.

and this one: http://wl500g.info/showthread.php?t=10577&highlight=hellanzb for installing HellaNZB (it's a bit confusing, 'cause it starts with ninan.. but read down to the bottom.. instructions are there)


HTH

DrChair
01-11-2007, 12:42
I managed to get the par2cmdline from the oleg feed working and also solved the bug in python.

Therefore the startpost is updated a bit.

Also I've made a few patches for SABnzbd. The most important one, adds an option that pauses the download queue during yenc assembly and during postprocessing. This seems to increase stability quite a bit.
I'm still waiting for commit access so I can add the package for sabnzbd.

In the mean time, i'll see if i can make a package of hellanzb aswell.

mistraller
01-11-2007, 21:57
Can you tell me how to upgrade from the former version? I upgraded par2cmdline, but how about patching sadnzbd?

raas
02-11-2007, 14:34
DrChair,

well, that's very good news, I will look into that when I get home.


--oth--
SAB & Hella are driving me nuts.
each one has its pros and cons.
SAB is doing everything, but it is very unstable
Hella is stable but will only unpack like 10% of the downloaded packages.

Right now, I'm using Hella to download, because it actually downloads and keeps running. I've disabled unraring in Hella and wrote my own script to handle the unpacking.
The script is now in an testing stage and the only thing I have to solve is how to unpack files which have a space in the filename:
(I parse the filename to a variable, check the variable for its contents, it has the content it is supposed to have, but then when issueing the unrar command the filename gets cut-off after the first space, So 'Friends Bloopers.rar' is ending up in
unrar e -o-- 'Friends.rar' instead of
unrar e -o-- 'Friends Bloopers.rar'

Nevertheless, when polishing off the script is done so it can handle this too I will publish it here.

In the mean time, I will look into the updates you've made to the tutorial. Thanks for the good work !

DrChair
08-11-2007, 20:36
SAB & Hella are driving me nuts.
each one has its pros and cons.
SAB is doing everything, but it is very unstable
Hella is stable but will only unpack like 10% of the downloaded packages.


I agree about the pros and cons. I'm also running hellanzb now for a few days and it seems much less hungry for memory. I've downloaded about 10 packages and they all unpacked just perfect.

Have you tried reinstalling your router?
And if you download a failed package again, does it also fail again?
Have you tried turning on debugging in hellanzb?

But to get back on the topic, packages for both SABnzbd and Hellanzb have been added to the feed.
Check startpost for instructions on SABnzbd.
instructions for Hellanzb will follow in a separate topic.

Chillout
21-05-2008, 19:52
Sabnzbd is running smoothly on my WL500GP, although I'm kinda suffering from a memory leak in the package, or at least that's what I can tell from looking at my performance graphs. As far as I know, there's a new version of sabnzbd, called sabnzbd+ (version 0.3). Has anyone already made a package of this version?

artur72
22-05-2008, 22:47
One more vote for Sabnzbd+ :) , nice look. Visit http://www.sabnzbd.org/ for details. Can anyone already made a package for this.
Thx.

raas
23-05-2008, 07:40
Hi,

if I recall right, when when the previous version(s) first started running on the Asus there wheren't any packages to install sabnzbd.

It used to run python scripts to work, so we installed pythong on the router and sabnzbd on top of that. works.

I don't know about sabnzbd+ but if it still runs on python, you should be able to get it running on the asus.

HTH

Chillout
25-05-2008, 21:45
yes it still runs on python, but I don't know anything about it.
I compared the files in the directories of both the new tar and the "old" install on my Asus, some things match up, others don't, so I am not sure if I want to try... can't any one make an updated version of the package? that would be easier for everyone...

raas
26-05-2008, 19:08
Hi,

I suppose no-one has even tried sabnzbdplus otherwise there would be a bigger chance that a package already existed.
only software which runs stable, more or less, is turned into a package

you could be the 'first' and test is

I think, that you only have to install the python package that has already been compiled, and which is also in this tutorial, and configure

then, download sabnzbdplus and extract it to let's say /opt/local/bin/sabnzbdplus
or some other location you find convenient for your apps.

configure the config file which came with the package and put it in /opt/etc/sabnzbdplus.ini

Create a startup file /opt/bin/startsabnzbdplus



touch /opt/bin/startsabnzbdplus
nano /opt/bin/startsabnzbdplus


with the contents



#!/bin/sh
/opt/local/bin/sabnzbdplus.py -f /opt/etc/sabnzbdplus.ini -d


make it executable



chmod +x/opt/bin/startsabnzbdplus



then, if all is good, try to start sabnzbdplus with


/opt/local/bin/sabnzbdplus


I haven't tried this myself with sabnzbdplus, but this procedures is taken from the original way to install sabnzbd on the asus before a package existed and it had to be installed on python.

HTH

DrChair
27-05-2008, 21:19
I suppose no-one has even tried sabnzbdplus otherwise there would be a bigger chance that a package already existed.
only software which runs stable, more or less, is turned into a package


I did look at it a couple of weeks ago.
Problem with the new version, is that it doesn't have a setup.py to install it anymore.
The new authors of Sabnzb assume you just unpack the tar-ball somewhere on your router and execute it.

So it is a very different way of installing, and I haven't had the time yet to figure out how to put this new way into a package.

Raas his explaination should work, with a couple of modifications:

- the 'executable' is SABnzbd.py instead of sabnzbdplus.py
- you need to edit SABnzbd.py and change the first line into:
#!/opt/bin/python -OO
- the new version doesn't come with a sample ini-file. You can either use the one from the older version, or a new one gets generated, the first time you start it.

(Still in my opinion SABnzbd (and Hellanzb) use way to much memory...
I'm running nzbget, and if I don't use nzbgetweb, my router doesn't use any swap)