Bekijk de volledige versie : nzbget - binary newsgrabber
Hi guys,
I proudly present you the new version of nzbget - 0.3.0.
NZBGet is a binary newsgrabber, which downloads files from usenet based on information given in nzb-files. NZBGet can be used in standalone and in server/client modes.
New version has many improvements including fast yEnc-decoder, par-check and repair.
It runs very well on WL500gP (in contrast to previous version). Moreover this device was the primary development-target for new version.
You can read more info on nzbget on its web site (http://nzbget.sourceforge.net/) and sourceforge project page (http://sourceforge.net/projects/nzbget).
The new version is now available in a package repository. You need to use oleg-branch. A swap-file is also highly recommended.
To install:
ipkg update
ipkg install nzbget
It will install a samle configuration file under /opt/share/doc/nzbget/nzbget.conf.example. Please rename it to .nzbget and put into your home directory. Alternatively the config file can be stored under following locations:
/etc/nzbget.conf
/usr/etc/nzbget.conf
/usr/local/etc/nzbget.conf
/opt/etc/nzbget.conf
Choose the one you like the most.
You need to set at least the option "MAINDIR" and one newsserver in configuration file. Please note, that the MAINDIR must exist, it will not be created automatically.
For usage description please refer to web site (http://nzbget.sourceforge.net/#Usage) and to help screen "nzbget -h".
Basically you start a daemon first:
nzbget -D
To add a new download:
nzbget -A /path/to/you/file.nzb
To list current download queue:
nzbget -L
To see the server output (remote mode):
nzbget -C
or (to directly use curses-output, if it was not selected in configuration file)
nzbget -o outputmode=curses -C
Please let me know, if you have any problems installing or running nzbget.
Best Regards,
hugbug.
methanoid
10-12-2007, 12:43
Hmmm interesting and thanks, I didnt spot that this was out.
I might try this on my Linkstation (I don't have the WL500g but this is a good forum and similar to NAS-Central in getting ideas for things to try)
Is there any way to remotely administer or monitor it (web interface) ??
Have you compared the CPU and memory usage with HellaNZB?
Does it have any dependencies (Hella needs Python etc)?
Do you use the faster yEnc module like Hella does?
Is this compiled for ARM or MIPSEL (Linkstation is ARM)
I'd certainly try it....
methanoid
Is there any way to remotely administer or monitor it (web interface) ??
The server can be administered remotely. The built-in console-client allows to monitor of log-messages, current state of queue, current download speed. It can also edit the download queue (pause/unpause, move, delete items). There are three output modes, the most advanced is ncurses (use "nzbget -o outputmode=curses -C" to start a client in this mode), on Windows it looks similar to Far or Midnight Commander (console interface).
The server and client is the same executable, where the running mode depends on passed command-line parameters. To add a new file to download queue a separate call needed (nzbget -A filname.nzb) or you just can put a nzb-file into nzbget's monitoring folder.
NZBGet (server and client) can be compiled and run on many platforms, including Linux and Windows.
You can download windows-version from NZBGet-site. Then you can use windows-version to monitor downloads on your Linux-server.
Note: after version 0.3.0 was published, I found, that I did not pay an attention to the fact, that endianness affect data transmitted over network. The clients and server of version 0.3.0 are compatible only if they use the same endianess. Since x86 and mipsell are both little endian, it works. ARM-processors can work in both modes (little and big endian), depending on OS used.
This compatibility issue was addressed and already fixed in source code (svn-repository on sourceforge.net), but a new release (probably 0.3.1) is not published yet.
Have you compared the CPU and memory usage with HellaNZB?
NZBGet uses very little CPU power. On my WL500gP with 2 MBit/s connection the top-command says something like 60-80% cpu idle time. Mem-usage is not so low as it could be, because the program maintains the complete list of articles in download queue in memory. The typical usage of swap is not higher as 10 MB.
I did not used Hellanzb heavily and therefore can not make a good comparison. I just can say that in my quick tests the cpu idle time reported by top was about 5%.
Does it have any dependencies (Hella needs Python etc)?
NZBGet is written in C++. It needs following packages (for oleg-branch of optware): ncurses, libxml2, libpar2, libsigc++, libstdc++, zlib.
Do you use the faster yEnc module like Hella does?
NZBGet has a very fast internal decoder for yEnc.
Is this compiled for ARM or MIPSEL (Linkstation is ARM)
The optware-repository from nslu2-linux.org has nzbget compiled for many targets. Which one do you use, what your /opt/etc/ipkg.conf says?
I've also used nzbget to download a few nzb's and it does it's job nicely.
However I do miss the automatic unrarring.
And because hellanzb still maxes out my connection (450 kbyte/s), I'll stick to that for a while.
A few suggestions which would be nice in a future version:
- automatic unpack
- par2 verify 'on the fly' so that each .rar gets checked right after it has been downloaded (like newsleacher), instead of checking all the rars when the whole nzb is downloaded. This shoud speed up the proces.
automatic unpack
I did not develop a good unrar-strategy yet. If nzb-collection contains more than one archive, I need to find this out to unpack all of them, before I delete rars.
However nzbget can execute a user-defined post-process script after download of nzb-collection is completed. The unraring could be done in this script.
par2 verify 'on the fly' so that each .rar gets checked right after it has been downloaded (like newsleacher), instead of checking all the rars when the whole nzb is downloaded. This shoud speed up the proces.
Libpar (and par2cmdline) has no ability to save/reload par-check-state (like quickpar does). If we want to hold the state, we must keep parrepair-object alive (this is what nzbget does during waiting for extra par-blocks). If the download queue was rearranged (for example if new download was added to the top of queue), we need either to create another parrepair-object for a new nzb-collection (and allow the first object to keep used memory) or stop the first object and lost par-check-state.
This makes the "on the fly"-thing not easy, but if we make it adjustable (something like an option for "max keeped objects") it might work.
I will keep both features on a wish-list for next releases.
I did not develop a good unrar-strategy yet. If nzb-collection contains more than one archive, I need to find this out to unpack all of them, before I delete rars.
Hellanzb has the same problem with unpacking. SABnzb however does support nzbs with multiple archives. I guess it either sees all the rar's with the same name before the dot as one sub-collection, or it parses all the empty .par2 files to see which files belong the each sub-collection.
(or a combination of both, if there are multiple archives with only one par2 set)
If you need some testing on a newer release, let me know.
Libpar (and par2cmdline) has no ability to save/reload par-check-state (like quickpar does). If we want to hold the state, we must keep parrepair-object alive (this is what nzbget does during waiting for extra par-blocks).
Oke, I didn't know that. I thought it would verify -> download extra par2s -> start a repair (beginning again with verify) -> if some of the downloaded par2s are damaged, download some more -> again start a repair (beginning again with verify). In this scenario it would verify the whole nzb 3 times, but because you use libpar2 and keep the object alive, it would only verify the rars once, after that it verifies the downloaded par2s, and starts repairing.
So if you were to verify the files one at a time and log the result, it would be quicker if all files are oke, but slower if repairing is needed, because than all files need to be verified again, before repairing can commence.
Currently nzbget has only one parrepair-object. It monitors download queue and when the last file from some nzb-collection is downloaded (except additional par-files, which are paused by default), it starts par-check for this nzb-collection (actually for each par-set of this collection, but typically one nzb-collection contains only one par-set). If after verifying par-checker finds out, that it needs extra pars, it unpauses them and waits (keeping libpar's parrepair-object alive). After all neccessary par-files are downloaded, par-checker verifies them and starts repair. It does not need to verify them again. This is the main difference to hellanzb, ninan and sabnzbd (they use par2cmdline).
On my WL500gP verifying (just verifying!) of one dvd lasts about one hour. So nzbget should complete download with par-check/repair one hour early than other programs :)
So if you were to verify the files one at a time and log the result, it would be quicker if all files are oke, but slower if repairing is needed, because than all files need to be verified again, before repairing can commence.
With libpar we cannot verify just one file. Parrepair-object takes par-file as parameter and verifies all files belonging to a par-set it coud find in a directory. Then it can verify extra files we pass to it. That's why we cannot use approach like "create parrepair-object, verify one downloaded file, destroy parrepair-object" or use one parrepair-object for files from different par-sets. We must keep one parrepair-object for each par-set.
In this case we do not need to start verify again, if our parrepair-object for this nzb-collection (actually for the par-set) was keeped alive. So verify/repair should not be longer in any case.
The problem occurs only, if files from different nzb-collections (or par-sets from one collection) in download queue are mixed (remember, nzbget allows to control (I mean move) individual files in download queue, not just nzb-collections). If we want to verify a file, which does not belong to existed parrepair-object, we need to create a new parrepair-object or we can cancel verify-process in existed parrepair-object (and lose all verify-info for this par-set).
But if we suppose, that the download queue is not edited often,we could achieve good results with a limited number of parrepair-objects (we need to be able to configure the max count).
This also means a lot of coding/testing-work and I'm not sure if it (another one hour advantage for dvdr) worths.
Jeroen van Omme
12-12-2007, 09:36
Hugbug,
Can you explain the client/server option of nzbget?
From what I understand, I could have a nzbget-server running on my "download server", and add nzb-files using a nzbget-client running on my pc. Does this mean I can switch of my pc after adding a file, or does nzbget still need the client to communicate?
Cheers for making a very stable downloadtool!
Jeroen
Can you explain the client/server option of nzbget?
From what I understand, I could have a nzbget-server running on my "download server", and add nzb-files using a nzbget-client running on my pc. Does this mean I can switch of my pc after adding a file, or does nzbget still need the client to communicate?
You can shutdown client (PC) anytime.
Few tips:
1) Create a shortcut with command "nzbget.exe -o outputmode=curses -C" on desktop. If you want to check the state of server start the client via shortcut. Then use Q-key to stop client, this key work only in curses-outputmode, for other outputmodes you can use Ctrl+C or just close the window.
2) Create a shortcut with command "nzbget.exe -A". To add a file to download queue drag'n'drop the file on the shortcut. You can also put the file into monitoring folder via samba-share or ftp.
3) To add a file to the top of download queue create and use a shortcut with command "nzbget.exe -A -T".
4) You can also create new items in explorer's context menu for nzb-files. In this case you need to add "%1" at the end of commands, for example: nzbget.exe -A "%1".
Jeroen van Omme
21-12-2007, 07:48
Guys,
I've run into a weird problem. I run nzbget on my WL-HDD. Placing nzb-files and retrieving downloaded file is done via FTP.
Nzbget has run succesfully for a number of weeks.
However when I want to retrieve a particular fileset I get an 550 error (Failed to change directory) error when I want to view/copy/delete it. All other downloads can be copied without a problem, just this one has issues.....
Any suggestions?
Jeroen
PS. Happy holidays!
However when I want to retrieve a particular fileset I get an 550 error (Failed to change directory) error when I want to view/copy/delete it.
If you can access the directory using telnet, then it is eventually a problem in filename - it may contain characters, ftp-server does not like.
If you can't access the directory, it should be a problem with filesystem. Try to repair it.
Jeroen van Omme
21-12-2007, 08:28
If you can access the directory using telnet, then it is eventually a problem in filename - it may contain characters, ftp-server does not like.
If you can't access the directory, it should be a problem with filesystem. Try to repair it.
Thanks for the quick reply, hugbug.
- Even with telnet I can't change to the directory, so it's not limited to ftp.
- How would I go about fixing my filesystem?
It depends on filesystem. For me it is simple - I'm using usb-hdd with fat32-filesystem, so I just attach it to pc and check under windows.
For ext2/ext3 there is a tool - e2fsck (http://linux.die.net/man/8/e2fsck). The page suggests to not check mounted filesystems. Is it possible to unmount a disk on WL-HDD? Try to ask for help in WL-HDD subforum.
But may be you just have no permissions to access the directory (don't know why)? Try to change them with chmod. Even if you logged in as admin (root) you need access rights to enter a directory. As admin (root) you can give the rights to yourself.
Jeroen van Omme
24-12-2007, 09:57
Hugbug,
I tried e2fsck, but it didn't solve my problem.
I also tried rebooting the WL-HDD (thinking maybe some process was locking the dir), but this also didn't work. It's just plain weird: All previous files can be viewed/removed etc, and files downloaded at a later time also do not give any problem.....oh well, for now I'll just leave this dir alone, it's not hogging that much space on the HD......
Jeroen
I recently switched over from hellanzb to nzbget, because I realized it doesn't take longer to unpack the files from my router to my desktop than to just copy it.
And yesterday I ran into the following situation:
I was downloading a collection which I got from exporting to nzb in Newsleecher. This collection contains a few broken files and the complete reposted versions of those broken files.
The result: a broken file.part17.rar and a complete file.part17.rar_duplicate1
In my opinion, in case of broken files, the duplicates of those files should also be verified, to see if they are better (just like Quickpar does). Or is this also a limitation in libpar2 ??
If that's the case, perhaps it might be possible to determine the best version of the duplicates before starting to verify, by looking at the filesize? Cause in my situation, all the broken files where smaller than they should be, and the duplicates had the right size.
One last thing that is bothering me: If I add a lot of nzb's at once, the memory usage gets quite big (about 8 MB per dvd5). But worse, it doesn't seem to free the used memory. I downloaded a collection containing 6 dvd5's -> memory usage is 54 MB. However the last dvd finished repairing 4 days ago, and still the memory usage is 54 MB.
Tomorrow i emptied the queue (all paused par2 files) but still it uses 54 MB.
The only thing that works is to stop and start the daemon.
Could you look into why nzbget doesn't free up memory, when it isn't needed anymore?
And would it be possible to add an option to specify the number of collections to keep in memory. (cause now when i want to download a lot of nzbs, I keep all the nzb's in another directory, and move them into the nzbget directory 2 at a time).
Finally, it might be useful the have a command to empty the whole queue (nzbget -E D -I * isn't working)
Hi DrChair,
In my opinion, in case of broken files, the duplicates of those files should also be verified, to see if they are better (just like Quickpar does). Or is this also a limitation in libpar2 ??
Normally libpar2 (like par2cmdline) scans only files, listed in par2-file. But it is possible to load additional files (via special function) after the scan is completed. I'll see what I can do in your case. I will need to parse filenames to not load all files in a directory (this costs time), but only duplicate-files. Could you please send me your nzb-file for tests.
One last thing that is bothering me: If I add a lot of nzb's at once, the memory usage gets quite big (about 8 MB per dvd5).
It is strange, it should free the memory. Anyway I have already addressed this issue in a current (development) version. Now nzbget keeps articles' info on disk and loads it just before the download of a file starts. In my test I have added few big-nzbs (10-15GB data in each, about 60 GB all) and the memory consumption was as little as 2-3 MB for the whole program (Virt.mem usage reported by "top") and the usage of swap-file was 4 MB.
Finally, it might be useful the have a command to empty the whole queue (nzbget -E D -I * isn't working)
In current (development) version I have implemented new commands for group-editing (delete/move/pause/unpause all files belonging to the same nzb) with one simple command: e.g. "nzbget -E G D 1" deletes all files from the nzb-group, which the file with ID=1 belongs. You can do this also via curses-frontend (use G-key to switch group-view on/off).
If you have compile environment for optware, you can get the mentioned version of nzbget from svn using command:
svn co --revision 52 http://nzbget.svn.sourceforge.net/svnroot/nzbget/trunk nzbget-0.3.1-testing
Then run "configure" and "make dist", copy created file "nzbget-0.3.1-testing.tar.gz" to "optware/downloads", edit version number in "optware/make/nzbget.mk" to "NZBGET_VERSION=0.3.1-testing" and compile the package.
Or I can send you the compiled ipk-package instead :)
Let me know if you want to test this version.
ChangLog for revision 28 (http://nzbget.svn.sourceforge.net/viewvc/*checkout*/nzbget/trunk/ChangeLog?revision=28&pathrev=52), not all new features listed, see svn change log (http://nzbget.svn.sourceforge.net/viewvc/nzbget/trunk/?view=log) for changes after revision 28.
Note: the communication protocol was changed in version 0.3.1, you will not be able to use windows-version 0.3.0 with server 0.3.1. I could send you the updated compiled windows-version too.
I'm going to release the version 0.3.1 in a few next weeks.
nevermind the memory issues...
I just found the svn repository and compiled 0.3.1+r52
The result: still using only 10 MB with 9 nzbs in queue :D
Just a cosmetic tip: add a \n at line 721 (and perhaps line 722) in Options.cpp
And I just found a nzb that gets parsed wrong. When in queue, it displays only a part of the subject, instead of the filenames. Also it can't see the difference between rar's and par's, so it doesn't pause the par's.
I'll pm you that nzb.
norberto
02-01-2008, 22:28
Hi all,
I installed and configured nzbget on my Synology DS-106 and I'm having the following error message when trying to launch it (in server or client mode) :
DiskStation> nzbget -s
FATAL ERROR: Invalid option "DaemonUserName"
I tried to change the DaemonUserName in the configuration file
DaemonUserName=root
with "admin" or any user name from my Syno but it doesn't change anything.
I'm sure I'm missing something simple...
Can you help ?
Thanks.
DiskStation> nzbget -s
FATAL ERROR: Invalid option "DaemonUserName"
Looks like you are trying to use a configuration file from a different (newer) program version, as installed.
You probably installed the program from a package-repository (version 0.3.0), but took the configuration file from svn/trunk (current development version).
You should have an appropriate configuration file example installed in "/opt/share/doc/nzbget/nzbget.conf.exampe" (it comes with optware-package). It can be also downloaded from svn/tags/0.3.0 (http://nzbget.svn.sourceforge.net/viewvc/*checkout*/nzbget/tags/0.3.0/nzbget.conf.example?revision=2).
Or you can just comment out (with "#"-character) all (new) options the program complains about.
norberto
03-01-2008, 00:35
You're right : I followed the instructions found on the Synology forum (http://www.synology.com/enu/forum/viewtopic.php?t=5864&highlight=nzbget) and downloaded a wrong configuration file.
It seems it's working fine now.
Thanks a lot for your prompt answer !
norberto, great.
DrChair
And I just found a nzb that gets parsed wrong. When in queue, it displays only a part of the subject, instead of the filenames. Also it can't see the difference between rar's and par's, so it doesn't pause the par's.
The parsing of artcile's subject was improved for better extracting of filename part from it. This fixes the error with parsing of the mentioned file.
I was downloading a collection which I got from exporting to nzb in Newsleecher. This collection contains a few broken files and the complete reposted versions of those broken files.
The result: a broken file.part17.rar and a complete file.part17.rar_duplicate1
There is now a dupe check for files from the same nzb-request to detect reposted files. Now the program chooses the best (biggest) file from them. This works only if there were exactly two files with the same name. (if there were more, the program may decide, it has inccorectly parsed filenames from subjects and may try to use full subjects as filenames). You need to activate the option "dupecheck" in config-file.
Please take the new version from svn.
Thanx... I was already using newest svn version and I saw you fixed it, but I didn't know the option DupeCheck needed to be enabled.
I thought (as explained in nzbget.conf.example) that that option would only check for dupes on disk, but now you made it also check for dupes in the nzb.
Perhaps you need to update the nzbget.conf.example with this info?
Perhaps you need to update the nzbget.conf.example with this info?
Thanks for the tip, I forgot about it. I'll update the description.
Found a few bugs:
Like i already pmed you, the new decoding in svn r57 only works when DirectWrite is disabled.
Second bug: when I cancel a download (I choose Group -> Edit -> select the one to cancel -> Delete), it logs the following:
INFO Collection XXXXXXXX completely downloaded
INFO Queueing XXXXXXXX for par-check
INFO Checking pars for XXXXXXXXXXX
Also it doesn't delete the already downloaded articles from tmp, atleast with DirectWrite=no, I don't know what happens with DirectWrite=yes, cause thats broken atm
This is only the case for downloads that have been started (i.e., already downloaded atleast one article)
DrChair, many thanks for testing and bug reports!
the new decoding in svn r57 only works when DirectWrite is disabled.
Fixed.
Second bug: when I cancel a download (I choose Group -> Edit -> select the one to cancel -> Delete), it logs the following:
INFO Collection XXXXXXXX completely downloaded
INFO Queueing XXXXXXXX for par-check
INFO Checking pars for XXXXXXXXXXX
I would say it is not a bug but "behaviour by design", but the design seems to be not so good now :)
PostProcessor reacts to event "last unpaused file from collection was downloaded OR deleted = there no more unpaused files for this collection"). Then it starts par-checker and/or executes postprocess script.
I can change it to react to only completed files before starting par-check.
But should the postprocess-script also not be executed if the last file was deleted?
when I cancel a download (I choose Group -> Edit -> select the one to cancel -> Delete), it logs the following:
INFO Collection XXXXXXXX completely downloaded
INFO Queueing XXXXXXXX for par-check
INFO Checking pars for XXXXXXXXXXX
Fixed in r59. Now par-checking is not started by deleting of files or collections.
Also it doesn't delete the already downloaded articles from tmp
I'll look into it. But this is not a big problem, because on next start server will delete orphaned files from temp dir.
allochtheun
08-01-2008, 13:42
I've installed this great package. The install is going good. but when I start nzbget -D and put an NZB file into the right dir.
The logfile is telling me that it cannot create the queue en nzbget.log file.
I find that rather strange because I've fully access to the dir (/tmp/harddisk/download). and I can create en modify files/dir's through ftp
I double checked if the dir's exist.
Other programs like lighttpd en vsftpd also writing to /tmp/harddisk and there i've no issues with
At the moment I'm not capable of posting some examples ,like (df , mount or logfile's) I'will post them tonight if nessecary.
Hope someone could help me with this one.
Greetings
Allochtheun
IThe logfile is telling me that it cannot create the queue en nzbget.log file.
Please check if all necessary directories exist, expecially the main directory (option "$MAINDIR" in config file). The program does not create the main dir automatically (this will be fixed in next version).
allochtheun
08-01-2008, 15:05
Please check if all necessary directories exist, expecially the main directory (option "$MAINDIR" in config file). The program does not create the main dir automatically (this will be fixed in next version).
I've checked that. They exists I created "$MAINDIR":/tmp/harddisk/downloads
I've manualy added the dir's such as queue etc.
I found it rather strange that he tels me that hij cannot write to /tmp/harddisk/downloads/queue/queue. Why is there twice the queue name.?
I'wil post some screendumps tonight so you can see.
Thnx anyway for your quick reply
Allochtheun
I found it rather strange that he tels me that hij cannot write to /tmp/harddisk/downloads/queue/queue. Why is there twice the queue name.?
It creates a file named "queue" in dir "queue" :) So it's OK to have this name twice in the error message. Strange is why it cannot write to the directory.
You better trying to start the program in console mode (not daemon) to see error messages - use "nzbget -s".
To check all things again:
1) create completely new directory, for example "/tmp/harddisk/downloads/test";
2) copy nzbget.conf.example to "/tmp/harddisk/downloads/test/nzbget.conf";
3) edit "/tmp/harddisk/downloads/test/nzbget.conf" and change ONLY "$MAINDIR" to "/tmp/harddisk/downloads/test";
4) start the program:
nzbget -c /tmp/harddisk/downloads/test/nzbget.conf -s
What it says now?
allochtheun
08-01-2008, 16:12
What it says now?
[sebastiaan@Allochrouter root]$ nzbget -c /tmp/harddisk/downloads/test/nzbget.conf -s
/usr/local/root/tmp/harddisk/downloads/test/dst/nzbget.log: No such file or directory
[INFO] nzbget server-mode
5 threads running, 0 KB/s, 0.00 MB remaining
/usr/local/root/tmp/harddisk/downloads/test/dst/nzbget.log: No such file or directory
I find i very strange the rights on the harddisk are right i Think, Because I can write.
------------------------------------------------------------
[sebastiaan@Allochrouter root]$ mount
/dev/root on / type squashfs (ro)
none on /dev type devfs (rw)
proc on /proc type proc (rw)
ramfs on /tmp type ramfs (rw)
usbfs on /proc/bus/usb type usbfs (rw)
/dev/scsi/host0/bus0/target0/lun0/part2 on /opt type ext3 (rw)
/dev/discs/disc0/part5 on /tmp/harddisk type ext3 (rw,noatime)
Is also changed the path to /opt/share/test and put the nzbget.conf there, also with no result.
-------------------------------------------------------------
[sebastiaan@Allochrouter root]$ nzbget -c /opt/share/test/nzbget.conf -s
/usr/local/root/opt/share/test/dst/nzbget.log: No such file or directory
[INFO] nzbget server-mode
5 threads running, 0 KB/s, 0.00 MB remaining
/usr/local/root/opt/share/test/dst/nzbget.log: No such file or directory
I find it strange, maybe do you have another tip
Greetings,
Allochtheun
Please post the output of "ipkg list_installed". May be there is a problem in versions of system packages (like libuclibc++). I will compare your versions with my.
Try (for test) to create a new dir in /tmp (on ram-disk) and point the program to this dir.
allochtheun
08-01-2008, 16:52
Please post the output of "ipkg list_installed". May be there is a problem in versions of system packages (like libuclibc++). I will compare your versions with my.
Try (for test) to create a new dir in /tmp (on ram-disk) and point the program to this dir.
Here are the results of the test on Ram disk, they are the same nl
---------------------------------------------------------------
[sebastiaan@Allochrouter root]$ nzbget -c /tmp/downloads/test/nzbget.conf -s
/usr/local/root/tmp/downloads/test/dst/nzbget.log: No such file or directory
[INFO] nzbget server-mode
5 threads running, 0 KB/s, 0.00 MB remaining
And the results of ipkg;
--------------------------------
bzip2 - 1.0.4-2 - Very high-quality data compression program
coreutils - 6.9-6 - Bunch of heavyweight *nix core utilities
cron - 4.1-7 - Standard vixie cron, with cron.d addition
cups - 1.3.4-1 - Common Unix Printing System
cyrus-sasl-libs - 2.1.22-2 - Provides client or server side authentication (see RFC 2222).
e2fsprogs - 1.40.2-4 -
ffmpeg - 0.4.9-pre1+r6153-1 - FFmpeg is an audio/video conversion tool.
freetype - 2.1.10-4 - Free truetype library
gdbm - 1.8.3-2 - GNU dbm is a set of database routines that use extensible hashing. It works similar to the standard UNIX dbm routines.
glib - 2.9.6-1 - The GLib library of C routines.
gnutls - 1.6.3-1 - GNU Transport Layer Security Library.
ipkg-opt - 0.99.163-9 - The Itsy Package Manager
libart - 2.3.17-2 - 2-d graphics library.
libdb - 4.2.52-3 - Berkeley DB Libraries
libdlna - 0.2.3-1 - Reference DLNA (Digital Living Network Alliance) open-source implementation for Linux.
libgcrypt - 1.2.4-1 - GNU cryptography libray, needed by gnutls
libgpg-error - 1.4-1 - Error handling library for libgcrypt
libjpeg - 6b-2 - collection of jpeg tools
libpar2 - 0.2-3 - A library for performing common tasks related to PAR recovery sets
libpng - 1.2.23-1 - Portable Network Graphics Libraries
libsigc++ - 2.0.18-1 - libsigc++ implements a typesafe callback system for standard C++.
libtasn1 - 0.3.10-1 - ASN.1 structure parser library.
libtiff - 3.8.2-1 - Tag Image File Format Libraries
libuclibc++ - 0.2.2-4 - C++ standard library designed for use in embedded systems
libupnp - 1.6.0-1 - The Universal Plug and Play (UPnP) SDK for Linux provides support for building UPnP-compliant control points, devices, and brid
libxml2 - 2.6.30-2 - Libxml2 is the XML C parser and toolkit developed for the Gnome project.
libxslt - 1.1.22-1 - An XML Stylesheet processor based on libxml2
lighttpd - 1.4.18-1 - A fast webserver with minimal memory footprint.
mc - 4.6.1-3 - Midnight Commander File Manager
nano - 2.0.7-1 - A pico like editor
ncurses - 5.6-3 - NCurses libraries
ncursesw - 5.6-1 - NCurses libraries with wide char support.
net-tools - 1.60-5 - Network Config and Debug tools (route, arp, netstat etc.)
nzbget - 0.3.0-2 - A command-line client/server based binary newsgrabber for nzb-files.
openldap-libs - 2.3.38-1 - Open Lightweight Directory Access Protocol
openssl - 0.9.7m-3 - Openssl provides the ssl implementation in libraries libcrypto and libssl, and is needed by many other applications and librari
par2cmdline - 0.4-4 - tool to apply the data-recovery capability concepts of RAID-like systems to the posting & recovery of multi-part archives on Us
pcre - 7.4-1 - Perl-compatible regular expression library
php - 5.2.5-1 - The php scripting language
php-fcgi - 5.2.5-1 - The php scripting language, built as an fcgi module
popt - 1.7-2 - A C library for parsing command line parameters.
readline - 5.2-2 - The GNU Readline library provides a set of functions for use by applications that allow users to edit command lines as they are
rrdtool - 1.2.23-3 - Round-Robin Database tool. Database collator and plotter
samba - 3.0.28-1 - Samba suite provides file and print services to SMB/CIFS clients.
slang - 2.1.2-1 - S-Lang is a multi-platform library designed to allow a developer to create robust multi-platform software.
sqlite - 3.5.4-1 - SQLite is a small C library that implements a self-contained, embeddable, zero-configuration SQL database engine.
uclibc-opt - 0.9.28-12 - micro C library for embedded Linux systems
unrar - 3.6.8-2 - unrar is an application that can decompress files and archives created using the RAR compression scheme
vsftpd - 2.0.5-1 - ftp daemon with an emphasis on speed and security
xinetd - 2.3.14-7 - Highly configurable, modular and secure inetd
zlib - 1.2.3-2 - zlib is a library implementing the 'deflate' compression system.
Thnx in advanced, hope you can find something
Allochtheun
I guess you forget to remove "~"-character in path.
You should have in your config:
$MAINDIR=/tmp/harddisk/downloads/test
not
$MAINDIR=~/tmp/harddisk/downloads/test
allochtheun
08-01-2008, 17:32
I guess you forget to remove "~"-character in path.
You should have in your config:
not
You're right now is starting.;
[sebastiaan@Allochrouter root]$ nzbget -c /tmp/harddisk/downloads/test/nzbget.conf -s
[INFO] nzbget server-mode
5 threads running, 0 KB/s, 0.00 MB remaining
I'wil configure it further and let you know.
Thnx for your great support.
Allochtheun
allochtheun
09-01-2008, 07:47
@ HugBug
Fur sure he can find the dir's and can write to them.
But when I look at the log I'm getting the following error
[ERROR] Could not create file $MAINDIR}/tmp/6.041.tmp
Connection to the newsserver is okay.
Also I see that there are file's in the DST/(nameznb) folder but they are all 0kb.
Do I forget something:confused:
Allochtheun
[ERROR] Could not create file $MAINDIR}/tmp/6.041.tmp
Is this the exact message? If so, it means, the variable "$MAINDIR" was not substituited, probably because of the absence of the open brace "{".
Check option "TempDir" and/or show me your config file (as attachment and strip your username/password).
allochtheun
09-01-2008, 08:52
Is this the exact message? If so, it means, the variable "$MAINDIR" was not substituited, probably because of the absence of the open brace "{".
Check option "TempDir" and/or show me your config file (as attachment and strip your username/password).
Indead you're right It was missing the brace "{" i've added it and downloading went right on. Thnx again for your quick and good response.
And keep up the good work.:D
Allochtheun
found another couple of bugs: (running svn r62)
1. When you move a group up to first position (up from 2nd, or top from lower than 2nd), the order of the nzb gets reversed.
(moving it down, or moving it up from for example 3rd to 2nd doesnt seem to change the order)
2. DupeCheck is partialy broken. It does recognize duplicate files in the nzb, it also recognizes duplicates in queue, but it doesn't recognize duplicate files on disk.
And another question: Is it oke to 'exit' nzbget daemon with kill -9 `pidof nzbget` ??
Or should i always use nzbget -Q ??
1. When you move a group up to first position (up from 2nd, or top from lower than 2nd), the order of the nzb gets reversed. (moving it down, or moving it up from for example 3rd to 2nd doesnt seem to change the order)
Confirmed, need to fix it.
2. DupeCheck is partialy broken. It does recognize duplicate files in the nzb, it also recognizes duplicates in queue, but it doesn't recognize duplicate files on disk.
It works for me. May be you expect too much? It checks only in DestDir/NzbName folder; the duplicate file must be added from the nzb-file with the same name. It does not check in all subfolders of DestDir.
Is it oke to 'exit' nzbget daemon with kill -9 `pidof nzbget` ??
You may use kill `pidof nzbget`, which sends SIGTERM; nzbget caught this signal and terminates gracefully. kill -9 terminates the process immediately and you should avoid it.
May be you expect too much? It checks only in DestDir/NzbName folder; the duplicate file must be added from the nzb-file with the same name. It does not check in all subfolders of DestDir.
I accidently deleted a group that was currently downloading (thought D was for Down instead of Delete). So I renamed the nzb from .queue back to .nzb to re-add it. The result: It starts downloading at the beginning again, not removing the already downloaded files.
Because that nzb was complex (and didn't have the filename in quotes), I tried it with a more simple nzb.
I took a download from last week, of which i hadn't deleted the files yet, and renamed the nzb-file from .queue back to .nzb
The result: it queued all files, starts downloading and thus creates duplicates on disk.
DrChair
Probably the second nzb was also not quit so simpe for subject parser to extract filenames. Did download queue show exactly same names as files on disk?
Could you please send me this file for testing?
DrChair
The second error was also confirmed.
Both errors are fixed in r63.
chief12345
10-01-2008, 20:26
Hello,
I'm trying to run nzbget 0.3 on a linksys NSLU2 with unslung 6.10.
Starting the server with nzbget -s works, but I can't connect to the server via client. When I try nzbget -C, I get the message:
Unable to send request to nzbserver at 192.168.0.15 (port 6789)
or I get a segemtation fault.
Am I missing something in installation or cofiguration ?
Please give me a hint, what Icould do.
These are the packages I have installed:
bash - 3.2.17-1 - A bourne style shell
bzip2 - 1.0.4-2 - Very high-quality data compression program
cpio - 2.9-3 - file archive utility
e2fsprogs - 1.40.3-1 - Ext2 Filesystem Utilities
findutils - 4.2.31-3 - File finding utilities
glib - 2.9.6-1 - The GLib library of C routines.
groff - 1.19.2-2 - front-end for the groff document formatting system
ipkg - 0.99.154-r2 -
ipkg-web - 7-7 - A web frontend for ipkg
kernel - 2.4.22.l2.3r63-r10 -
kernel-image-2.4.22-xfs - 2.4.22.l2.3r63-r10 -
less - 416-1 - Less file browser
libc6-unslung - 2.2.5-r5 -
libdb - 4.2.52-3 - Berkeley DB Libraries
libgcc - 3.4.4-r3 -
libipkg - 0.99.154-r2 -
libpar2 - 0.2-3 - A library for performing common tasks related to PAR recovery
libsigc++ - 2.0.18-1 - libsigc++ implements a typesafe callback system for stand
libstdc++ - 5.0.7-6 - Standard C++ library, needed for dynamically linked C++ pr
libupnp - 1.6.0-1 - The Universal Plug and Play (UPnP) SDK for Linux provides su
libxml2 - 2.6.30-2 - Libxml2 is the XML C parser and toolkit developed for the G
man - 1.6f-1 - unix manual page reader
man-pages - 2.33-3 - unix manual pages
mc - 4.6.1-3 - Midnight Commander File Manager
ncurses - 5.6-3 - NCurses libraries
ncursesw - 5.6-1 - NCurses libraries with wide char support.
nfs-utils - 1.1.1-1 - Kernel NFS Server
nslu2-linksys-libs - 2.3r63-r2 -
nzbget - 0.3.0-2 - A command-line client/server based binary newsgrabber for nzb
openssh - 4.7p1-1 - a FREE version of the SSH protocol suite of network connecti
openssl - 0.9.7m-3 - Openssl provides the ssl implementation in libraries libcry
par2cmdline - 0.4-4 - tool to apply the data-recovery capability concepts of RAI
portmap - 4-5 - Portmap daemon for NFS
py-hellanzb - 0.13-1 - Hellanzb is a Python application designed for *nix enviro
py-twisted - 2.5.0-1 - A networking engine written in Python.
py-yenc - 0.3-1 - A module that provides only raw yEnc encoding/decoding with bu
py-zope-interface - 3.3.0.1-1 - A separate distribution of the zope.interface pa
python24 - 2.4.4-6 - Python is an interpreted, interactive, object-oriented prog
readline - 5.2-2 - The GNU Readline library provides a set of functions for use
screen - 4.0.3-2 - A screen manager that supports multiple logins on single term
slang - 2.1.1-1 -
slingbox - 1.00-r8 -
termcap - 1.3.1-2 - Terminal emulation library
unrar - 3.6.8-2 - unrar is an application that can decompress files and archives
unslung-rootfs - 2.3r63-r11 -
update-modules - 1.0-r4 -
util-linux - 2.12r-4 - A suite of essential utilities for any Linux system.
zlib - 1.2.3-2 - zlib is a library implementing the 'deflate' compression system
chief12345
where do you run the client, on the same NSLU2-box or on pc? If on pc, have you tried the client on NSLU2?
chief12345
10-01-2008, 21:47
Neither on my NSLU2 nor on my PC via Windows-Client.
I can't make a connection.
On NSLU2:
In the first putty-session I have the server running:
# nzbget -s
[INFO] nzbget server-mode
3 threads running, 0 KB/s, 0.00 MB remaining Limit 48 KB/S
In a second putty-session (or a second screen-window) I try to connect:
# nzbget -C
Segmentation fault
chief12345
We need to build the program in debug-mode to get debug messages. Do you have build environment to compile the program? If not, I'll try to do this. Which feed are you using, is it http://ipkg.nslu2-linux.org/feeds/optware/nslu2/cross/stable/ ?
chief12345
10-01-2008, 22:37
No, I don't have a compile environment.
cross-feed.conf in /etc/ipkg says:
http://ipkg.nslu2-linux.org/feeds/optware/nslu2/cross/stable
I'd realy appreciate your help.
I'll let you know as soon as I have working build environment.
Have you tried other client commands?
nzbget -L
nzbget -G 100
nzbget -A /path/to/filename.nzb
nzbget -P
nzbget -U
nzbget -Q
None works?
Also try to put a nzb-file to income-directory ($MAINDIR/nzb), but you need to create the directory before you start the server. The program should find the file (after about one minute) and start downloading.
chief12345
11-01-2008, 07:22
Hi hugbug,
Other client commands don't work either.
At least I tried
nzbget -L
nzbget -A /path/to/filename.nzb
nzbget -Q
None of them are working.
Putting a nzb-file into $MAINDIR/nzb works. After a few minutes the server starts to download.
For info: chief12345's problem on nslu2 is solved: his server was configured to use the same port already used by another daemon. Therefore the communication between server and client did not work.
The server's code was extended to report of binding errors (like "port already in use" in this case).
New version - nzbget-0.3.1 - is released and available in optware-repository.
New version has got many improvements which can be divided into two categories: better usage of system recourses (optimized algorithms, less memory usage, decreased disk-io) and new handy features (group-view and group-commands, more accurate speed indicator and throttling, etc.).
Full list of changes available here (http://nzbget.sourceforge.net/history.html).
New version is highly recommended especially for routers and NAS-devices, because of greatly reduced memory consumption.
methanoid
04-02-2008, 19:13
What would get me using NZBget would be PAR/RAR support like with SABnzbd and HellaNZB. SABnzbd+ is now out and fixes the memory leak left from earlier versions.
Would be cool also if this could be controlled by TorrentFlux b4rt ... ;)
What would get me using NZBget would be PAR/RAR support like with SABnzbd and HellaNZB. SABnzbd+ is now out and fixes the memory leak left from earlier versions.
The memory leak may have been fixed, but SABnzbd still uses alot of memory. 8 MB with nothing in its queue, and 16 MB with a few nzb's. (and perhaps a lot more when parring).
Nzbget on the other hand, has memory usage of only about 2 MB (and a little more during parring). And if you really want automatic unrar, you can realize this with the post-processing-script.
What would get me using NZBget would be PAR/RAR support like with SABnzbd and HellaNZB.
NZBGet has (imho) better par-support than Sabnzbd/Hellanzb, because nzbget does not scan all files again after the download of additional par-files.
The unrar-support is in todo-list, but you can already use external post-script to unrar the files. However on wl500gp the automatic unrar is not useful anyway. The router has very bad disk-speed (about 3 MB/s), so the direct unpacking from router to pc requires the same time as the copying of (already unpacked) files.
Web-interface (via PHP-application, like Zussaweb for Hellanzb 0.13) is planned for the next version.
I have one request for you: if you have fast (10 MBit/s or better) internet-access, it would be great if you could test the download speed in NZBGet to compare it to Sabnzbd and Hellanzb. My tests on pc with local newsserver showed, that nzbget 0.3.1 is about 1.5-2 times (50-100%) faster than sabnzbd 0.3 and about 30-50% faster than Hellanzb. However it would be very interesting to see how it performs in real world on a device with limited hardware capabilities like router or NAS.
If you decide to do the test, I recommend to activate the option "DirectWrite" (for ext2/ext3 partitions). With that option nzbget writes decoded articles directly to destination file without creating of intermediate files and therefore eliminates the necessity of joing articles (this means less disk-io).
methanoid
04-02-2008, 21:43
NZBGet has (imho) better par-support than Sabnzbd/Hellanzb, because nzbget does not scan all files again after the download of additional par-files.
The unrar-support is in todo-list, but you can already use external post-script to unrar the files. However on wl500gp the automatic unrar is not useful anyway. The router has very bad disk-speed (about 3 MB/s), so the direct unpacking from router to pc requires the same time as the copying of (already unpacked) files.
Web-interface (via PHP-application, like Zussaweb for Hellanzb 0.13) is planned for the next version.
I have one request for you: if you have fast (10 MBit/s or better) internet-access, it would be great if you could test the download speed in NZBGet to compare it to Sabnzbd and Hellanzb. My tests on pc with local newsserver showed, that nzbget 0.3.1 is about 1.5-2 times (50-100%) faster than sabnzbd 0.3 and about 30-50% faster than Hellanzb. However it would be very interesting to see how it performs in real world on a device with limited hardware capabilities like router or NAS.
If you decide to do the test, I recommend to activate the option "DirectWrite" (for ext2/ext3 partitions). With that option nzbget writes decoded articles directly to destination file without creating of intermediate files and therefore eliminates the necessity of joing articles (this means less disk-io).
I would but its only a 4mb connection which both Sabnzbd and Hellanzb have no trouble maxing out :( But the connection gets free upgrade to 10mbit soon (or so they tell me!)
I'd be happy to offer suggestions or constructive criticism on web interface having used Zussaweb and HellaPHP (the latter I persuaded the author to make some changes to) and now I am using SABnzbd web interface.
If you want to exchange ideas my email is easy - myusername@gmail.com ;)
Hi,
I have a 12mbit connection and it does not max out with Hella.
I can report 900+kbyte/s with some spikes to 1+mbyte/s only pure downloading (no verify or unrar). with hellanzb. Sabnzbd got me as far as 500kbyte/s, with some occasional spikes.
the verify and unrar takes up time true, but the files hardly every leave the asus so I don't copy to pc, that's why I want the downloads verified and extracted.. still using Hella for that.
methanoid
04-02-2008, 22:00
Hi,
I have a 12mbit connection and it does not max out with Hella.
I can report 900+kbyte/s with some spikes to 1+mbyte/s only pure downloading (no verify or unrar). with hellanzb. Sabnzbd got me as far as 500kbyte/s, with some occasional spikes.
the verify and unrar takes up time true, but the files hardly every leave the asus so I don't copy to pc, that's why I want the downloads verified and extracted.. still using Hella for that.
raas, Hella and Sab can both max a 20mbit connection on a Linkstation (ARM), Sab performance has improved much with the + version. However I am quite intrigued by nzbget.
I have to say I admire Hugbug for coming on here and sharing his views with us and telling us what is planned. I know its not a competition for users but I for one prefer using actively developed programs and those where the author listens to what his users want.
Me, I'd still like unRAR - as a disable option like with SAB and Hella - not everyone will be using these progs on really slow CPUs and I like to leave a download and come back to the files I wanted, regardless of time. Fire and forget! Its cheaper to leave NAS box on 24/7 than a PC.
Me, I need web control. If its quiet at work I like to add NZBs to my queue! Another plus for Hugbug, he even admits all security risks of his software. The others are just as open programs but the authors don't mention it!
I'm certainly tracking this program! :)
Hi Methanoid,
I've never tried the + version of Sab, so I don't have an opinion/experience with it. The figures I mentioned are from the old version. Just posted my results and experiences. If someone has better experiences that's even better. :)
You're right. I also appreciates hugbug's work (also from other who contribute with awesome software.) I'm programming M$ for over 10 years now, but only recently did my first scripts in a *nix environment. I can't contribute as much as I wanted. (so for now I just concentrate on building images which can be installed, but even that is not working 100% at the moment.)
Indeed, the approach of having the NAS on all the time which is much cheaper than a pc, leads me to having as much functionality on this device, That's also why I'm still using hella, fire and forget, like you said.
At the moment it's still essential that packages get extracted on the box itself, As i mentioned in other posts I listen/view this content throught an modded xbox (1) with xbmc running. So I hardly ever copy content to the pc to view/edit it there, or burn it to a disk.
Just like you I'm still tracking this software, and when the moment comes that all things I've been looking for are implemented, I will definately consider making the switch to nzb, because it's really 'open' and writtn/ported towards this piece of hardware.
Anyway, thumbs up for hugbug and may he succeed.
raas
Thanks for benchmars-results.
With local newsserver I achieved about 1100 KB/s average in nzbget.
methanoid
What device do you use: wl500gp or Linkstation?
Hello,
I'm not able to compile nzbget under Linux
I get the following errors:
ParChecker.o: In function `slot_rep':
/usr/include/sigc++-2.0/sigc++/functors/slot_base.h:84: undefined reference to `sigc::trackable::trackable()'
ParChecker.o: In function `slot1<sigc::bound_mem_functor1<void, ParChecker, std::basic_string<char, std::char_traits<char>, std::allocator<char> > > >':
/usr/include/sigc++-2.0/sigc++/functors/slot.h:526: undefined reference to `sigc::slot_base::slot_base(sigc::internal::slot_r ep*)'
ParChecker.o: In function `sigc::signal1<void, std::basic_string<char, std::char_traits<char>, std::allocator<char> >, sigc::nil>::connect(sigc::slot<void, std::basic_string<char, std::char_traits<char>, std::allocator<char> >, sigc::nil, sigc::nil, sigc::nil, sigc::nil, sigc::nil, sigc::nil> const&)':
/usr/include/sigc++-2.0/sigc++/signal.h:1799: undefined reference to `sigc::signal_base::connect(sigc::slot_base const&)'
ParChecker.o: In function `~slot1':
/usr/include/sigc++-2.0/sigc++/functors/slot.h:495: undefined reference to `sigc::slot_base::~slot_base()'
/usr/include/sigc++-2.0/sigc++/functors/slot.h:495: undefined reference to `sigc::slot_base::~slot_base()'
ParChecker.o: In function `slot_rep':
/usr/include/sigc++-2.0/sigc++/functors/slot_base.h:84: undefined reference to `sigc::trackable::trackable()'
/usr/lib/libpar2.so: undefined reference to `sigc::slot_base::slot_base()'
/usr/lib/libpar2.so: undefined reference to `sigc::slot_base::~slot_base()'
/usr/lib/libpar2.so: undefined reference to `sigc::internal::signal_impl::sweep()'
/usr/lib/libpar2.so: undefined reference to `sigc::signal_base::~signal_base()'
/usr/lib/libpar2.so: undefined reference to `sigc::signal_base::signal_base()'
/usr/lib/libpar2.so: undefined reference to `sigc::slot_base::slot_base(sigc::slot_base const&)'
collect2: ld returned 1 exit status
make[1]: *** [nzbget] Fehler 1
make[1]: Leaving directory `/home/innersoul/Desktop/nzbget-0.3.1'
make: *** [all] Fehler 2
I'm pretty sure all requirements are installed, configure went fine, but i'm stuck at "make".
google wasn't my friend on this one. any idea?
jajaja
Since, this thread is mainly for discussion of nzbget on optware, I moved your question to forum on SourceForge (http://sourceforge.net/forum/forum.php?thread_id=1936054&forum_id=353742). Please see my answer there, you can post anonymously if you do not have a SF-account and do not want one.
nzbget-0.4.0 and nzbgetweb-1.0 released
The most important improvement in the new version of NZBGet is the added support for XML-RPC or JSON-RPC. Based on these protocols a web-interface was developed - NZBGetWeb. It is a separate application written in PHP, which can be installed on any web-server with PHP-support.
Other changes in NZBGet: support for UU-format in internal decoder; better handling of communication errors and bad articles; redirection of output from post-process-scripts to server’s log; displaying of accurate progress-information during par-check/repair (in web-interface); other small improvements and bug fixes.
Full ChangeLog (http://nzbget.sourceforge.net/history.html).
----------------------------
To update from previous version:
ipkg update
ipkg upgrade nzbget
I recommend to use the updated nzbget.conf.example file. It contains new options. If you decide to keep your old config you need to change the option "decoder=yEnc" to "decode=yes".
----------------------------
About installing of web-interface.
NZBGetWeb is a PHP-application. It must be installed separately (package "nzbget" in repository does not include it).
You need to download it from NZBGet home-page (https://sourceforge.net/project/showfiles.php?group_id=102632&package_id=271100), unpack to router, configure NZBGetWeb (please read README) and your web-server.
I use php-thttpd, but it has its own issues (for example due a bug in thttpd the uploading of nzb-files does not work). You may decide to use other server.
For hints on how to install web-interface on lighttpd please refer to this thread (http://wl500g.info/showthread.php?t=11481&highlight=hellanzb) from DrChain where he explains how to install ZussaWeb for HellaNzb. The installation of NZBGetWeb is very same (in fact NZBGetWeb is based on ZussaWeb). You should skip a part about fixing of a bug in "functions.php" as it is already fixed in NZBGetWeb.
Jeroen van Omme
22-04-2008, 08:44
Hi,
just updated NZBget using the procedure above.
I changed the decode parameter, but after starting I get:
" for option "CreateBrokenLog"
Any suggestions?
Is it exactly the text you get? It looks strange. Can you please post a screenshot?
It also makes sense to check the option "CreateBrokenLog" in config-file.
Jeroen van Omme
22-04-2008, 09:59
Hi,
I seems the old config file was broken. I used the one packaged with the latest version and I've got it running.
Sorry for the panic :D
Jeroen
allochtheun
07-05-2008, 10:46
Hi,
I'am a very satisfied user :D of NZbget and i'm recently have done the upgrade to the latest version.
Is still find it a bit strange that when I do a download through nzbget I can download with approx 390 / 400 Kb's . And when I do the exact download on a Windows machine I received a throughput a 750/800 kb's. What can seems to be the problem here. I've checked all kinds of settings within the Nzbget.conf, but nothing seems the speed up my connection. Does anyone might have a suggestion??
Kinds regards
Allochtheun
methanoid
22-05-2008, 09:12
Hi, lurker returns.
I've been playing with Hellanzb for a while and then to SABnzbd and to be honest SAB was fine for me when I was on 4mbit connection with my Linkstation, although obviously slow for PAR/RAR operations. Now I finally got upgraded to 10mbit and the Linkstation struggles to max out the 10mbit connection, so I am looking at options.
Has anyone actually done any peformance comparisons with fast connections and the WL500 (or other NAS-like hardware) so we can see if NZBget does deliver a performance increase?
Meth
Hi Methanoid,
You'll remember me from the LinkStationwiki forum where I wrote up the guide to getting SABnzbd working on LinkStations.
I spent a few hours getting nzbget, nzbgetweb and postprocessing scripts working on my LinkStation Live.
nzbget is a *MUCH* better app than SAB for LinkStations. Particularly if you have a fast connection.
I have a 20mbit connection (16mbits/sec real throughput) and my LinkStations can barely manage 8mbits/sec.
nzbget maxes out my line with no trouble. Fantastic app, this. Well done!
Also, when downloading or par checking large files with SAB, the LinkStation would often crash overnight.
I downloaded, par checked and unpacked a 27GB download with nzbget and it sailed through, using as little as 3% of the LinkStation Pro's RAM.
It's fairly easy to install nzbget -- ipkg install nzbget!
Setting up lighttpd and nzbgetweb took a bit more effort, but not much.
Futzing with the post-processing scripts took longer.
One major problem (hugbug?) is that nzbget on the LinkStation Pro doesn't appear to be setting the umask correctly.
I ended up adding a line:
chmod -R 777 "$DownloadDir"
to the end of a simple unrar script to fix this.
However, this meant I couldn't use any of the sophisticated perl scripts around as I could not work out how to add a chmod command in perl -- any ideas on that?
I may write up a guide on the linkstationwiki forums if there's interest.
Cheers,
Neil.
methanoid
20-06-2008, 21:11
Guide would be great... esp since my SABnzbd on Linkstation has been flaky with big files recently....
Its the Web interface thats the issue!!
One major problem (hugbug?) is that nzbget on the LinkStation Pro doesn't appear to be setting the umask correctly.
Could you please describe the problem? There was a bug with umask, that prevented the setting of write-permissions for created directories to "group" and "others". It was fixed in Revision 178 of 0.4.1-testing. That version was not released yet, but is available from subversion-repository as source and also as binary package for few optware-platforms (see download page (http://sourceforge.net/project/showfiles.php?group_id=102632&package_id=264752&release_id=606973)).
What optware-target is used on linkstation? I could try to build the binary for you.
Could you please describe the problem? There was a bug with umask, that prevented the setting of write-permissions for created directories to "group" and "others". It was fixed in Revision 178 of 0.4.1-testing. That version was not released yet, but is available from subversion-repository as source and also as binary package for few optware-platforms (see download page (http://sourceforge.net/project/showfiles.php?group_id=102632&package_id=264752&release_id=606973)).
What optware-target is used on linkstation? I could try to build the binary for you.
If I set umask to 777 in nzbget.conf, directories created are not accessible from other machines.
LSPro feed: http://ipkg.nslu2-linux.org/feeds/optware/cs05q3armel/cross/stable/
I've downloaded the 178 rev, will try out later.
Cheers!
EDIT: Upgraded to new version. UMask=0777 or UMask=777 still has same problem -- can't open directory of downloaded nzb in dst.
UMask=0777 or UMask=777 still has same problem -- can't open directory of downloaded nzb in dst.
UMask uses different syntax in contrast to chmod.
To give to all created directories and files full-permissions (except execute-permission for files, but you do not need it) - use "UMask=000". It will result in permissions 777 for directories and 666 for files.
See UMask (http://en.wikipedia.org/wiki/Umask) for more info.
methanoid
21-06-2008, 10:45
Come on Neilt0, you've had several hours. Where is that guide?? ;)
UMask uses different syntax in contrast to chmod.
To give to all created directories and files full-permissions (except execute-permission for files, but you do not need it) - use "UMask=000". It will result in permissions 777 for directories and 666 for files.
See UMask (http://en.wikipedia.org/wiki/Umask) for more info.
I'm trying 000 now and will report back.
Come on Neilt0, you've had several hours. Where is that guide?? ;)
I'm at work today, so not now. Maybe later. Try it yourself though, it's not that hard. :D
methanoid
21-06-2008, 15:12
I'm at work today, so not now. Maybe later. Try it yourself though, it's not that hard. :D
I downloaded the web i/f and the docs made it look too difficult for a lamer like me! ;)
I'm trying 000 now and will report back.
Works!
Note that with the version on the optware feed, you can open folders and files from another machine, but can't delete.
Only with the latest revision on sourceforge can you delte files,
Thanks hugbug!
Quick guide for nzbget on a LinkStation Live or Pro.
Methanoid, try it out if you like -- at your own risk. Make backups etc. blah blah.
If you spot any mistakes, let me know.
Please do not post this guide anywhere else without my permission. Thanks.
First: read all of http://nzbget.sourceforge.net/
ipkg install nzbget lighttpd php php-fcgi
For correct umask to enable file deletion from other machines, download from sourceforge, then change to the directory where you put the ipk and:
ipkg install nzbget-0.4.1-testing-r178-bin-cs05q3armel-arm.ipk
vi /opt/etc/nzbget.conf
edit as follows:
1. $MAINDIR=/mnt/disk1/share/nzbget
(or wherever you want it)
2. Fill in news server section. 20 connections is no problem for nzbget on a LinkStation Pro!
3. UMask=000
or whatever you want.
4. DirectWrite=yes
5. I turned CRC on and retries for CRC on. Still fast.
6. WriteBufferSize=-1
Download nzbgetweb and put it in /opt/share/www/nzbgetweb/
opt/etc/init.d/S80lighttpd stop
vi /opt/etc/lighttpd/lighttpd.conf
change
# "mod_fastcgi",
into
"mod_fastcgi",
change
url.access-deny = ( "~", ".inc" )
into
url.access-deny = ( "~", ".inc", ".sqlite" )
vi /opt/share/www/nzbgetweb/settings.php
Settings here are well documented in the file.
/opt/etc/init.d/S80lighttpd
Find a postprocessing script you like, put it wherever.
Edit nzbget.conf to tell it where you put it.
chmod a+x /path/scriptname.sh
nzbget -D
http://LINKSTATION_IP_ADDRESS:8081/nzbgetweb/
Done and done!
methanoid
21-06-2008, 19:43
Thanks Neil, done half of it already by myself but have to go out.. will try later and report... Did you check speed by comparing file size to time or take the figure nzbget gave you?
What options did you enable in the CONF? PAR2? UnRAR?
Did you check speed by comparing file size to time or take the figure nzbget gave you?
What options did you enable in the CONF? PAR2? UnRAR?
Par repair in nzbget.conf, other postprocessing (unrar etc.) has to be done by a script.
The speed indication on the latest revision is more accurate. I didn't actually benchmark it, I just took it as read. It's definitely quick...
EDIT:
Benchmarks
LinkStation Pro (400MHz ARM)
128MB RAM
750GB SATA drive
XFS
My max actual throughput ~16.1mbps
Test file: 399.3MB NZB = 361.5MB actual download
20 connections
Web interface refresh rate: 1 second
Download: 3 minutes, 6 seconds = 15.56mbit/sec
Par check (repair not needed): 1 minute, 55 seconds
Unrar: 1 minute, 3 seconds
I'm happy with 15.5mbps+ of my 16.1mbps. :D
Note also, I'm getting 50% of my max throughput using just ONE connection to the usenet server. This is a VERY efficient usenet downloader.
methanoid
22-06-2008, 00:18
Dont need to ipkg install php as php-fcgi includes it?
nzbgetweb/settings.php
Settings here are well documented in the file. - but I think for a guide we need to be a tad more explicit.. last line for example of the conf file?
Could expand this:
Find a postprocessing script you like, put it wherever.
Edit nzbget.conf to tell it where you put it.
chmod a+x /path/scriptname.sh
Anyway I got the webserver up but not working. Whenever I try to upload a file I get
"Error: Check the path and the permissions for the upload directory (option nzbdir)"
I notice nzbget doesnt create directories when run so I had to create them manually in Windows so they show as nobody|nogroup for owner but I CHMOD -R 777 for them but still no joy...
So currently I cannot use nzbget yet...
And even when I can.. well it needs some extra tool as SABnzbd has a cool Firefox addin NZBstatus which would be nice to have with nzbget.
I'm certainly willing to change... one plus for nzbget is it wont interfere with running a website like SAB can with port usage....
I notice nzbget doesnt create directories when run so I had to create them manually in Windows so they show as nobody|nogroup for owner but I CHMOD -R 777 for them but still no joy...
It must create all directories. Probably a permission-problem. Are there any errors in log-file?
Try to start the program from root-account.
Anyway I got the webserver up but not working. Whenever I try to upload a file I get
"Error: Check the path and the permissions for the upload directory (option nzbdir)"
First check if nzbget works by uploading nzb-files via samba or ftp. Later we could try to fix the problem with web-interface.
methanoid
22-06-2008, 12:19
It must create all directories. Probably a permission-problem. Are there any errors in log-file?
Try to start the program from root-account.
First check if nzbget works by uploading nzb-files via samba or ftp. Later we could try to fix the problem with web-interface.
hi, OK it was a warmware problem :D
I editted the sample conf file and missed a ~ so my NZBget dir was ~/mnt/disk1/share/nzbget rather than /mnt/disk1 etc.... Ooooops
nzbget now creates the dirs but wont let me copy files into them so I had to chmod -R 777 nzbget directory and now I can copy an NZB file in but NZBget isnt doing anything with it... I run NZBget with "nzbget -D"
Nothing in log file bar NZBget running in Daemon mode.
if nzbget is as good on a Linkstation as Neilt0 says I'll be keeping this Linkstation and saving buying a low power usage PC for downloads...
My fingers are crossed!! :o
nzbget now creates the dirs but wont let me copy files into them so I had to chmod -R 777 nzbget directory
There was a permission bug. Install the latest version from nzbget's home page with
ipkg install http://surfnet.dl.sourceforge.net/sourceforge/nzbget/nzbget-0.4.1-testing-r178-bin-cs05q3armel-arm.ipk
Set the option "umask=000" in the configuration file.
The created directories now should be accessible from all user-accounts, including samba.
now I can copy an NZB file in but NZBget isnt doing anything with it...
Check options "NzbDirInterval" and "NzbDirFileAge". Value "5" for the last option should be enough. Check if date/time is set properly on you linkstation and if nzb-file has a proper date/time (files downloaded from some indexing servers have errorneous timestamp in the feature).
methanoid
22-06-2008, 14:50
Yeah I had the correct nzbget and umask - I did read the "guide" post...
It was the time though.. well spotted... as soon as I changed my NTP server it worked.. The default LS time server was dead.. put a proper one in and its downloading away.. I'll test web interface next!
Thanks Hugbug, great spot!
methanoid
22-06-2008, 16:55
Well it downloads beautififully and easily maxes my connection which SAB didnt do... but I need to add post processing script to unrar (IMHO this should be in NZBget as standard!!), just integrate ydrol's script.
Some questions if I may
1) to add ydrol's script which is Perl I assume I have to add Perl - which? Full Perl or would MicroPerl do? Details at http://ipkg.nslu2-linux.org/feeds/optware/cs05q3armel/cross/stable/Packages
2) likewise I assume I have to install unRar (done) ?
3) Reason web i/f wont work and allow UL of NZB MIGHT be the lack of any option in $rpc_api='' - which do I use? I have installed PHP-FCGI and used XML_RPC before when I used to use Hellanzb so is that what I choose?
If I am totally honest I am VERY impressed with NZBget's speed but it's not as easy to configure as SABnzbd and as friendly. SAB has many more features but I guess they cost in one way or another? I like the way that SAB integrates with Newzbin website (can auto download bookmarked NZBs) and can process NZBs into categories (different file locations for say TV shows from Music downloads). I like auto PAR and auto RAR. I also have a great Firefox addin called NZBstatus which monitors SAB's progress.
But speed matters... really.. and I think NZBget could be developed further (and seems to be)...
Is there a road map for the future of this great package Hugbug? Please share it with us if you will
I do not use any postprocessing scripts myself. Sorry I can't help on configuring them. I hope neilt0 could share his experience on that part.
The reason to not using postprocessing and unrars for me is, that asus wl500gp has very slow speed accessing usb-attached hard drive (about 2-3 MBytes/sec). Because of that I can not use it as a shared drive, but need to copy downloaded files to desktop. And here we come to the point - the speed of copying files from wl500gp to desktop pc is the same as unraring the files from wl500gp to desktop pc. So unraring on wl500gp would be just unnecessary work for the router.
3) Reason web i/f wont work and allow UL of NZB MIGHT be the lack of any option in $rpc_api='' - which do I use? I have installed PHP-FCGI and used XML_RPC before when I used to use Hellanzb so is that what I choose?
The option affects communication between web-interface-app and nzbget-server. If you see download queue and log in web-interface, that means the communication works.
You may leave the option empty. On optware-platform json-extension (built into php) will be used automatically; this is the best option for speed.
The option does not affect uploading of the files. The uploading does not need any communication with nzbget-server. Web-app just put uploaded files into nzbget's incoming directory. If it does not work check permissions for that directory. Also try some other nzb-file. I have had problems with some files (with special characters in their names). For test rename any nzb-file to something very simple (test.nzb) and put it into "C:\", then try to upload it.
BTW, from a local network it is easier to upload files using network-dir. I created a shortcut for nzbget's nzb-dir to sendto-menu of windows explorer. So any file can now be easily sent to nzbget via explorer's context menu.
Is there a road map for the future of this great package Hugbug?
I'm sorry to disappoint you, but I'm not planning any new features.
Perl is easy: ipkg install perl
Web interface:
vi /opt/share/www/nzbgetweb/settings.php
I use:
$serverip = '192.168.0.16';
Make sure that's set to your LSPro IP.
Other than that, dunno why it's not working for you. Hugabug may. All I can recommend is carefully check all settings in the configs.
The major feature I miss from SAB is nzbdstatus, but nzbget is so fast at uploading NZBd via the web interface, it's not a huge loss.
There are major advantages to nzbget: speed, RAM usage, libpar2 instead of par2 for single pass par2, scripts can be more sophisticated.
SAB's fine for a fast PC -- I've used it for years, but for a device like the LSPro, I don't think there's any choice. It has to be nzbget.
methanoid
22-06-2008, 19:51
Another warmware error.. I mistyped the NZB file directory in the Settings.php
Will need to remove PHP from the list of install requirements as PHP-FCGI installs PHP anyway and put in Perl if post processing script used and optional unRAR into install list.
Files upload to NZB dir fine but won't start downloading - still looking into.
Have to confess I am disappointed that no new features are planned at all. That's one thing I have liked about SAB, the devs have added stuff I have requested. Not big stuff but still they keep tweaking and improving.
Fingers crossed the postprocessing script works now... my net connection gets throttled from 4pm so its chugging a bit...
Are all my files supposed to be called
(null)
(null)_duplicate1
(null)_duplicate2
(null)_duplicate3 etc?? Seems I have another issue!
Are all my files supposed to be called
(null)
(null)_duplicate1
(null)_duplicate2
(null)_duplicate3 etc??
No, they don't :)
I suppose these names were created by nzbget, not by a postprocess-script? Try a different nzb-file from a different source and report if it works.
methanoid
22-06-2008, 20:30
No, they don't :)
I suppose these names were created by nzbget, not by a postprocess-script? Try a different nzb-file from a different source and report if it works.
Seems to work on others.. musta been a rogue!
I really don't like the way it keeps NZB files when they have been completed and you have to delete them manually.. oh well... I guess I better hope the developers of the post processing scripts do all the things I still want to do.
I'll have to ask them how to fix such things like
WARNING PostProc:Unable to rename "/mnt/disk1/share/nzbget/nzb/Big Brother (UK) - 2008-06-20 - Big Ears Show 5.nzb" to .processed ;)
It downloads fine now from web uploaded NZBs and ones copied via SMB to the NZB directory. I think I may miss the auto download of Newzbin bookmarks!
Seems to work on others.. musta been a rogue!
Please send me a problem nzb-file for testing.
I really don't like the way it keeps NZB files when they have been completed and you have to delete them manually
Why bother? Delete them once a month :).
I believe both perl-scripts delete them also.
methanoid
22-06-2008, 21:42
Why bother? Delete them once a month :).
I believe both perl-scripts delete them also.
"I'll have to ask them how to fix such things like
WARNING PostProc:Unable to rename "/mnt/disk1/share/nzbget/nzb/Big Brother (UK) - 2008-06-20 - Big Ears Show 5.nzb" to .processed "
didnt work.. I'll investigate that... actually I have some good ideas for post processing scripts
What you would want to do with .queued files? Just deleted them upon successful download/par-check? I could add an option to nzbget.
methanoid
24-06-2008, 20:21
What you would want to do with .queued files? Just deleted them upon successful download/par-check? I could add an option to nzbget.
Yeah that would be nice.... but someone (??) said
"I'm sorry to disappoint you, but I'm not planning any new features."
So I didnt ask any more..
I'm torn between two options now (not bad ones though).
I love SABnzbd for its usability but they won't improve the speed and said it's cos you write in C++ and they use Python that you can beat them for raw speed. I asked for a few other tweaks (because no program is ever perfect for you unless you write your own!!)
I love NZBget (yes, that WAS quick) because it's so quick. In many ways I am coming around to the idea of post processing by scripts, especially when ydrol's script uses PAR/RAR as lower priority processes. Nice script. But I still want more features in NZBget and I can understand your reluctance to add so many things. Have to also say that (touch wood) NZBget has been very stable!
I guess I'll have to make my mind up what I want. :o
Yeah that would be nice.... but someone (??) said
"I'm sorry to disappoint you, but I'm not planning any new features."
he-he, I did not have plans, because I did not have feature requests ;)
I have implemented most of requested features (auto-par, xml-rpc, web-interface, etc).
But unraring can be done via postprocess-scripts even better than from nzbget. If there were "libunrar", nzbget could provide some benefits over scripts (like a better progress-indication in par-check/repair).
they use Python that you can beat them for raw speed
Not just c++, but also:
- better algorithms like DirectWrite-feature (to avoid joining-step - reduce disk-io);
- many speed optimisations (after code-profiling);
- very thrifty memory usage.
To compare: nzbget is two (or three, don't remember numbers) times faster than Newsleecher, which is written in Delphi (as fast as c++).
But: you want write xml-rpc server in python, you use internal classes (part of python) and only implement your rpc-methods. In c++ you need to write web-server first, then write xml-parser and only then implement rpc-methods. If you got luck, you could find a suitable library on sourceforge, but it would have incompatible license or will not work on windows or will need other big libraries, which is not ported on optware yet.
------------
How about new (planned?) feature - categories:
- each nzb-file can be assigned to category;
- incoming-directory can have subdirectories (one level enough?); by uploading a nzb-file to the subdirectory, the file is automatically categorized. Like multiple blackholes in sabnzbd;
- for each category nzbget will create a subfolder under dst-directory;
- the ability to change category via web-interface;
Someone need that?
How about new (planned?) feature - categories:
- each nzb-file can be assigned to category;
- incoming-directory can have subdirectories (one level enough?); by uploading a nzb-file to the subdirectory, the file is automatically categorized. Like multiple blackholes in sabnzbd;
- for each category nzbget will create a subfolder under dst-directory;
- the ability to change category via web-interface;
Someone need that?
I'm very happy with nzbget as it is, but it would be handy to have categories. This would be most useful in combination with integrating into nzbget an NZB downloader from Newzbin -- feed a NewZbin ID to nzbget via the web interface -- e.g. 3093573, nzbget would then download the NZB from NewZbin using their API, download the collection and unpack it into the appropriate category (e.g. TV).
That would also open the door for a third party to come up with an nzbdstatus like add-on (in fact nzbdstatus itself could easily support nzbget).
Anything more complicated than that and you may as well move the directories manually?
P.S. nzbget is currently downloading on my LinkStation at a rate faster than any of my Windows PCs can. It's fantastically fast!
This would be most useful in combination with integrating into nzbget an NZB downloader from Newzbin
Yes, Newzbin-integration would be the next step.
Anything more complicated than that and you may as well move the directories manually?
I'm not getting what you mean :confused:
If you have to monkey around with manually creating directories, it doesn't seem that much easier than dragging stuff manually?
Well I guess it's a bit easier. Um.
methanoid
25-06-2008, 12:48
But unraring can be done via postprocess-scripts even better than from nzbget. If there were "libunrar", nzbget could provide some benefits over scripts (like a better progress-indication in par-check/repair).
There IS a libunrar http://www.freshports.org/archivers/libunrar/
But I cannot see any ARM port of this :(
If you have to monkey around with manually creating directories, it doesn't seem that much easier than dragging stuff manually?
The destinanion directories will be created automatically.
You need to create manually only the incoming subfolders and only if you want to put nzb-files in that folders manually.
If you upload files via web-interface, you just choose a category (via combobox probably). The same for newzbin - the category will be read from newzbin.
There IS a libunrar http://www.freshports.org/archivers/libunrar/
It's interesting. Thanks.
I looked in Debian repository, in optware repository on rarsoft-site and did not found any libunrars. I thought there is none :)
I'll look into it.
This version contains fixes for several bugs, including the bug, that broke the web-interface on CPUs with Big-Endian architecture (ARM, PPC, etc). Other bugfixes are not so important but you might want to update anyway.
See the full list of changes here (http://sourceforge.net/project/shownotes.php?release_id=612630&group_id=102632).
The new version is already in optware-repository.
-----------------
Roadmap (methanoid ;)):
- I'm now working on category-feature;
- next (probably): newzbin-integration.
methanoid
11-07-2008, 19:50
Roadmap (methanoid ;)):
- I'm now working on category-feature;
- next (probably): newzbin-integration.
Thanks, can you make an iPhone version too? ;)
I just did a clean install of nzbget on a new LinkStation Pro, all is working except post-processing perl scripts are seemingly skipped. Not sure why?
Didn't it even say "Executing post-process-script for..."?
Can you please test with binary "0.4.1-r185-testing" from sourceforge? And if it still does not work, then with "0.4.1-r178-testing", which already worked for you, just for test.
The first feature for version 0.5.0 is now implemented and available for testing (release "0.5.0-r191" in package "nzbget-testing").
About categories:
- each nzb-file can have a category assiged to it;
- for each category nzbget will create a subfolder under dst-directory (option <AppendCategoryDir>);
- incoming-directory can have subdirectories (one level of nesting, isn't enough?); by uploading a nzb-file to the subdirectory, the file is automatically categorized;
- new command-line switch <-K> used with switch <-A> to set the category by adding of new nzb-file to queue;
- new subcommand <K> in command-line-switch <-E> to change the category of existing group;
- new parameter to post-process-scripts;
- new parameter in XML-/JSON-RPC-command <editqueue>;
- automatic moving of already downloaded files after the changing of category.
There is also a new version of web-interface with ability to change the category (release "1.1-2" in package "web-interface-testing").
Appreciate any feedback.
Chillout
31-07-2008, 17:43
Hi all,
Thanks for this topic, NZBGet is a GREAT alternative to sabnzbd.
Currently I'm using the following post-processing script:
#!/bin/sh
# 1 - path to destination dir, where downloaded files are located;
# 2 - name of nzb-file processed;
# 3 - name of par-file processed (if par-checked) or empty string (if not);
# 4 - result of par-check:
# 0 - not checked: par-check disabled or nzb-file does not contain any
# par-files;
# 1 - checked and failed to repair;
# 2 - checked and sucessfully repaired;
# 3 - checked and can be repaired but repair is disabled;
# 5 - state of nzb-job:
# 0 - there are more collections in this nzb-file queued;
# 1 - this was the last collection in nzb-file;
# 6 - indication of failed par-jobs for current nzb-file:
# 0 - no failed par-jobs;
# 1 - current par-job or any of the previous par-jobs for the
# same nzb-files failed;
Sendmail=False
from=MyBook@isp.com
sendto=ReMiA@isp.com
server=mail.isp.com
nzbdir=/tmp/harddisk/torrent/target/
UNRAR=True
declare -a RARFILES
dir=$1
dest=/tmp/harddisk/torrent/target/
PauseServer=True
if [ "$5" = "1" ]
then
chmod -R 766 "$1"
subject="Download of $1 completed"
if [ "$4" = "0" ] ; then
echo "PAR2 not checked: par-check disabled or nzb-file does not contain any par-files" > "$nzbdir/message.txt"
elif [ "$4" = "1" ] ; then
echo "PAR2 checked and failed to repair" > "$nzbdir/message.txt"
elif [ "$4" = "2" ] ; then
echo "PAR2 checked and sucessfully repaired" > "$nzbdir/message.txt"
elif [ "$4" = "3" ] ; then
echo "PAR2 checked and can be repaired but repair is disabled" > "$nzbdir/message.txt"
elif [ "$4" = "" ] ; then
echo "No PAR2 returncode" > "$nzbdir/message.txt"
fi
if [ -f "$1/_brokenlog.txt" ] ; then
echo >> "$nzbdir/message.txt"
echo "Broken Files:" >> "$nzbdir/message.txt"
cat "$1/_brokenlog.txt" >> "$nzbdir/message.txt"
fi
fi
if [ $UNRAR = "True" ] ; then
# PAUSE Server
if [ $PauseServer = "True" ] ; then
/opt/bin/nzbget -P
fi
cd "$dir"
RARFILES=(`ls | grep -E [.][pP][Aa][Rr][Tt][0]*[1][.][Rr][Aa][Rr] | tr ' ' '§'`)
number_rar=${#RARFILES }
if [ $number_rar -ge 1 ] ; then
for b in "${RARFILES[@]}"
do
rar=${b//§/ }
f=${rar/[.][pP][Aa][Rr][Tt][01]*[.][Rr][Aa][Rr]/}
destdir=$dest$f
mkdir "$destdir"
unrar x -y -o- -p- "$rar" "$destdir"
echo "UNRARRED $rar to $destdir" >> "$nzbdir/message.txt"
done
else
RARFILES=(`ls | grep -E [.][Rr][Aa][Rr] | tr ' ' '§'`)
number_rar=${#RARFILES }
if [ $number_rar -ge 1 ] ; then
for b in "${RARFILES[@]}"
do
rar=${b//§/ }
f=${rar/[.][Rr][Aa][Rr]/}
destdir=$dest$f
mkdir "$destdir"
unrar x -y -o- -p- "$rar" "$destdir"
echo "UNRARRED $rar to $destdir" >> "$nzbdir/message.txt"
done
fi
chmod -R 766 "$destdir"
fi
# UN-Pause server
if [ $PauseServer = "True" ] ; then
/opt/bin/nzbget -U
fi
fi
if [ $Sendmail = "True" ] ; then
sendEmail -t $sendto -f $from -u $subject -s $server -o message-file="$nzbdir/message.txt" -q
fi
exit 0
But there's an issue with the 'declare' in rule 27... what can I do about this? Or if anyone has a working post-process script to unrar and clean up files, post it here, it seems there are more people interested!
The following script is used on popcornhour box (http://www.popcornhour.com).
It was not designed to be easily portable, so you need to change the absolute paths.
#!/bin/sh
DownloadDir="$1"
NzbFile="$2"
ParCheck=$4
NzbState=$5
ParFail=$6
# Make a logfile
log="$DownloadDir"/unrar.log
export PATH=$PATH:/opt/sybhttpd/localhost.drives/HARD_DISK/Download
# Check if all is downloaded and repaired
if [ "$NzbState" -eq 1 -a "$ParCheck" -eq 2 -a "$ParFail" -eq 0 ]
then
cd "$DownloadDir"
# Make a temporary directory to store the unrarred files
mkdir extracted
# Remove the Par files
rm *.[pP][aA][rR]2
# Pause NZBGet download until script finishes
/mnt/syb8634/bin/nzbget -P -c /opt/sybhttpd/localhost.drives/HARD_DISK/.nzbget/nzbget.conf
# Unrar the files (if any) to the temporary directory, if there are no rar files this will do nothing
if (ls *.rar >/dev/null)
then
/mnt/syb8634/bin/unrar x -y -p- -o+ "*.rar" ./extracted/
fi
# Remove the rar files (Temporaly disabled just remove # to activate it)
# rm *.r[0-9][0-9]
# rm *.rar
# rm *.s[0-9][0-9]
# Go to the temp directory and try to unrar again.
# If there are any rars inside the extracted rars then these will no also be unrarred
cd extracted
if (ls *.rar >/dev/null)
then
/mnt/syb8634/bin/unrar x -y -p- -o+ "*.rar"
fi
# Delete the Rar files (Temporaly disabled just remove # to activate it)
# rm *.r[0-9][0-9]
# rm *.rar
# rm *.s[0-9][0-9]
# Move everything back to the Download folder
mv * ..
cd ..
# Clean up the temp folder
rmdir extracted
chmod -R a+rw .
rm *.nzb
rm *.1
# Rename img file to iso so the NMT can read it
# It will be renamed to .img.iso so you can see that it has been renamed
if (ls *.img >/dev/null)
then
imgname=`find . -name "*.img" |awk -F/ '{print $NF}'`
mv $imgname $imgname.iso
fi
# Unpause NZBGet
/mnt/syb8634/bin/nzbget -U -c /opt/sybhttpd/localhost.drives/HARD_DISK/.nzbget/nzbget.conf
fi
#################
My observations:
- the command "export PATH=$PATH:/opt/sybhttpd/localhost.drives/HARD_DISK/Download" is probably not needed;
- the commands to pause/unpause are optional, you may comment them out to allow nzbget download files during unraring.
That script actually works, I've tested it on popcornhour. It should work on asus as well.
The script was written by Philos and Werner from pocornhour forum (http://www.networkedmediatank.com). See that (http://www.networkedmediatank.com/viewtopic.php?t=4047&postdays=0&postorder=asc&start=0) thread.
Chillout
31-07-2008, 20:00
I understand, I know that it should work, but my log says:
Thu Jul 31 18:26:07 2008 DETAIL Post-Process: /tmp/harddisk/torrent//.nzb/scripts/myscript.sh: /tmp/harddisk/torrent//.nzb/scripts/myscript.sh: 27: declare: not found
Thu Jul 31 18:26:07 2008 DETAIL Post-Process: /tmp/harddisk/torrent//.nzb/scripts/myscript.sh: /tmp/harddisk/torrent//.nzb/scripts/myscript.sh: 60: Syntax error: "(" unexpected (expecting "fi")
OK, I'll test it on asus.
Chillout
31-07-2008, 21:35
cool, thanks for your effort!
This script (formatted version to view in browser (http://nzbget.svn.sourceforge.net/viewvc/nzbget/trunk/postprocess-example.sh?revision=200&view=markup), download link (http://nzbget.svn.sourceforge.net/viewvc/nzbget/trunk/postprocess-example.sh?revision=200)) works on asus. I tested it.
Chillout
06-08-2008, 13:18
working great, thanks for the tip!
I also like the clean-up feature (added some of my own stuff to it as well).
Testing-version can be downloaded from SourceForge (http://sourceforge.net/project/showfiles.php?group_id=102632&package_id=264752).
Hi all,
Since we are on an asus forum here, I wonder if anyone has succeeded to install newer nzbget versions on the Asus box. Command ikpg list only offers version 0.2.3-1 which installs perfectly but doesnot seem to work with nzbget-web. I tried installing newer versions 0.4.x-x but this fails since version 0.4 has some additional dependencies (libpar2, libncurses). I tried to find ipk packages for Olegs optware but didn't find them sofar. Is there somebody who knows where to find libpar2 and libncurses that work with the asus optware?
You need to install oleg firmware and set ipkg-path to oleg-branch of optware repository.
Detailed how to (http://www.wl500g.info/showthread.php?t=10307).
After that you should be able to install the latest stable version of nzbget (0.4.1) with "ipkg install nzbget".
Thanks Hugbug,
pointing to the right package library did the job and everything works fine now.:D
For more please visit nzbget forum (http://apps.sourceforge.net/phpbb/nzbget/viewtopic.php?f=4&t=2).
To upgrade installed version:
ipkg update
ipkg upgrade nzbget
SirAuron
08-03-2009, 14:15
Hi!
I'm sorry I have to push this old thread, but I'm desperate.
I tried to set up nzbget on my Asus WL-500gP v1 with the latest Oleg Firmware 1.9.2.7-10.
I used wengi's guide to set up the main functions and for nzbget i used this how to:
http://wl500g.info/showthread.php?t=17485&highlight=nzbget
but without the HeadStart II part.
Previously I set up hellanzb successfully, but I had to remove it, because of the missing IPv6 support.
My problem is that every nzbget command I tried to run gave me the same error message on the command line:
nzbget: can't resolve symbol '__libc_pthread_init'
I don't get it. I can't find any solution to this message via google.
Can someone explain this to me?
Here is the list of installed ipkg packages on the machine. Maybe there's a clue in it.
coreutils - 6.12-2 - Bunch of heavyweight *nix core utilities
cron - 4.1-7 - Standard vixie cron, with cron.d addition
diffutils - 2.8.1-6 - contains gnu diff, cmp, sdiff and diff3 to display differences between and among text files
e2fslibs - 1.41.4-1 - Ext2 Filesystem Libraries
findutils - 4.2.32-1 - File finding utilities
glib - 2.16.6-1 - The GLib library of C routines.
groff - 1.19.2-2 - front-end for the groff document formatting system
hdparm - 9.12-1 - Linux hard drive parameter utility
ipkg-opt - 0.99.163-10 - The Itsy Package Manager
less - 418-1 - Less file browser
libcurl - 7.19.4-1 - Curl is a command line tool for transferring files with URL syntax, supporting FTP, FTPS, HTTP, HTTPS, GOPHER, TELNET, DICT, FI
libpar2 - 0.2-5 - A library for performing common tasks related to PAR recovery sets
libsigc++ - 2.0.18-1 - libsigc++ implements a typesafe callback system for standard C++.
libtorrent - 0.12.4-1 - libtorrent is a BitTorrent library with a focus on high performance and good code.
libuclibc++ - 0.2.2-8 - C++ standard library designed for use in embedded systems
libxml2 - 2.7.3-1 - Libxml2 is the XML C parser and toolkit developed for the Gnome project.
man - 1.6f-1 - unix manual page reader
man-pages - 2.76-1 - unix manual pages
mc - 4.6.2-1 - Midnight Commander File Manager
nano - 2.1.9-1 - A pico like editor
ncurses - 5.7-1 - NCurses libraries
ncursesw - 5.7-1 - NCurses libraries with wide char support.
nzbget - 0.5.1-1 - A command-line client/server based binary newsgrabber for nzb-files.
openssl - 0.9.7m-5 - Openssl provides the ssl implementation in libraries libcrypto and libssl, and is needed by many other applications and librari
procps - 3.2.7-6 - PROCPS System Utilities
rtorrent - 0.8.4-1 - rtorrent is a BitTorrent client for ncurses, using the libtorrent library.
screen - 4.0.3-2 - A screen manager that supports multiple logins on single terminal
slang - 2.1.4-1 - S-Lang is a multi-platform library designed to allow a developer to create robust multi-platform software.
termcap - 1.3.1-2 - Terminal emulation library
uclibc-opt - 0.9.28-13 - micro C library for embedded Linux systems
unzip - 5.52-3 - A (de)compression library for the ZIP format
xmlrpc-c - 1.11.00-3 - A library providing modular implementation of XML-RPC for C and C++.
zlib - 1.2.3-3 - zlib is a library implementing the 'deflate' compression system.
Thanks
Auron