PDA

Bekijk de volledige versie : Torrent connection encryption - bypass ISP throttling



patrickquek
10-03-2007, 17:29
Hi,

My ISP throttles bittorrent connection bandwidth so I need to have a client that encrypts the connection.

I'm looking for a client on the ROUTER that is able to do this. AFAIK, the Download Manager, ctorrent, enhanced ctorrent are not able to do this.

Are there any other options? If not, i basically have to run off my torrent client on my laptop and can't use the router's client....

thanks!
Patrick

sensimilla
10-03-2007, 20:11
I believe rTorrent does protocol encryption.

You need to use the updated Uclib-opt package feed

more info here http://wl500g.info/showthread.php?t=7743

I am using it on my box and it works very well with a modest amount of torrents.

If you need more instructions on how to install it, let me know :)

wirespot
11-03-2007, 02:28
The problem with rtorrent is that it has no queue, all torrents either run at the same time or not. If you can live with that, it's a great client.

You can also try BitTornado, the latest version has added encryption. Or better use torrentflux-b4rt, which is a web interface and comes with the latest BitTornado included. You'll need PHP, PHP-FastCGI, Sqlite, Python and a web server though, and you won't be able to run more than one or two torrents at the same time because it's heavy on system resources.

Give the torrents some time to pick up speed. I've tried a throttling ISP once myself. Even with Azureus, who has some of the best encryption, it takes a while to build up a proper peer group. Some trackers don't have proper handling, some peers don't use it etc.

Don't expect miraculous speeds. If you double your throttled speed you're lucky. To completely evade throttling you'd have to use full stealth encryption, which in turn limits the choice of peers and trackers even more. Unfortunately not all BT users and trackers use it (yet).

patrickquek
11-03-2007, 03:39
Thanks sensimilla & wirespot for your replies - that's exactly what i'm looking for.

I found this excellent post by wirespot which describes the various alternatives and pro/cons - http://87.239.14.2/~d02042/showthread.php?t=8197.

A question - transmission.cgi in the Oleg firmware seems highly rated, but i guess it doesn't support encryption?

What I'm hoping for in my Torrent client is this:
1. Encryption (mandatory - else will be throttled by ISP)
2. web management
3. Able to download at least 4 simultaneous torrents at the same (i assume for each download can have a reasonable number of connection to other peers)
4. queue management - something usually taken for granted on PC bittorrent clients. I assume minimally it should be able to automatic retry, start a new torrent once another torrent has fully completed.

If someone has the knowledge to put together a quick comparison chart of the top 3-4 torrent client options available and their advantages/disadvantages, i think that will be very invaluable cos that's prob one of the key reasons why we buy the WL500G.

Btw i have not bought the WL500G, I'm about the execute my purchase but just wanna make sure its able to do what I want. If there's any other similar products that are likewise featured *and* linux based that i should consider, i'd be most grateful for your headsup.

Thanks all!
Patrick

sensimilla
11-03-2007, 09:04
I think Wirespot sums up the bittorrent client situation very well.

I believe it would be possible to knock up a simple torrent queueing script for rTorrent that would start the next torrent when one completes, maybe I will try to make one. However rTorrent does not have a web GUI and I doubt that it ever will have. I use it because some of the trackers I use are very fussy about clients and rTorrent is one of the only clients not banned.

I got my WL-500Gp a few months ago and I was initially quite disappointed, the Asus firmware is generally poor especially for torrent downloads. I switched to Oleg's firmware which is much more versatile and powerful but after trying the torrent clients available I was not impressed. So I just kept using my PC.

The beauty of the WL-500 routers however is that people compile additional software packages all the time and make scripts to add features, write tutorials and suchlike. Now I run amule and rTorrent 24/7 and lots of other useful bits of software and I am am very happy with my little router :D

I would recommend buying one if you don't mind tinkering around and learning new stuff. As an out of the box solution however it is not all that.

wirespot
11-03-2007, 15:01
@patrickquek, you're looking for the Holy Grail, like we all do. :) Of course, we all want a small, lightweight client, with a web or ssh interface. That's why many of us bought the Asus in the first place!

transmission and Oleg's additions (transmissiond and transmission.cgi) is usually best because it has a decent combination of features. It has primitive but fully functional queueing, it has a decent (albeit rudimentary) web interface, it has a global bandwidth limit (albeit not tunable nor self-adjustable on the fly). And it doesn't have encryption AFAIK. :(

rtorrent is a superb torrent client, it has encryption, and the lack of web interface can be countered by running it under screen and connecting to it via ssh. Recent versions have added a sort of event handling, so it can watch a directory and automatically start torrents dropped there, it can stop seeding considering your preferences and you can even schedule bandwidth cap adjustments for certain hours of the day. But it doesn't have queueing. :( There seems to be something called "kqueue" in recent versions, I'm trying to investigate what that is but I have a hard time getting it to compile. If it's a functional queue then rtorrent will become the best all-around client, if you can live with the ssh interface instead of Web. And FWIW, a generic API is available, and if anyone cares to ever write a web client for libtorrent instead of rtorrent, it's perfectly possible.

There are other console clients, but they are much simpler, they act like wget: you give them 1 torrent and they work on it. There's no cooperation between clients (for global bandwidth cap for instance), no pretty interface etc. We have vanilla transmission, mainline, ctorrent, enhanced-ctorrent and BitTornado here.

There's torrentflux-b4rt, which is a PHP+sqlite web interface. It's very complex and full featured and can use any choice of transmission, mainline or bittornado as clients. You can even choose a client per torrent. It has a very good interface, with AJAX and everything. It allows you to choose certain files from a torrent. The bittorrent client has encryption. It comes with a hacked transmission client, better suited for control from the Web. (FWIW, Oleg's transmission is also hacked, not vanilla.) But the combination of PHP, sqlite, Python is very heavy and you'll stress your router a lot more than rtorrent or Oleg's transmission. :(

There's also a newcomer called SCTCS (http://wl500g.info/showthread.php?t=8420). It's basically a web interface based on a control API for enhanced-ctorrent. It was developed long ago, but only recently the web interface was fine tuned and made available for the Asus routers. I don't know much about it. From what I've seen it seems like a good choice next to rtorrent and Oleg's transmission (performance-wise), but I don't recall it having encryption. :(

Now, for some performance tips. Always remember that the Asus is a limited power machine (slow CPU, small RAM). Running 4 torrents at the same time is pretty much the upper limit. Oleg's transmission manages it by limiting connections to 60 per torrent and by being a very lightweight client in the first place. rtorrent can achieve the same, but be careful how you set the upper connection limit, and see what performance hit you get for encryption. torrentflux-b4rt & bittornado is a hog and it's best to run just 1-2 torrents at any time, and even so you'll get loads of 1.5 and up compared to 0.5 average for Oleg/rtorrent.

In regard to encryption: there are several encryption modes (http://www.azureuswiki.com/index.php/Message_Stream_Encryption#Implementation_Notes_for _BitTorrent_Clients) available. Clients that support encryption (rtorrent, bittornado) will default to mode 2: try encrypted headers, try plain headers if that fails. Mode 1 is plain transfers only, which is used by the other clients I mentioned. Mode 3 is to use encrypted headers only; teoreticaly bittornado and rtorrent support this, but I haven't figured out how to activate this mode. There's also a mode 4 (full stealth), implemented by Azureus, which in addition to encrypted headers also encrypts the entire transfer stream.

The most efficient to evade ISP throttling is to add as much encryption as possible (a higher mode is better). But as you raise the bar you get fewer and fewer peers and trackers that support the same features, so you lose peers and lose speed. Depending on how bad the throttling is, you may still get a better speed than usual, or not.

patrickquek
11-03-2007, 16:52
@wirespot: Thank you very much for the detailed reply! Very informative indeed and would be a good start for me!

Slight off-topic question:

Should I get the WL-500G Premium or the WL-700gE?

The former retails for SGD188 and the latter is $388, quite a premium for that built-in 160GB harddisk. AFAIK, the 2 are about the same (feature wise) except that the 700gE has 3 USB ports and the HD connects internally via IDE so it has a better transfer rate.

I read somewhere that the 500g Premium is a newer model compared to 700gE... not sure if i remembered correctly?

Thanks!
Patrick

wirespot
11-03-2007, 17:09
I'm not sure that the internal HDD alone is worth it. Indeed, an USB external HDD will have a transfer rate capped to about 2 MB/s (Linux driver limitation, may be fixed in the future), so you'll have to live with that whenever you get your downloads through FTP or Samba. I find it enough for my needs.

Secondly, you may not need 160 GB. I use an old 20 GB I got for a few bucks used, and an external 2.5" USB rack. It's enough for me because I move stuff to my computer once it's done downloading, not leave it on the router HDD. The HDD+rack cost me about 45$, and that's a big difference from what I'd have to pay extra for a 700. But the 700 has a faster CPU I believe, as well as a 3.5" IDE interface with RAID and SMART, and wireless enhancements. You decide.

EddieZ
11-03-2007, 17:36
For BT clients on PC's: Utorrent seems to have that feature.

patrickquek
12-03-2007, 02:15
@wirespot - thanks for the response, its been very helpful.

One other angle I'm unclear about on the 3rd party firmware availability and stability.

Which has more mature, stable and feature rich 3rd party firmware? I know the 700gE is mostly on Oleg, the 500gE on OpenWRT. OpenWRT has only been available on 700gE not too long ago.

Irregardless of which firmware i use, are 3rd party packages like rTorrent interoperable across the different firmware?

The 700gE has 2MB ROM and 64R RAMwhile the 500gP has 4MB ROM and 32MB RAM. Does the 700gE's 2MB limit the available 3rd-party firmwares that can be flashed?

wirespot
12-03-2007, 07:30
It could, but I'm sure that firmware makers take these limits into account.

Firmware is generally not related to extra packages. The firmware offers you a working basic system which covers everything the router is supposed to do (routing, various types of connections, firewall, bandwidth limitation), and a set of essential Linux console tools. The extra packages require an extra HDD (for space), but they can be anything. I believe you can combine firmware from one maker with a package repository from another, save for a few tweaks. So as long as you mount your HDD in the same place on both firmwares, they should work pretty much the same.

Note: of course, I'm assuming you mean machines with a fairly similar architecture. You won't be able to use MIPS packages on an ARM machine, naturally.

patrickquek
12-03-2007, 08:26
Hi wirespot,

Many thanks for your response once again!

cheers,
Patrick

wirespot
12-03-2007, 12:51
Seems like "kqueue" is not what I thought, it's just a different way of handling events. So still no queueing aka "download scheduler" for rtorrent. :(

wirespot
21-03-2007, 10:58
I've done some tests of my own on an ISP that throttles bittorrent. Here's the outline: use rtorrent, set the "encryption" options to full all-around encryption, then install tor and privoxy, start them, and tell rtorrent to use privoxy for the tracker connections. I will describe the entire setup in more detail in my tutorial thread (http://wl500g.info/showthread.php?t=8197).

The results are pretty much what I suspected. If you're lucky and you manage to find enough peers who are willing to establish fully encrypted connections, you'll have evaded the throttling completely. If you don't get enough peers you won't pick up speed even though you won't be throttled.

The only better solution is to establish an encrypted VPN to an outside location and use regular download through that. Or to ditch the throttling ISP, because if they're desperate enough to choke P2P they will try to cut traffic any way they can, even if it means shaping your entire connection indiscriminately.

One possible trick that might work well is to encrypt only your tracker requests. ISP filtering usually works by peeping at your requests to the tracker, where they can see exactly what peers the tracker is offering you. Then it's very easy for them to apply limitations to exactly those IP's. So if you can encrypt that their only choice left would be to examine all connections to see if they are torrent handshakes, which is something extremely costly in terms of performance and it's pretty hard for an ISP to attempt. So you'd have plain peer connections, which would offer you the most available peers, and no easy way for the ISP to detect them, as opposed to seeking only peers who do encryption, which are scarce.

The trouble is encrypting your tracker connections. Unfortunately very few trackers use SSL because it would stress their server. I don't even know if torrent clients are prepared to deal with tracker connections over SSL. One solution is to use privoxy+tor, which are both in the Oleg packages. The Tor network is supposed to encrypt all traffic, but I'm not sure if it encrypts the first step, to the first Tor node; if it doesn't it's useless. I've tried Tor but I got mixed results.

A technique which is definitely successfull is to set up an encrypted TCP tunnel to another machine than the one you have registered with the ISP, and have a HTTP proxy wait there at the other end of the tunnel. Then tell your bittorrent client to use the tunnel as a proxy. stunnel is available as an Oleg package for this. You can also use SSH for this.

A combination of encrypted proxy and full encrypted peer connections will give the best results to evade throttling (but will be limited by the low number of peers using encryption).