data counting WL 500b

[kamilek]

Is it possible to count amount of transfered data (in/out) for each IP? I have searched forum but i have found only quide for statistic of speed. I don´t want to draw charts. I only need to write amount of transfered data to txt file on USB disc. I´m using 1 WAN IP and 4-5 local IP (NAT). I thing that a lot of people are searching for the same, but I´m not able to do it because i don´t understand Linux. Is here somebody who could do it?

Thanks a lot.

[barsju]

It's quite possible. I'm doing a similar thing my self. What you need is some basic knowledge of iptables and shell scripts. What you do is set up a chain in the iptables to count data for each ip in each direction. Then you must write a shell script that can parse results from "iptables -L -vx".

To use awk you must install a full busybox, which is as easy as unzipping the busybox binary you find in the download pages to you usb disk.

For setting up iptables you can use a script like this:

#!/bin/sh
AWK="/tmp/harddisk/busybox/busybox awk"

# create two chains for in and outbound traffic of LAN
iptables -N traffic_out
iptables -N traffic_in

# add rules for both directions for each IP address
# You can also use a simple textfile with ipadresses and use:
# for ip in `cat ipadresses.txt`
for ip in `$AWK '/192/ {print $2}' /etc/ethers`
do
        iptables -A traffic_in -i eth1 -d $ip -j RETURN
        iptables -A traffic_out -o eth1 -s $ip -j RETURN
done

#append general rules to track all traffic from unknown LAN addresses
iptables -A traffic_in -i eth1 -j RETURN
iptables -A traffic_out -o eth1 -j RETURN

# attach chains to FORWARD
iptables -I FORWARD -j traffic_in
iptables -I FORWARD -j traffic_out

For parsing results/making text files you might use something like:

#!/bin/sh
iptables -L traffic_in -vxn | $AWK '/all/ {print strftime("%Y-%m-%d %H:%M", systime()), $9, $2}'  >> /tmp/harddisk/traffic_in.txt
iptables -L traffic_out -vxn | $AWK '/all/ {print strftime("%Y-%m-%d %H:%M", systime()), $9, $2}' >> /tmp/harddisk/traffic_out.txt

Then you can set up a cron job that measures traffic at certain intervalls. To calculate transfered data you can either calculate difference from last measurement or you can zero the counters with

iptables -Z traffic_in
iptables -Z traffic_out

A good tip is to download the graphical ip accounting archive and see the scripts there. Otherwise there are good tutorials on shell scripting, awk and iptables out there (google).

S.

[kamilek]

:-) It looks very easy. But I´m really Linux beginner. I have found Busybox and unzipped on my USB disk (from www.busybox.net). I tried the instruction Make config (as described in file install) but nothing happend. It seems that I haven´t Make instruction on my Asus. And next problem I don´t know how to create scipts you have written in your post. I don´t know how to run cron job. Could you help me,pls?
And last question: what about router reset? When the router starts again will the logging start too? Or I wil have to start it manualy?

K

[barsju]

Ok. You need to get busybox from downloadsdirectory her at chupa.
Go to frontpage (http://wl500.info) direct download-> wl500g-> firmware -> app.
Download and unzip that busybox.

Then read. http://oleg.wl500g.info. It should explain cron, and how to keep settings on reboot. (Look for post-boot/post-mount scripts) And lots more.

Play around with that for a while, and read a tutorial on vi (file editor).

Then tomorrow i will post some scripts for you. no time now..

S.

[barsju]

OK. Here are the scripts I use:
Add iptables script:

#!/bin/sh
AWK="/tmp/harddisk/busybox/busybox awk"

# create two chains for in and outbound traffic of LAN
iptables -N traffic

# add rules for traffic to and from router
if [ "$1" = "" ]; then
        ip=`ifconfig eth1 | $AWK '/inet / {print substr($2, 6, 18)}'`
else
        ip=$1
fi
iptables -A traffic -i eth1 -d $ip -j RETURN
iptables -A traffic -o eth1 -s $ip -j RETURN

# add rules for both directions for each IP address
# You can also use a simple textfile with ipadresses and use:
# for ip in `cat /tmp/harddisk/ipadresses.txt`
for ip in `$AWK '/192/ {print $2}' /etc/ethers`
do
        iptables -A traffic -i eth1 -d $ip -j RETURN
        iptables -A traffic -o eth1 -s $ip -j RETURN
done

#append general rules to track all traffic from unknown LAN addresses
iptables -A traffic -i eth1 -s ! 10.0.0.1 -j RETURN
iptables -A traffic -o eth1 -j RETURN

# attach chains to FORWARD, INPUT and OUTPUT
iptables -I FORWARD -j traffic
iptables -I INPUT 2 -j traffic
iptables -I OUTPUT -j traffic

To calculate traffic:

#!/bin/sh  
AWK="/tmp/harddisk/busybox/busybox awk"
iptables -L traffic -vnx | $AWK '/all/ {if ($7=="*") inn=$2; else print $8, strftime("%Y-%m-%d %H:%M", systime()), "out", $2, "in", inn;}' > /tmp/traffic.dat

This results in a file like this:

*WAN_IP* 2005-04-07 09:36 out 29823 in 17965
192.168.100.100 2005-04-07 09:36 out 0 in 0
192.168.100.101 2005-04-07 09:36 out 0 in 0
192.168.100.102 2005-04-07 09:36 out 0 in 0
192.168.100.103 2005-04-07 09:36 out 0 in 0
192.168.100.104 2005-04-07 09:36 out 79434 in 276350
192.168.100.105 2005-04-07 09:36 out 0 in 0
192.168.100.106 2005-04-07 09:36 out 0 in 0
192.168.100.107 2005-04-07 09:36 out 0 in 0
192.168.100.108 2005-04-07 09:36 out 0 in 0
0.0.0.0/0 2005-04-07 09:36 out 0 in 0

What I do with this file is upload it to another server using curl where it is put in a db. You might want to store it on the usb-disk in some fashion. A tip here is to look at the account_traffic.sh script in the Graphical IP Accounting-scripts. That is where I got my "inspiration".

To make this script run every hour I use this script:

#!/bin/sh
if [ ! -f /var/spool/cron/crontabs/admin ]
then
 mkdir -p /var/spool/cron/crontabs/
fi
echo "0 */1 * * * /tmp/harddisk/account.sh" >> /var/spool/cron/crontabs/admin
crond -L /dev/null

Then finally to start everything on reboot I put these lines in my post-mount script (See http://oleg.wl500g.info for details on post-boot/-firewall/-mount scripts):

. /tmp/harddisk/add_iptables.sh
. /tmp/harddisk/start_cron.sh

And I put this in post-firewall:

if [ -f /tmp/harddisk/add_trackers.sh ] ; then
  . /tmp/harddisk/add_iptables.sh $2  
fi

This is how I do it anyway, but if you look into shell scripting, awk and iptables you can do what you want with it. I am a bit of a Linux-n00b too, but the tutorials you need are out there.

S.

[kamilek]

Thank you very much for your help. I will try your scripts. I tried to run GraphicalIPAcounting from wikipedia and the new version from Mark Sodb but there are missing some scripts. So I haven´t been successful. :-( .

[barsju]

What scripts are missing? I managed to get the wiki-scripts working, but also had problems with mark's scripts.

S.

[kamilek]

In script traf.sh there are 2 files to be run: traffic_out.sh and traffic_in.sh , but I can´t find them anywhere.

[kamilek]

And in Trafic_day.sh is link to traffic.sh which isn´t there too.

[kamilek]

So I started accounting from SODB (from wikipedia) but there is some problem. Data aren't counted right. With command ifconfig I see that on br0 is transfered 2322497 bytes in and 61500 bytes out (all from one IP, the other PC's are off). But with command iptables -L traffic_in -vx is only 507 bytes transfered and traffic_out is 0 bytes transfered.

[barsju]

Post your the results from iptables -L, and I'll take a look.
But I think that br0 is the LAN interface, so it counts all traffic between PC's in the LAN. eth1, however, is the external interface (WAN) which is what is used for accounting.

But post your iptables and i'll have a look.

S.

[kamilek]

Here you are my files. When I´m using br0 interface, is it OK. With eth1 interface there are no data counted . But with command ifconfig I see data on eth1 too. I tried to copy about 100 MB between two PC's and it didn´t have any effect on any interface . So I think that I can use br0 without problems (maybe printing on USB printer connected to router will generate some traffic).
I had adapted the ipaccounting script to save files to USB stick. I have created post-boot script, but I think that while loop inside it isn't good idea. But I don't know how to ensure that commands inside post-boot will run after the USB stick is mounted. With 30 s loop I have some problem with synchronizing time with NTP server.