Bekijk de volledige versie : Graphical IP accounting
For those who are interesed I created some scripts to do graphical IP accounting on the ASUS. A description and some installation details are on the WikiPage:
GraphicalIpAccounting (http://wiki.wl500g.info/index.php/GraphicalIpAccounting)
Tarballs are attached!
Hope you like it :)
*cough* *cough*
I'm sober now...
Mark Koops
24-10-2004, 13:29
Good Work, sodb!
Your scripts inspired me to start exploring the world of bash scripts/awk/iptables/etc. And the working model was of great help to start understanding how to deal with these progs.
I found it a nice exercise to adapt the scripts to extend their functionality.
v2 of tarballs are attached.
changes:
all usage data is stored in relative usage counters instead of absolute iptables counters
router: counts for in/outbound traffic of the router itself (ftp/http) to the WAN
other: counts for those LAN ip addresses that are not monitored individually (in my case 192.168.1.17-192.168.1.254)
introduced 2 new "ip addresses" in the usage logging:
store all data in a data folder that is set in the init.sh script
changed structure of data files:
per ip address:
hourly usage of today in the file <ip adress>.day.dat
hourly usage of a earlier day in a file like <ip adress>.20041012.dat
daily total usage of this and earlier month in a file like <ip adress>.200410.dat
totals for all completes months up to now in the file <ip adress>.history.dat
over all ip addresses:
total usage of today up till now per ip address in the file all.day.dat
total usage of completed days of this month per ip address in the file all.month.dat
all history data is being kept and can be shown by web page
all accounting scripts combined to 1: account_traffic.sh:
script runs every hour
script will detect and act to day changes (at midnight)
script will detect and act to month changes (at midnight, 1st day of month)
script can be run with parameter "init", "daychange", "mothchange" to force a init of iptable counters, day change or moth change resp.
changed web pages:
combined all web pages into 1 --> traffic.cgi
all presentations done by graphs (no more select boxes etc)
the bars in the graphs can be clicked to zoom in on details
my scripts depend on awk being found by the system, which you can arrange by the command:
ln -s <your path to busybox full version> /bin/awk
Hope you like it and I am waiting for some comments of you.
Mark
Mark Koops
24-10-2004, 14:15
found some small mistakes in the tar files so i updated them in my original posting
xlephant
16-11-2004, 16:47
@mark.
can i also use the howto from sodb for install?
is there something different if i want to use graphical ip accounting with fw 1.8.xx?
thx in advance...
xelephant,
you certainly can use it with new firmware. Change the mount point in the init.sh script. The wiki page is not describing the scripts that Mark wrote. I installed Mark's scripts with some small changes (e.g. the root crond file is lacking in the tar ball). Using the post of Mark I will update the wiki page so that it describes the latest version,
regards
nunosilva
23-11-2004, 22:26
Hi
I've a Asus WL-500g at home to distribute internet access for 4 computers, 3 Windows PC and 1 Macintosh. We are all students living in the same house.
So here's what I'm searching: I would like to have, at the end of the month, a graphic showing the percentage of every computer's downloads so that each one could pay the ammount according to the internet's use. I'm really new to all this and none of us here at home are familiar with coding.
Is "Graphical IP accounting" is suitable to do what I want? If so, there is any way to run a script automatically without having to type everything by hand?
Thank you in advance and forgive my ignorance
Best Regards
Nuno
I canīt find the installation manual for Mark's ipaccounting. I think that quide which is described on http://wiki.wl500g.info/index.php/GraphicalIpAccounting is still showing installation process for old scripts, isnīt it? Where could I find new installation quide? Iīm beginner in Linux.
- after router restarting is it working again or I will have to run it manualy?
Thanks a lot.
Kamil.
xelephant,
you certainly can use it with new firmware. Change the mount point in the init.sh script. The wiki page is not describing the scripts that Mark wrote. I installed Mark's scripts with some small changes (e.g. the root crond file is lacking in the tar ball). Using the post of Mark I will update the wiki page so that it describes the latest version,
regards
I canīt find installation guide for Mark's scripts. Do anybody have it? I think that the guide on wiki pages is still for older ipaccounting scripts.
What will happend after router restarting? Will I have to start scrips manualy again? And how to dissable scripting? I would like to print via USB port sometimes.
K
What will happend after router restarting? Will I have to start scrips manualy again?
That is what post-boot/-firewall/-mount is for.
And how to dissable scripting? I would like to print via USB port sometimes.
Once you unplug your usb stick, anything on /tmp/harddisk/ is unaccesible and scripts will fail, but no harm is done. If you like you can just stop the crond (killall crond) and then restart it when you plug your stick back in. But the easiest of course is to buy an usb-hub. Then you can have everything pluged in all the time! Not very expensive either.
S.
found some small mistakes in the tar files so i updated them in my original posting
Hello Mark,
I need you help regarding to Graphical IP Accounting V2. I would like to use your vesrion. For installing I used the HOWTO from SODB, but in your tarball I couldnot find the root for crond. I used from V1.
In our wifi site we used Ips with step 10, as 10, 20, 30, 40, 50. In a case of INIT.SH setting I have to use LOWERIP=10 UPPERIP=50, but it takes very long time for ASUS to calculate and to prepare a dat files .. around 70minutes! And the generated HTML pages is very big ... a lot of unused IPs beetween 10-20-30-40-50. There is some possibilities or setting how to meka it little bit faster and how I can select just a used IP-s to HTML generating and accounting?
Thank you
Wizi
Hello,
Mea Culpa .... my fault .... WORKING FINE !!!!!
.............. just found some features, which doesnot works :(
... to get the traffic to the web I had to use eth0
... but still the field OTHERS and ROUTER doesnot calculate the traffic - but the router_in and router_out chains contains data !
... on the web are still missing the HISTORYCAL data, but in history folder the data are ....
can somebody help me?
Thanks a lot, WiZiPoK
Hi together,
I am using Marks script, but traffic is not counted. I think the cause are wrong values in the init.sh. Could someone tell me please what to type in? Here is some inormation about my config.
My router has LAN adress 192.168.1.1
I do not have a fixed WAN adress
I have clients in the range from 192.168.1.50 to 192.168.1.61
Here is init.sh:
#directory where accounting scripts reside
accountdir=/tmp/harddisk/accounting
datadir=/tmp/harddisk/accounting/data
# pointer to AWK applet
AWK="/bin/busybox awk"
LAN="192.168.1.1"
WAN="???.???.???.???"
#ipprefix consists of the first 3 number of your
#LAN side ip addres. Together with a 4th number in the
#range lowerip to upperip, it makes the range of ip addresses
#to be monitored in your LAN.
#
#warning: upperip relates to LANMask as:
#ipprefix.upperip is the highest address that fits in ipprefix.0/LANMask
ipprefix="192.168.1"
let lowerip=50
let upperip=61
LANMask=???
What should I use as WAN address?
What is a LANMask and which value should I use?
Thanks for your help
Thomas
Use ifconfig (ifconfig eth1 | grep inet) to find WAN address, and LAN mask should be 255.255.255.0
B.
I know my actuall Ipadress but is changes every day. So I think i were not very useful to change it every day manually.
255.255.255.0 is the subnet mask. But LANMask seems to be something different. And the example show a value of 28. But what should that be?
Examplescript:
#directory where accounting scripts reside
accountdir=/mnt/usbfs/accounting
datadir=/accountingdata
# pointer to AWK applet
AWK="awk"
LAN="<LAN side ip address of your router>"
WAN="<WAN side ip address of your router>"
#ipprefix consists of the first 3 number of your
#LAN side ip addres. Together with a 4th number in the
#range lowerip to upperip, it makes the range of ip addresses
#to be monitored in your LAN.
#
#warning: upperip relates to LANMask as:
#ipprefix.upperip is the highest address that fits in ipprefix.0/LANMask
ipprefix="192.168.1"
let lowerip=2
let upperip=16
LANMask=28
Well 24 is just another way of saying 255.255.255.0 (32 bits where the 24 first are 1 others are 0) so 28 would translate to 255.255.255.255.240 which means only the 16 first adresses are valid..
You should propably just set it to 24.
B
Hi,
for me the Mark's script starts to calculate when I changed the ETH1 to ETH0 inside add_iptables.sh script, just try it ...
But I couldnot discover why the ROUTER and OTHER strings not contains any traffic :(
WiZiPoK
But I couldnot discover why the ROUTER and OTHER strings not contains any traffic :(
WiZiPoK
I have the same problem :-(
:confused:
Its still not working. dat files are created but they contain value 0 only :
06/06/05 23:00 out in
06/06/05 23:13 out in
Could someone post the files "add_iptables.sh" and "init.sh" forWL500gx from a installation that is logging traffic please?
Thank you so much.
Thomas
Hi, here is my init.sh script
#directory where accounting scripts reside
accountdir=/opt/accounting
datadir=/opt/accounting/data
#pointer to AWK applet
AWK="awk"
LAN="192.168.1.0"
WAN="192.168.0.2"
#ipprefix consists of the first 3 number of your
#LAN side ip addres. Together with a 4th number in the
#range lowerip to upperip, it makes the range of ip addresses
#to be monitored in your LAN.
#
#warning: upperip relates to LANMask as:
#ipprefix.upperip is the highest address that fits in ipprefix.0/LANMask
ipprefix="192.168.1"
let lowerip=9
let upperip=12
LANMask=28
minute=`date +"%M"`
hour=`date +"%H"`
year=`date +"%Y"`
month=`date +"%m"`
day=`date +"%d"`
yday=$day
at ADD_IPTABLES everywhere were is ETH1 I changed to ETH0
....but it is unbelievable, TODAY 08.june at 00:00 my INIT SCRIPT stop with ERROR. I dont understand why ... When I set the DATE back to 07.june 23:50 the INIT.SH script run without error, after mindnight stopd with failure.
the WRONG SINTAX WAS :
yday=$day-1
after removing "LET" and "-1" everything is working OK .... but the :"yesterday" day doesn.t set correctly :(
and everything is runnig FASTER.
I realy don't understand this DATE issue :((
WiZiPoK
to wizipok: and why did you change eht1 to eth0?
to wizipok: and why did you change eht1 to eth0?
just to start to count :)
I solved the probs around the OTHERS and ROUTER counting ... There was a differenet string compare inside the "iptables -L -vx" and account_traffic.sh
There was enough to change the sring to "anywhere" :)
WiZIPoK
A while ago I did some fiddeling with these scripts my self and made them work ok. But in the end I made a new, easier and better solution that worked for me. Basically I just calculate usage each our and then upload the data to a web server I have runnig with a php script that stores the values in a mysql db. Then I have a php page there that shows my statistics. But for those who doesn't have a php server lying around that might not work.
But anyways, I said I made the Gr. accountings scripts kinda work before I made my own so I figured I could post them here. You can have a look, but I can't give you any garanties that they'll work for you, as I did a bit of tinkering with them..
The important files to look at and adapt are:
init.sh, ipaddresses.txt, mkcron, root
And the ones that does the work are:
add-iptables, and account_traffic.sh
Maybe it'll be a help to someone..
B.
just to start to count :)
I solved the probs around the OTHERS and ROUTER counting ... There was a differenet string compare inside the "iptables -L -vx" and account_traffic.sh
There was enough to change the sring to "anywhere" :)
WiZIPoK
So what do I have to change in account_traffic.sh?
So the line "let yday=$day-1" does problem only 8. and 9. day in each month. The other days are OK. Rewriting it to "yday=$day" is not solution because of problems with history data. What to do with it?
ah simple arithmetic like that doesn'e work in shell script.
try:
$yday=`expr $day - 1`
And take a look at a scripting tutorial:
ex: http://www.freeos.com/guides/lsst/ch02sec07.html
Hi Barsju,
yes, I used very close script to yours. In some scripts on 500gx I have to replace $WAN with the realy IP with numbers .. After is Everything start to work ...
What kind of improvement did you done in your script?
The one think what is very strange, inside the code (init.sh)
minute=`date +"%M"`
hour=`date +"%H"`
year=`date +"%Y"`
month=`date +"%m"`
day=`date +"%d"`
let yday=$day-1
the last rows " let yday=$day-1 " doesnot work ..... or its works DEPENDING ON ACTUAL DATE .....
As KAMILEK wrote, doesnot wor on 8th and 9th day in month :) Why?
WiZiPoK
ah simple arithmetic like that doesn'e work in shell script.
try:
$yday=`expr $day - 1`
And take a look at a scripting tutorial:
ex: http://www.freeos.com/guides/lsst/ch02sec07.html
You are RIGHT !
yes, the expression :
yday=`expr $day - 1`
inside init.sh works fine ..
WiZiPoK
In your scripts in /www is absent file traffic_curr.cgi.
J donīt know, when I can run script account_traffic_bckup.sh, account_traffic_current_hour.sh.
What is your crontabs file look like?
Can you send me full script?
Oh. You don't really need those, and I guess I should not have included them. The current hour is to show traffic for current hour. (Normally you only calculate traffic for completed hours.) But I'll include the traffic_curr.cgi for completeness.
As for the crontab, it is in the root file. Take a look there. And mkcron..
S.
Then I have a php page there that shows my statistics. But for those who doesn't have a php server lying around that might not work.
Any way that you can make this public ?
Maybe the empty sql db and the needed php ?
Regards
Hallo all :)
is this allstats working on a gx with wl500g-1.9.2.7-5a ?
if i start account_traffic.sh me got a error from sed
# ./account_traffic.sh
setting/updating usage counter files
sed: bad format in substitution expression
sed: bad format in substitution expression
sed: bad format in substitution expression
sed: bad format in substitution expression
sedsed: bad format in substitution expression
: bad format in substitution expression
append usage to day dat files
tailing counter files
finished.
and the cgi display a "Segmentation fault"
Anyone with the same error and had fixed this ?
Best Regards
found the "Segmentation fault" error but have one more q. :)
Is there some tuning that not only IE display the graph correct ?
Me have some prob with Firefox.
Best Regards
[admin@WL500g accounting]$ ./add_iptables.sh
./add_iptables.sh: 3: awk: not found
[admin@WL500g accounting]$
Plese help
[admin@WL500g accounting]$ ./add_iptables.sh
./add_iptables.sh: 3: awk: not found
[admin@WL500g accounting]$
Plese help
Upgrade to latest firmware.
Upgrade to latest firmware.
I have in WL500g firmvare version 1.9.2.7-4. This wersion it not supports?
:confused: Now notification:
[admin@WL500g accounting]$ ./add_iptables.sh
./add_iptables.sh: 3: /tmp/harddisk/accounting: Permission denied
I have in WL500g firmvare version 1.9.2.7-4. This wersion it not supports?
Upgrade to 1.9.2.7-6b.
Upgrade to 1.9.2.7-6b.
With firmware 1.9.2.7-6b:
[admin@WL500g accounting]$ ls
account_traffic.sh
graph_day.awk
mkcron.sh
accounting_v2.tar
graph_month.awk
monthtot.sh
add_iptables.sh
httpd.conf
rem_iptables.sh
daytot.sh
init.sh
usage.sh
graph_all.awk
ipaddress.txt
[admin@WL500g accounting]$ ./add_iptables.sh
./add_iptables.sh: 3: /tmp/harddisk/accounting:Permission denied
where's AWK applet? In file "init.sh" is line with text:
# pointer to AWK applet
AWK="AWK"
I Attache main files for verification.
Just remove the following lines from add_iptables.sh, because they are not used:
# ipnumbers of PC's in LAN
ipnumbers=`cat $accountdir/ipaddress.txt| $AWK '{print $2 }'`
Have you installed the scripts in the /tmp/harddisk/accounting directory? If not, change installation directory in init.sh accordingly.
BTW: I am working at a graphical version based on rddtools. It generates nice usage graphs in gif format. Will post things soon!
Ahh, thats cool if you can make one with rddtools :))
Regards
There is allready something for openwrt and rrd
http://wiki.freifunk-leipzig.public-ip.org/index.php/LinksysNetzwerkStatisik
I tried to install these packages. Everything works fine, except the libpng or other graphical libraries are not included. Any feature leading to real graphics causes a 'segmentation fault'. Therefore, I compiled the rrdtool myself and used static linking to the libraries. Bin's included.
Later I will post some scripts to log traffic to different pc's in lan (based on add_iptables.sh). The graphs are generated on the asus and then I FTP the resulting graphs and databases regularly to a webserver with a proper access to the internet.
I want to use this feature, too.
But what am I have to do to install this?
Should I use the wiki-instructions?
sodb, is your tool ready to use?
(How to use this?)
I want to use this feature, too.
But what am I have to do to install this?
Should I use the wiki-instructions?
sodb, is your tool ready to use?
(How to use this?)
better is using RRD tools from www.macsat.com . Look at http://www.home.karneval.cz/0220603501/zdenek . There is showed what can you do do with rrd tool. But it is only in Czech language. But you will understand it. Maybe :-)
Excelent work, this is what I need.
Can you post more details of how do you do this? Scripts. Do you change iptables?
Hi,
please KAMILEK, pleaseee .. I would like to have the same possibilities to put the individual IPs into the graph .... As a normal RRDToll with the WAN/LAN/WLAN accounting working fine, but yours are BETTER :)
Prosim, hodne prosim :)
WiZi from CB
Probably the same as KAMILEK produced, here are my scripts.
Using the older scripts of my earlier html-based graphical IP accounting, together with the improvements Mark Koops made, and with the HTML layout found at
http://www.home.karneval.cz/0220603501/zdenek I constructed some scripts to do graphical IP accounting using RRDtools.
Installation of RRDtools are discussed earlier in this topic. I include my own compiled version in the tar ball.
I use the following method: iptables collects traffic in up/down directions for individual PC's in LAN. Ifconfig is used for counting bytes transfered over the wan interface. Because of the frequent updates of counters (e.g. every 5 min), the RRD files are stored in RAM (/tmp/rrd). USB flash would wear out quite soon (10000 cycles are reached in 34 Months). Graphs are generates every single hour and also stored in RAM. Then, and that is a step that you might want to change, the graphs are uploaded to an external web server using ncftpput util (also included).
To initalize the logging proces you have to do the following: 1) edit ipaddress.txt file to meet your requirements, 2) edit init.sh to change $wanint WAN port and lanmask, and directories, 3) add admin entries to crontab and see to it that crond is executed from post-boot. 4) execute add_iptables.sh (can also be put in post-boot), 5) initalise RRD's (might not be necessary of stored on HD or USB flash, 6) initiate HTML files and copy all *.html files to path or server served by webserver.
The following files are included (untar with tar -xzvf rrd_iptables_tar.tgz, and then again tar -xvf rrd_iptables.tar) (edit) added correctversion of update_traf.sh:
ipaddress.txt
A two column file with the PC name and the PC address for the PC's to be minitored in the LAN
init.sh
Used for setting pathes, WAN port, LAN subnet and a mask that defines all the PC's addresses in the LAN, used with negation for determining 'other' traffic. See earlier in Mark Koops' description.
add_iptables.sh
Generates the correct entries in iptables based on ipaddress.txt file.
gen_html.sh, gen_menu_html.awk, gen_graph_html.awk
Generates menu.html, and the individual html code for each of the entries in the menu.
footer.html, title.html, traf.html
HTML templates for traffic page.
init_traf.sh
Generates initial Round Robin Databases for each of the entries in ipaddress.txt and other, $wanint and router entries.
admin
Example crontab file. To be copied in /var/spool/cron/crontabs.
update_traf.sh (use seperate file)
Script used to update counters of iptables entries in the specific RRD database. Is called from crontab.
plot_traf.sh, plot_single.sh
Script called from crontab to generate png graphic files from rrd databases.
Hope this helps you installing the scripts. It is not very fancy, so you should adopt things to your personal needs.
Hi your Majesty ...
can you please put the missed PLOTPING.SH ? please :)
WiZi
OK. As soon as possible I will give here my rrd scripts. But I have them in Czech language so I will try to translate them to English.
Wizipok if you would like to have it in Czech, write me to kamil.krpec@seznam.cz a domluvime se. :-)
Hi ,
I don't exactly know why, but I cannot count the non registered IP's traffic. I already tried and tuned 3 different typas of accounting scripts, but the unregistered IP's traffic everything was gone ..
If I know the IP's I can count them, OK ..lets there are from 1-10. Using mask /24, that means that the "other" traffic is the:
All traffic - known traffic ==>> WANADDRESS/24 - SUM(traffic from IPs 1-10) = OTHERS
But from Iptables the string cutting doesn't works ... in a case when I want to get the traffic for "192.168.0.1/24"
any suggests?
WiZi
Hello WiziPok,
I don't quite understand what you mean by non registered IP traffic. I suppose you mean all traffic from and towards IP addresses in your LAN that are not fixed addresses or served with predifined DHCP IP addresses. Let's say you have 5 known PC's in the LAN and you give them IP addresses using the Ethernet based DHCP addresses 192.168.1.2-192.168.1.6 in a /24 LAN subnet. The routers own address in the LAN is 192.168.1.1/24. Rules for all traffic except that towards the router itself and the PC's can be defined by using a subnet in which as closely as possible fit all the predefined IP addresses. In this example that would be a 192.168.1.0/29 subnet (containing the addresses .1 through .7, the last being the broadcast address within the subnet. If you want all the traffic outside this subnet you can use the following iptable rules:
iptables -A traffic_in -i $wanint -d ! 192.168.1.0/29
iptables -A traffic_out -o $wanint -s ! 192.168.1.0/29
with $wanint your wan interface e.g. ppp0.
Regards,
Stefan
Ok. But here is the description of my situation:
LAN = 192.168.1.1
DHCP server for 192.168.1.2-16
MASK 255.255.255.0
FIXED IPs are 2-3-4-5-6
Your and other's script presenting the calulated traffic for the FIXED IPs. Other traffic from IPs served by DHCP or IPs set manually aren't presented on the graph. How can I calculate this traffic and put to the TRAFFIC FROM UNREGISTERED/UNKNOWN IPs?
Another question ...
the comamnd to get AWK-ed string from iptables doesn't works for me ... never cut any traffic from "iptables -L -v" however there is ..
thank you
You could add the IP addresses 7-16 in the ipaddress.txt file. Altough you don't know who is connected, the traffic gets sorted to the specific IP addresses. You need individual entries in iptable to differentiatie between traffic of each of the IP addresses. You could use the file /tmp/dnsmasq.log to search for the active PC's given an IP address using DHCP.
Tp get iptable output using awk test the following:
iptables -L -vx | awk '{print $2}'
When this works you can use sed or awk to filter the correct line and get the counters from this output. In my script I first store the iptables output in a temporary file and process that file later.
cooper.zl
19-10-2005, 11:34
I have one qustion about RRD Tool, CRON scheduled task and 128MB USB flash disk in WL-500g. Every flash disk have finite write cycles and after installing RRDTool and CRON on 1st partition with 2nd partition used as swap, then LED diode blinks quite often (I think 1 minute and then 4 minutes not). My question is if statistic for RRDTool is big file or not, if USB flash disk will be destructed after 6 months or not - I don't know why this flash disk blinks so much.
Thx for your answers.
Thank You sobd for the manual. I wollw Your instructionas and almost got everything working...except the *.gif files looks little strange...I checket the scripts for generating charts - they're running without errors. Any idea whats wrong with my installation? The trafic data seems to be logged correctly:
root@OpenWrt:/opt/usr/bin/iprrd# ./update_traf.sh
updating usage counters
IP Bytes In Bytes Out
192.168.150.2: 0, 0 ---;
eth0: 1380861, 1330602 ---;
router: 44708, 30401 ---;
other: 0, 0 ---;
Thanks
Kamilek sent me this skript http://www.home.karne...0603501/zdenek
and i translete it. when anybody have interest i uploade it to this forum
sorry here is the link
http://www.home.karneval.cz/0220603501/zdenek
alien433
26-10-2005, 20:24
can this tool already be implemented in the firmware?
because i am an beginner in this.
this is very easy to install, but now kamilek remake the script, when finished, i meke a litle how-to
but first you install from www.macsat.com this:
ipkg tutorial
php-thttpd tutorial
cron tutorial
Do somebody know which command should I use to make txt file which will contains 2 lines?
Something like this:
in 1
out 2
Thx.
Hi
this one should work:
echo X Y Z > file.txt - for creating and rewriting an existing file
echo X Y Z >> file.txt - for appending the lines into the existing file
for e.g.
echo "nazdarek" > /tmp/greetings.txt
echo "how are you?" >>/tmp/greetings.txt
WiZi
Thx Wizipok,
I know this command, but I donīt know how to read each line sepately. I tried to use AWK but this command read all lines together. Maybe any option in this command?
p.s. Jinak uz mam celkem slusne predelane ty skripty. Az to bude cele, ozvu se.
oversc0re
09-11-2005, 12:14
I'm slowly giving up on this installation, so I thought maybe it's time I ask for your help.
Everything works fine but the counting. I'm using Mark's v2 and the problem is:
1) iptables doesn't count PC traffic
Chain traffic_in (1 references)
pkts bytes target prot opt in out source destination
0 0 all -- eth0 any anywhere 192.168.0.10
0 0 all -- eth1 any anywhere 192.168.0.10
I was so desperate that I added both eth0 and eth1, but still nothing. It's the same with traffic_out. What should I check here?
2) Router traffic is counted, but isn't transfered to the log files:
iptables gives
Chain router_in (1 references)
pkts bytes target prot opt in out source destination
1109 146435 all -- any any anywhere anywhere
and accounting script searches for
bytesin=`iptables -L router_in -vx | awk '$8 ~ /'"$WAN"'$/ {print $2 }'`
$WAN which is not given in the iptables result. I know how to deal with thi problem but i'm confused why does it happen. I guess there must be a problem with iptables entries but I didn't modify anything. Could someone please help!
oversc0re
10-11-2005, 11:01
I just can't view ETH0 or ETH1 traffic. I created a test chain, inserted all interesting devices and attached it to FORWARD.
iptables -L itraffic -vx
Chain itraffic (1 references)
pkts bytes target prot opt in out source destination
2154 235103 all -- br0 any anywhere anywhere
1037 674518 all -- any br0 anywhere anywhere
1583 729608 all -- ppp0 any anywhere anywhere
1020 156826 all -- any ppp0 anywhere anywhere
0 0 all -- eth0 any anywhere anywhere
0 0 all -- any eth0 anywhere anywhere
0 0 all -- eth1 any anywhere anywhere
0 0 all -- any eth1 anywhere anywhere
ETH's are 0 all the time regardless of internal or external traffic. But when i check ifconfig ETH0 shows 400MB of Tx and 120MB of Rx traffic. Beats me :(
oversc0re
15-11-2005, 15:04
I'm having a monologue here... but maybe one day... :) I used ppp0 interface, and I got everything working now ... even the rrd graphs and stuff. But every now and then the iptables entries get deleted and the traffic isn't counted anymore. Looks like the router refreshes the entries and my custom ones get deleted. Is there a way to solve this without formating my USB flash to ext3?
tnx for your answers.
Had the same problem. Problem can be solved by putting iptables initialization script (add_iptables.sh) to the post-firewall script in /usr/local/sbin and activating with flashfs save / flashfs commit.
oversc0re
16-11-2005, 11:16
Hey ... thanx for the answer ... I was just about to delete my post because I found that solution on the forum 5 mins ago :)
This is a bit offtopic but anyway... I had to add the ip's manually to the post-firewall, because the usb flash device isn't mounted at the time. I also created post-mount script where I start httpd and init the rrd. The post-mount executes, but still too soon :confused: I can't use the USB flash for 2 or 3 minutes after the router wakes up. It is mounted but unreadable. Any ideas? (I really don't wanna format it to ext3)
Log states:
Nov 16 11:42:41 kernel: SCSI disk error : host 0 channel 0 id 0 lun 1 return code = 70000
Nov 16 11:42:41 kernel: I/O error: dev 08:11, sector 2
Nov 16 11:42:41 kernel: EXT3-fs: unable to read superblock
Nov 16 11:42:41 kernel: MSDOS FS: Using codepage 950
Nov 16 11:42:41 kernel: MSDOS FS: IO charset cp950
Nov 16 11:43:11 kernel: SCSI disk error : host 0 channel 0 id 0 lun 1 return code = 70000
Nov 16 11:43:11 kernel: I/O error: dev 08:11, sector 0
Nov 16 11:43:11 kernel: FAT: unable to read boot sector
Nov 16 11:43:11 kernel: FAT: freeing iocharset=cp950
Nov 16 11:43:41 kernel: SCSI disk error : host 0 channel 0 id 0 lun 1 return code = 70000
Nov 16 11:43:41 kernel: I/O error: dev 08:11, sector 0
Nov 16 11:43:41 kernel: FAT: unable to read boot sector
Nov 16 11:44:11 kernel: SCSI disk error : host 0 channel 0 id 0 lun 1 return code = 70000
Nov 16 11:44:11 kernel: I/O error: dev 08:11, sector 0
Nov 16 11:44:11 kernel: NTFS: Reading super block failed
Nov 16 11:44:41 kernel: SCSI disk error : host 0 channel 0 id 0 lun 1 return code = 70000
Nov 16 11:44:41 kernel: I/O error: dev 08:12, sector 2
Nov 16 11:44:41 kernel: EXT3-fs: unable to read superblock
Nov 16 11:44:41 kernel: MSDOS FS: Using codepage 950
Nov 16 11:44:41 kernel: MSDOS FS: IO charset cp950
Nov 16 11:45:11 kernel: SCSI disk error : host 0 channel 0 id 0 lun 1 return code = 70000
Nov 16 11:45:11 kernel: I/O error: dev 08:12, sector 0
Nov 16 11:45:11 kernel: FAT: unable to read boot sector
Nov 16 11:45:11 kernel: FAT: freeing iocharset=cp950
Nov 16 11:45:41 kernel: SCSI disk error : host 0 channel 0 id 0 lun 1 return code = 70000
Nov 16 11:45:41 kernel: I/O error: dev 08:12, sector 0
Nov 16 11:45:41 kernel: FAT: unable to read boot sector
Nov 16 11:46:11 kernel: SCSI disk error : host 0 channel 0 id 0 lun 1 return code = 70000
Nov 16 11:46:11 kernel: I/O error: dev 08:12, sector 0
Nov 16 11:46:11 kernel: NTFS: Reading super block failed
Nov 16 11:46:11 kernel: VFS: Can't find ext3 filesystem on dev sd(8,20).
Nov 16 11:46:11 kernel: MSDOS FS: Using codepage 950
Nov 16 11:46:11 kernel: MSDOS FS: IO charset cp950
Format does not help.
YAEGER:
Hi,
I read in this thread that youīll gonna write a nice how-to on this installation - have you done something like that?
Iīd be really interested - right now I have sucsessfully followed macsats tutorials on: ipkg, crond, thttpd
Earlier I also had the macsat version of the RRDTool IP Accounting running.
But with the limitation that it showed only the general traffic and not per IP address.
Thanks for any further hints,
greetings from austria,
schim
oversc0re
07-03-2006, 20:31
Well... I don't know about the howto, but if you read the whole thread and experiment a bit, you'll be able to get it working. If you get stuck somewhere, I can help. There are basically two different versions of the accounting described in this thread (none of them really worked for me, so I have written another combined from the two available). I can give you mine but it's totaly undocumented so maybe you should try a documented version first :rolleyes:
Greetings from over the border ;) (Slovenia)
over.
Hi oversc0re,
thanks for reply - I just tried the installation and it worked.
Everything looks fine except the fact that I canīt find the GIFs!
I found out that the plotping.sh file is missing - should this script generate the gifs?
Please upload this script if itīs of any need.
Thanks,
schim
EDIT: OK - I found it out myself: the orig. script plot_single.sh was written to send the gifs after creation via ftp to an external server and then DELETE the files in RAM - thatīs why I didnīt find them :-)
oversc0re
08-03-2006, 18:16
I don't know about the missing file in your version, i can just give you my files. As I've mentioned earlier this version is not compatible with any other version because it is mostly rewritten from scratch. I kept a few elements from other scripts (the web bar for total download with a totaly rewriten data feed) for which all the credits go to their authors.
If you want to see what you are about to install, click here (http://freeweb.siol.net/hebo/images/screenshot.jpg)
I have packed my whole /tmp/harddisk directory into the file attached. If you extract it back there, you shouldn't have problems with the paths.
Installation:
Extract zip to /tmp/harddisk
Edit the /accounting/ipaddress.txt file and enter your addresses. Addresses that are not entered are monitored under total traffic, but do not get a separate trace.
Edit /accounting/init.sh and check the ip's the ip prefix etc... Also check the paths which should be ok.
Search through all the files and look for word oversc0re. Change it with your router username.
Add iptables entries by running accounting/add_iptables.sh
Run everything else by accounting/run.sh
Now you have to wait for 10 minutes for first results. The results can be seen at http://router_ip:81/
If you get the script working, you can run the nvram_install.sh and it will add itself to post-firewall and post-mount and will be started at the router startup.
Note: in order to spare the USB stick (too frequent writes can damage the stick) the script copies all the vital data to ram and works from there. The data is backed up every night at 3:00 AM from ram back to usb stick. If you restart the router without running the ./backup.sh, the data from prevous backup until restart will be lost.
The traffic graphs are created at real time (on request) so we don't use the CPU time when it's not needed and that's why it takes a while for the graphs to show.
I'll stop now... If I missed something, just ask.
Since my attachment is just over 1MB i have to put it to an external location (http://freeweb.siol.net/hebo/files/ip_acc.zip)
Hi again,
impressive work! Thank you for the extra afford you made - I hope that other readers of this forum will also get a benefit from this.
At the moment I made the first modifications on sodb version by myself and one of the first things I missed was the total up/download in numbers below the graphs.
The other idea I had was: letīs just create the graphs if they are needed - because the routerīs CPU isnīt that fast.
In the past I had problems when streaming video over WLAN with the router running www.macsat.comīs Version of the IP Accounting. It ran every 15mins and drew the graphs which almost took 2mins with 100% CPU load - during this 2 mins the streaming video was rahter stuttering. It took me a while until I realised that the IP Accounting tool was the reason for this ;-)
Again many thanks - Iīll give your luxurous version a try and let you know what I think.
greetings,
schim
oversc0re
08-03-2006, 19:37
Well, I hope you make it work. There's one more thing... In the colors.txt file there are colors for traces on the graphs. I think it is ok, if there are more colors than traces, just make sure that there are not less colors.
Hope you like it.
greetz, over.
hi again,
I took a closer look to your version and could learn much of it (I´m new to linux and web programming).
I´d like to add one little gimmick to my version:
a simple button or even a hyperlink in my html file which executes a .sh script when you click on it.
After a lot of webresearch I´ll give up at this point. I don´t want to study this for too long - I just want generate the graphics when I click somewhere in the webinterface - shouldnt be to hard, or?
My environment:
using the thttpd webserver at port 81 with cgi directory - cgi should be the thing to exec .sh script or?
Do I have to start thttpd webserver with some specific options in order to use cgi?
thanks,
schim
oversc0re
09-03-2006, 18:06
You can put a simple hyperlink to a .cgi file and from .cgi file you run the .sh script. Good enough?
Did you maybe try to install my version? Because I think I have forgotten to include some files needed for the web interface :) I have updated the file at original location. Now it contains a rrddata directory with web interface files included. Be sure to update the /accounting/init_rrd.sh because it is updated. Other files are left unchanged.
greetz, over.
audiophil
01-06-2006, 12:50
I just can't get it to gather and plot data for my users. Only router data...
I'm running the router without any username/password on the ADSL.
3 fixed users, 192.168.1.2 - 4
Init.sh:
#!/bin/sh
#directory where accounting scripts reside
rrddir=/tmp/harddisk/rrd
# directory to store RRD database files
datadir=/tmp/rrd
# change these values depending on WAN connection
wanint="eth0"
LAN="192.168.1.1"
#masked my ip...
WAN="62.xx.214.55"
#ipprefix consists of the first 3 number of your
#LAN side ip addres.
#
ipprefix="192.168.1"
# define mask that includes all PC's in LAN
# e.g. addresses .2 .3 .4 .5 .6 fit within $ipprefix.0/29 mask.
LANMask=255.255.255.0
Ipaddress.txt:
Sveinung 192.168.1.2
Regin 192.168.1.3
Ingunn 192.168.1.4
Can anyone see any errors?
oversc0re
01-06-2006, 19:58
If it does not gaether data:
1) Check the IPTABLES entries and make sure that it produces the output proportional to the data transfer
2) Make sure crontab is running!
3) Try feeding rrdtool manually and check if it works. (you can see all the commands in my scripts)
Try being more specific describing your problem...
Greetz, over.
audiophil
01-06-2006, 20:27
Thanks for the reply oversc0re!
The cron is not the problem, I've tried to run the update_traf.sh manually and followed by plot_traf.sh.
This got a bit big, but heres my iptables -L output:
I'm aware that there are duplicate entries, I ran ./add_iptables.sh two times... But you see my data.
Chain INPUT (policy ACCEPT)
target prot opt source destination
router_in all -- !192.168.1.0/24 trondelag-dhcxxxxxecom.no
router_in all -- !192.168.1.0/24 trondelagxxxxxxxom.no
MACS all -- anywhere anywhere
DROP all -- anywhere anywhere state INVALID
ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED
ACCEPT all -- anywhere anywhere state NEW
ACCEPT all -- anywhere anywhere state NEW
SECURITY all -- anywhere anywhere state NEW
ACCEPT udp -- anywhere anywhere udp spt:bootps dpt:bootpc
ACCEPT tcp -- anywhere anywhere tcp dpt:ftp
DROP all -- anywhere anywhere
Chain FORWARD (policy ACCEPT)
target prot opt source destination
traffic_out all -- anywhere anywhere
traffic_in all -- anywhere anywhere
traffic_out all -- anywhere anywhere
traffic_in all -- anywhere anywhere
MACS all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
DROP all -- anywhere anywhere state INVALID
ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED
SECURITY all -- anywhere anywhere state NEW
ACCEPT tcp -- anywhere 192.168.1.2 tcp dpts:ftp-data:ftp
ACCEPT tcp -- anywhere 192.168.1.2 tcp dpts:6881:6882
ACCEPT udp -- anywhere 192.168.1.2 udp dpts:6881:6882
ACCEPT tcp -- anywhere 192.168.1.2 tcp dpt:31214
ACCEPT udp -- anywhere 192.168.1.2 udp dpt:31214
ACCEPT tcp -- anywhere 192.168.1.4 tcp dpt:32493
ACCEPT udp -- anywhere 192.168.1.4 udp dpt:32493
ACCEPT udp -- anywhere anywhere udp dpt:6112
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
router_out all -- tronxxxxxxcom.no !192.168.1.0/24
router_out all -- tronxxxxxxxxxxom.no !192.168.1.0/24
Chain MACS (2 references)
target prot opt source destination
Chain SECURITY (2 references)
target prot opt source destination
RETURN tcp -- anywhere anywhere tcp flags:SYN,RST,ACK/SYN limi t: avg 1/sec burst 5
RETURN tcp -- anywhere anywhere tcp flags:FIN,SYN,RST,ACK/RST limit: avg 1/sec burst 5
RETURN udp -- anywhere anywhere limit: avg 5/sec burst 5
RETURN icmp -- anywhere anywhere limit: avg 5/sec burst 5
DROP all -- anywhere anywhere
Chain logaccept (0 references)
target prot opt source destination
LOG all -- anywhere anywhere state NEW LOG level warning tc p-sequence tcp-options ip-options prefix `ACCEPT '
ACCEPT all -- anywhere anywhere
Chain logdrop (0 references)
target prot opt source destination
LOG all -- anywhere anywhere state NEW LOG level warning tc p-sequence tcp-options ip-options prefix `DROP'
DROP all -- anywhere anywhere
Chain router_in (2 references)
target prot opt source destination
all -- anywhere anywhere
all -- anywhere anywhere
Chain router_out (2 references)
target prot opt source destination
all -- anywhere anywhere
all -- anywhere anywhere
Chain traffic_in (2 references)
target prot opt source destination
all -- anywhere 192.168.1.2
all -- anywhere 192.168.1.3
all -- anywhere 192.168.1.4
all -- anywhere !192.168.1.0/24
all -- anywhere 192.168.1.2
all -- anywhere 192.168.1.3
all -- anywhere 192.168.1.4
all -- anywhere !192.168.1.0/24
Chain traffic_out (2 references)
target prot opt source destination
all -- 192.168.1.2 anywhere
all -- 192.168.1.3 anywhere
all -- 192.168.1.4 anywhere
all -- !192.168.1.0/24 anywhere
all -- 192.168.1.2 anywhere
all -- 192.168.1.3 anywhere
all -- 192.168.1.4 anywhere
all -- !192.168.1.0/24 anywhere
I've made a start.sh which I run from post-boot (it runs after mounting usb-drive).
Start.sh:
#/bin/sh
/opt/rrd/add_iptables.sh
/opt/rrd/init_traf.sh
/opt/rrd/update_traf.sh
/opt/rrd/plot_traf.sh
Edit:
Just add that the image generation works just fine for eth0/eth1 (somehow they both contain data?). Which one is the "wan" port on the back of the router?
oversc0re
01-06-2006, 22:24
Emm ... I am having troubles understanding what is working and what not.. Is the plot working for any connection? Are the graphs plotted without data?
The only interface that contained valid data for me was the ppp0 interface. Not eth0 nor eth1 didn't work for me.
If you put
iptables -n -L traffic_in -vx into the command line you should get something like this:
Chain traffic_in (1 references)
pkts bytes target prot opt in out source destination
3220697 2549139449 all -- ppp0 * 0.0.0.0/0 0.0.0.0/0
1231 734960 all -- ppp0 * 0.0.0.0/0 192.168.0.10
18126 9919769 all -- ppp0 * 0.0.0.0/0 192.168.0.20
33810 34429352 all -- ppp0 * 0.0.0.0/0 192.168.0.30
3167530 2504055368 all -- ppp0 * 0.0.0.0/0 192.168.0.110
The bytes column contains transfered data and it should constantly increase. If it doesn't you have an IPTABLES problem. Check my scripts for an alternative configuraton of iptables (a few post below).
Greetz, over.
audiophil
01-06-2006, 22:47
[admin@(none) rrd]$ iptables -n -L traffic_in -vx
Chain traffic_in (2 references)
pkts bytes target prot opt in out source destination
0 0 all -- eth1 * 0.0.0.0/0 192.168.1.2
0 0 all -- eth1 * 0.0.0.0/0 192.168.1.3
0 0 all -- eth1 * 0.0.0.0/0 192.168.1.4
0 0 all -- eth1 * 0.0.0.0/0 !192.168.1.0/24
0 0 all -- eth1 * 0.0.0.0/0 192.168.1.2
0 0 all -- eth1 * 0.0.0.0/0 192.168.1.3
0 0 all -- eth1 * 0.0.0.0/0 192.168.1.4
0 0 all -- eth1 * 0.0.0.0/0 !192.168.1.0/24
Needless to say, something is wrong...
oversc0re
01-06-2006, 22:52
It might be the eth1 interface... try ppp0.
audiophil
01-06-2006, 22:56
Isn't ppp0 only for those who use ppp authing with their ADSL provider? I got "plug and play"-style ADSL.
Edit:
When using ppp0 and ./update_traf i get:
ifconfig: ppp0: error fetching interface information: Device not found
Which kinda rules out ppp0.
oversc0re
01-06-2006, 22:58
Sry... u're right about that. Well then try eth0 :) Keep on trying until the numbers start rolling.
Edit: And if you want to monitor all the router traffic, take a look at my first entry.
audiophil
02-06-2006, 12:35
Whyyy wont it work?! Frustrated.... What's so special about my setup?
I get readings from vlan0, vlan1, eth0, eth1, but nothing on the ips.
audiophil
02-06-2006, 16:55
Finally making progress, just one question oversc0re.
In your shineon.cgi you refer to a /opt/ew.sh which is not included in the archive you uploaded earlier...
oversc0re
02-06-2006, 18:52
errr ... well ... shineon.cgi is a script used for turning on computer via Wake on lan... I wanted to make a http server so that i would be able to turn on my pc when I'm not at home, but never did it.
Here's the contents of ew.sh
ether-wake -i vlan0 00:50:FC:E4:88:88
If you manage to create the server on a port that would be accessible from wan, please let me know.
Greetz, over
P.S. It's nice to hear you are making some progress.
audiophil
02-06-2006, 18:59
I just turned on enable web access in the web-admin system. Works with both web-admin and my server on :81. Could you just take a peek at my stat, I got these "blanks"... And the scale on the left is way off. I got a 1500kb/500kb connection.
And the "Overall download" doesn't contain any stats.
I'll pm you with the url to stats to avoid abuse.
oversc0re
02-06-2006, 19:59
About your problems:
1) The scale is dynamic. The scale max is allways a little more than the peak value displayed. As your download will increase, so will the scale. The red area and the red line are defined in the graph creation script. You can browse through files and modify it. If you can't find it, let me know, and i'll find it for you.
2) Those blanks are really strange... since there is no line (even at 0) it means, that the data was not captured. I suggest that you restart the router (first run the backup.sh script, so you won't loose the data you have captured so far) and hopefully it won't happen again.
3) The overall download script uses yearly rrd database that is updated only once a day. You wil get your first results in 24 hours.
Let me know if it works...
Hi,
I'm using scripts based on barsju's. (http://wl500g.info/showpost.php?p=16719&postcount=23). I changed scripts according to my needs (FUP etc...).
I log traffic made by router itself. I realized that router generates traffic all the time, as you can see on attached picture (There is IP 10.130.0.1 but it is traffic for router itself).
Part of my add_iptables is here:
# create two chains for in and outbound traffic of ROUTER (HTTP/FTP etc)
iptables -N router_out
iptables -N router_in
# add rules for both directions
iptables -A router_out -s $WAN -d ! 172.19.10.0/24
iptables -A router_in -d $WAN -s ! 172.19.10.0/24
# attach chains to INPUT/OUTPUT
iptables -I INPUT -j router_in -d $WAN -s ! $ipprefix.0/24
iptables -I OUTPUT -j router_out -s $WAN -d ! $ipprefix.0/24
$WAN = 172.19.10.141
$ipprefix = 10.130.0
I think, that my add_iptables.sh is good, because WAN led on my router is always blinking. Traffic is being generated even if all computers on network are off.
Does someone know what is the traffic?
I try to glue the Oversc0re script on 500gp with -10 firmware.
After tinkering alot with permisions scripts work, get data from vlan1 (wan), but no success with seeing results on :81 ...
The index page is there but no access from lan , look like router block access ..
Is a way to make this work on 500gp ?
Y get the filing y am close ...
Look to me like the default route for www pages generated internally must be on /opt/share/www (on -10 fw) but this script put www on /tmp/harddisk/rrddata/www/ ...
But now i am stuck, someone have a ideea for a beginner like me ?
And dont find where y can specify the port used by this script ..
Now i can open page directly on router with lynx, but is no graphic there :) because of lynx ..
LE: Now i can access page on 81, was my mistake failing to edit all scripts. I keep you updated.
Now its ok, who want script updated for 500 gp ask here....actually is just Overscore said, only some tinkering with makind all .sh executable and update user name and IPs.
One problem now, executing nvram_install.sh kill my samba and transmission ...
Another adventure now, guys its like a safari :)
LE: find the problem, nvram_install.sh flushed the line "/opt/etc/init.d/rc.unslung start" from post-mount, y added and reboot, and samba and transmision are ok.
The another thing was flushed, post-firewall, and i cant sleep good now, because here y cant understand a thing :
the new post-firewall look like this:
iptables -N traffic_out
iptables -N traffic_in
iptables -A traffic_in -i vlan1
iptables -A traffic_out -o vlan1
iptables -A traffic_in -i vlan1 -d 192.168.1.1
iptables -A traffic_out -o vlan1 -s 192.168.1.1
iptables -A traffic_in -i vlan1 -d 192.168.1.2
iptables -A traffic_out -o vlan1 -s 192.168.1.2
iptables -A traffic_in -i vlan1 -d 192.168.1.3
iptables -A traffic_out -o vlan1 -s 192.168.1.3
iptables -I FORWARD -j traffic_in
iptables -I FORWARD -j traffic_out
I get the filling now y am a sitting duck ....
So, how i can add my rules with 65534 and 22 only open to outside ?
I add my rules in front or what ?