vsftpd crashes when logfile accessed - help needed!

[newbiefan]

Hi all,
hopefully someone had this particular problem.
I use Ily's FW on my asus, connected behind a fritzbox (modem/wlanrouter/voip), everything works stable and fine, even from internet. I use interal vsftpd as well as opt-lighttpd - as mentioned everything works very stable.
The problem for me are the brute force attacks. In order to avoid brute force attacks, I've written together with wpte a script which is doing just one thing - the script analizes the logfile for failed vsftpd logins and when more as x failed logins are detected, the ip is blocked by iptables and/or xinetd.
The script works fast and stable, except when someone is accessing vsftpd at the same time (internal or external). Then vsftpd and the script are crashing, just a reboot on a new console helps.

Now my question: My guess is the logfile /opt/var/log/syslog.log - when vsftpd writes to log and the script is accessing the logfile at the same time, vsftpd and the script crash.
How can I ensure, that just one prog. is accessing the logfile at the same time?
Has anybody an idea or hint?

Thanks
newbiefan

[avberk]

Let the script make a copy of the log file and access the copy

[newbiefan]

Let the script make a copy of the log file and access the copy

Well, would be nice to make a copy without accessing the logfile.
Further, then I would need to make a copy every minute, as vsftpd writes to log!?

No seriously: with delay_failed_login=15 and max_login_fails=3 as option in vsftpd.conf, I dropped down brute force attacks to a handful failed logins a day. Works pretty good - but the script would be better, because then I can deny any access to my asus from banned ips (even webserver) and I can collect bad networks (with netmask).
But I got some ideas how to solve this problem.....will let you know when finished (will take some days).
have fun
newbiefan

[al37919]

how do you access it in your script?

If you do something like open/read/close then I can imagine, however if it is something like:

cat logfile | grep something 

I can't believe

Well if so, you may test e.g.:

[ -r logfile ] && cat ... 

[newbiefan]

Hi al37919!

Many thanks for your input....
It was exactly my idea, therefore I'll take out all writes to logfile.
Will do it in the next couple days.
Anyhow, here is the script - just adjust the cut-line, depending to your vsftpd conf (Xferlog).
The script is not absolutly finished, one if statement more is needed. Right before the search.

thks again
newbiefan

edit: ah, I forgot - when xinetd is used, take out only_from....

[al37919]

I guess safer writing to the syslog could be:

logger -t checklog MESSAGE 

Alternatively, anyway you use separate logfile why overwhelm the main log with extra messages?

Finally, I beleive this approach is not very effective With time your ban list will definitely grow, and you have a separate rule for every ever banned IP...

I believe the ipt_recent approach is more effective. It just limits number of connections from every single IP to N within T seconds. If you use 1.9.2.7-d firmware you can enable it for ssh and ftp server just in the web-interface.

[newbiefan]

Hi al37919!

I agree, with logger it should be a safer writing.

Ok, I agree - it makes no sense to write everything to syslog. Will use an own logfile.....

Regarding growing ip-list: As the banned IP's are just stored in RAM and I do a reboot by cron every night, it should not grow so much, because after 3 failed logins the ip is banned and the script is ending himself when nothing is changed. But what I want to do is to ban some brasils, taiwanese IT university, some provider from korea, china and a few from ukraine and romania. Especially the taiwanese IT university is attacking me nearly daily from several servers. Maybe I am on some hacking list
Sometimes it's not only a brute force, looks like a DOS & brute force, just eating up my bandwidth.

Well and the tcpwrapper from nslu2-linux needs as package denyhosts and is just a nightmare, nothing else - because there is no tcpd, just some tools and very hard to find some useful documentation - further it's too much - the script is a way shorter and safe.

Would like to use ip_recent, but my asus is just an accesspoint (how can I enable it?) - when you point me in the right direction, I would appreciate it. I do not have a folder /proc/net/ipt_recent, and a 'find / -name "ipt_recent"' doesn't show me any file.
uname -a gives me 'Linux server 2.4.37 #1 2009-03-05 21:06:01 MSK mips GNU/Linux'

Many thanks would not be the first time, that you point me in the right direction.......

Newbiefan

EDIT: Ok, found something: nvram show | grep recent
... now I've set ftp to 1, give it a try....
EDIT2: ok found it, had a type mismatch (find).... insmod /lib/modules/2.4.37/ipt_recent.o....
Many thanks anyway!

[al37919]

Would like to use ip_recent, but my asus is just an accesspoint (how can I enable it?) - when you point me in the right direction, I would appreciate it. I do not have a folder /proc/net/ipt_recent, and a 'find / -name "ipt_recent"' doesn't show me any file.
uname -a gives me 'Linux server 2.4.37 #1 2009-03-05 21:06:01 MSK mips GNU/Linux'

I never was using it in AP mode but I think it should also work (with exclusion of the fact that I don't understand what does mean firewall in AP mode ). Check corresponding settings in the web under Internet Firewall -> Basic Config. When enabled, ipt_recent should be loaded by firmware itself.

To say frankly I didn't tested this feature with ftp server (only ssh), but I believe that it should work in the same way. So, your testing input would be quite useful. Good luck.

P.S. You can watch current state of the blocked items by:

cat /proc/net/ipt_recent/BRUTE 

And remember that it limits rate of all attempts, not only wrong ones. From this point of view I'm starting to think that the current settings are a bit too strict for ftp.

[newbiefan]

I never was using it in AP mode but I think it should also work (with exclusion of the fact that I don't understand what does mean firewall in AP mode ). Check corresponding settings in the web under Internet Firewall -> Basic Config. When enabled, ipt_recent should be loaded by firmware itself.

When switched to AP Mode, no Internet Firewall is available.
As I want to ban just an IP it's not a real firewall - furthermore it's a blocking of incomming traffic. And every ethernet-port in AP Mode is just handled as INPUT, except traffic from internal progs.

To say frankly I didn't tested this feature with ftp server (only ssh), but I believe that it should work in the same way. So, your testing input would be quite useful. Good luck.

Thanks, al37919 but it's tricky, because I do not want to lower the rates with every access. It means, that I have to read some hours to find the right way. Still do not know exactly how to use ipt_recent, but got several ideas. Anyway, it's a learning process and needs time - I bet I can archieve to ban my ftp-hackers with ipt_recent.

From this point of view I'm starting to think that the current settings are a bit too strict for ftp.

Well, as far as I know, I need a totally different setting - have to learn it anyway...
Thanks for your input!


I tried to use tcpwrapper, which doesn't work. Further,the vsftpd has no compiled in tcpwrapper-support. It would require a complete recompiling.

Xinetd can be used with vsftpd but requires a xinetd reload. And a script is used (checklog) to filter failed logins.

Iptables requires nothing, just the script checklog.

The ipt_recent with iptables requires nothing, just a propper configuration, but only once.

So I decided, as al37919 suggested, to use ipt_recent. When finished everything, I'll write a howto about vsftpd and ipt_recent in connection with iptables. Anyway, it will be limited to AP Mode - but should work in the same way when asus is used as firewalled router, because everything is treated as INPUT.

Will come back when finished.....

[al37919]

BTW, here is vsftpd 2.0.7 with tcp-wrappers support: http://wl500g.info/showpost.php?p=119458&postcount=132

Possible usage of ipt_recent I described here (in Russian): http://wl500g.info/showpost.php?p=69964&postcount=63 and tried to explain here: http://wl500g.info/showthread.php?t=13274 in German (but for my Deutsch it was a bit too complicated trial).

Just two more eurocents: ipt_recent has two modes of operation --- --update and --rcheck. Usually I use the most restrictive --update for ssh. However, at a glance for ftp --rcheck looks possibly better (I overlooked that fact when was creating ruleset in the 1.9.2.7-d firmware because I really don't use it for ftp). Share your experience, may be we need to update usage of this method built into the firmware.

[newbiefan]

Share your experience, may be we need to update usage of this method built into the firmware.

It's promised, I'll do it - but it will take a few days.
Thanks again

[newbiefan]

@al37919

well, here is my solution for ftp and ssh - works extremly good - since sunday evening I had no brute force or dos attacks. In general, I just changed your solution - maybe you can use your script which is shorter.

If you change this line of your script:

# Transfer all ssh connections to the SSH_EVAL chain 
iptables -A INPUT -p tcp --dport $SSH_PORT -j SSH_EVAL

to

iptables -A INPUT -m state --state NEW -p tcp --dport $SSH_PORT -j SSH_EVAL

you can avoid the '--state NEW' in both lines - thats all what I've found to improve your work....
I just added some hitcounts.....thats all - no more left-over for me to improve...you've done a real good job.
I just changed the defaults in nvram to 3600 instead of 600 (it rumored that there can be some probs when default values exeeded - don't know it for sure).

In any chase, I have to say thanks, because you inspired me to have a closer look at iptables and ipt_recent. Now almost of my problems are gone!
I've tested the script as much as I can, even from external ip's.
And during the next weekend, I'll write a howto in german and english - because this is working even in AP mode.

The only thing what remains is now my checklog script. Maybe I can extend the script to find a complete subnet of a blocked ip and write this to /opt/etc/hosts.deny
Ah, script ashow is just for iptables -L, cat /proc/net/ipt_recent/.........

Info for users who can not wait....
avbf to /opt/sbin/avbf
ashow to /opt/sbin/ashow
hosts.deny to /opt/etc/hosts.deny

Thanks again!
newbiefan

[al37919]

You are absolutely right!

I realized that I have more current implementation which now is built in 1.9.2.7-d (however as you said not available in AP mode):

iptables -A INPUT -p tcp -m tcp --dport 21 --syn -j BRUTE
iptables -A INPUT -p tcp -m tcp --dport 22 --syn -j BRUTE

... handling of white/blacklists in BRUTE chain ...
iptables -A BRUTE -m recent --update --seconds 600 --hitcount 3 --name BRUTE --rsource -j DROP
iptables -A BRUTE -m recent --set --name BRUTE --rsource -j ACCEPT 

I don't know which is better to use:
-m state --state NEW (new connection)
or
--syn (login)
I think they do the same job. Currently I'm using --syn

Just to underline once again --- ipt_recent counts not only unsuccessful login attempts, but all of them. Therefore by setting --seconds too high you may finish one day in locking yourself. Then you have to wait 3600 seconds until you can try to login yourself. Possibly better default strategy would be 1 connection within 60 seconds. It is enough to make bruteforce attacks useless, but will make minimum harm to lawful users.

If you have open ssh connection you can perform some handling of the stored table:

/proc/net/ipt_recent/* are the current lists of addresses and information
about each entry of each list.

Each file in /proc/net/ipt_recent/ can be read from to see the current list
or written two using the following commands to modify the list:
'echo xx.xx.xx.xx > /proc/net/ipt_recent/DEFAULT' to Add to the DEFAULT list
'echo -xx.xx.xx.xx > /proc/net/ipt_recent/DEFAULT' to Remove from the DEFAULT list
'echo clear > /proc/net/ipt_recent/DEFAULT' to empty the DEFAULT list.

[newbiefan]

I don't know which is better to use:
-m state --state NEW (new connection)
or
--syn (login)
I think they do the same job. Currently I'm using --syn

Will check it out....I guess it should be nearly the same.

Just to underline once again --- ipt_recent counts not only unsuccessful login attempts, but all of them. Therefore by setting --seconds too high you may finish one day in locking yourself. Then you have to wait 3600 seconds until you can try to login yourself. Possibly better default strategy would be 1 connection within 60 seconds. It is enough to make bruteforce attacks useless, but will make minimum harm to lawful users.

You are absolutely right, it's just my paranoia..
I have added (for the next days) for every hitcount a logentry. This will show me a necessity or not. I guess you are right - will shorten the script possibly.
But there are some (maybe hacked) different networks-when an ip is blocked, I get sometimes (pretty often) a new access to vsftpd and/or ssh just a second later from an other ip and not from the same netmask. Now I will write a software or script which blocks and overtake such an access to my blacklist. Have to investigate such a behaviour, because it looks sometimes like they have hosts around the world available.
But this will take a while..

Anyhow, thanks again - will come back when I know more.
Newbiefan

Edit: As there is the multiport module available, it should be possible to do the same for ssh and ftp with just one simple rule and without a new chain - would be extremly short. Have to check this.

[al37919]

Edit: As there is the multiport module available, it should be possible to do the same for ssh and ftp with just one simple rule and without a new chain - would be extremly short. Have to check this.

Not sure about multiport, but updated previous post to reflect current realization in 1.9.2.7-d more precisely.