squid для WL-500gP

Quote:

---

Originally Posted by xkir

Ñïàñèáî çà âûëîæåííóþ ñáîðêó, çàâåëîñü :) ïðàâäà ðóãàëîñü íà CRC ïàêåòà, ïðèøëîñü ñòàâèòü âðó÷íóþ.
Âîïðîñ: Ïîäñêàæèòå, êàê â íàñòðîéêàõ ñêâèäà ñîâñåì îòêëþ÷èòü äèñêîâûé êåø? ÷òî-òî â äîêàõ è FAQ íå íàøåë.

---

Ïëîõî èñêàëè :)

squid äîëæåí áûòü ñêîìïèëèðîâàí ñ îïöèåé: --enable-storeio="ufs,null,... " (òî åñòü êàê ðàç â ìîåé ñáîðêå ýòî åñòü), à çàòåì óêàçàòü â êîíôèãå:

Code:

---

cache_dir null /opt/var/squid/cache

---

Ñáîðêó ñêâèäà (è nginx òîæå) òîæå ñòàâëþ ïðåäâàðèòåëüíî ïîäìåíÿÿ êîíòðîëüíóþ ñóììó â ëèñòå äèñòðèáîâ, åñëè êòî çíàåò áîëåå "ãóìàííûé" ñïîñîá, íàïèøèòå...

ìíå êàæåòñÿ åñëè èìÿ ïàêåòà ïåðåèìåíîâàòü, íà íå ñòàíäàðòíîå òî îí íå íàéäåò íè÷åðòà â ñïèñêàõ è ïîñòàâèò òàêîé êàê åñòü ) íå ïðîáîâàë íèêòî?

ïîñëå ñìåíû redirector.sh è óñòàíîâêè grep 2.5.3
Âûâàëèëñÿ SQUID ñî ñëîâàìè

Code:

---

Dec  1 11:10:40 squid[255]: Squid Parent: child process 304 started
Dec  1 11:10:42 squid[304]: The url_rewriter helpers are crashing too rapidly, need help!
Dec  1 11:10:42 squid[255]: Squid Parent: child process 304 exited due to signal 6

---

à ñàì redirector.sh

Code:

---

./redirector.sh: ./redirector.sh: 7: Syntax error: Unterminated quoted string

---

Quote:

---

Originally Posted by serji

Ïðîñòèòå ãäå?

---

Íó ó ìåíÿ ëåæèò, íàñîáèðàë âðó÷íóþ ïðè ñåðôèíãå ïî çëà÷íûì ìåñòàì :)
ß èõ íå âûêëàäûâàþ, ïîòîìó êàê ó êàæäîãî ñïèñîê áóäåò ñâîé.

Ïðî÷èòàë â î÷åðåäíîé ðàç ìàíû ïðî ðåäèðåêòîðû Squid íåìíîãî óòî÷íèë ñâîé ðåäèðåêòîð, ëèøíèå ïåðåìåííûå óáðàë è ïðî÷åå:

PHP Code:

---

#!/bin/sh

while : 
do
  read url ip ident method
    [ "`echo "$url" | /opt/bin/grep -m1 -Ff /opt/bannerfilter/exceptions`" = "" ] && ([ "`echo "$url" | /opt/bin/grep -m1 -Ff /opt/bannerfilter/banners`" ] && echo 'http://192.168.1.1/null.gif' || echo) || echo
done 


---

Çàîäíî âûÿñíèë ÷òî 5 ïðîöåññîâ êðàéíå ìàëî (ó ìåíÿ çàòûê áûë èìåííî â ýòîì ìåñòå..), òàêæå âûÿñíèë ÷òî ìîé áðàóçåð (Firefox) ìîæåò äåðæàòü 16 êîííåêòîâ ñ ñàéòîì îäíîâðåìåííî, à ñêâèä ïîëó÷àåòñÿ ìîæåò îáðàáîòàòü òîëüêî 5. Ïîýòîìó ñäåëàë õîä êîí¸ì - óâåëè÷èë êîëè÷åñòâî ïðîöåññîâ äî 30. :)

Code:

---

url_rewrite_children 30

---

Òåïåðü âñå ëåòàåò è overload ãîðàçäî íèæå.

Ëþäè, çàïîñòüòå ïîæàëóéñòà _ìèíèìàëüíûé_ ðàáî÷èé êîíôèã äëÿ squid'a.
Âñåì ñïàñèáî! :)

Âîò ñ òàêèì îí çàïóñêàåòñÿ (âèäíî ïî ps), íî íå ñëóøàåò ïîðò 3128 :(

(NCSA àâòîðèçàöèÿ ìíå íóæíà, ò.å. îíà â êîíôèãå íå ñëó÷àéíî)

Code:

---

visible_hostname TEST_SQUID
http_port 127.0.0.1:3128

access_log /opt/var/squid/logs/access.log squid

coredump_dir /opt/var/squid/cache

auth_param basic program /opt/libexec/ncsa_auth /opt/etc/squid/users.acl
auth_param basic children 5
auth_param basic realm Squid on ONE-BOX
auth_param basic credentialsttl 8 hour

acl all src 0.0.0.0/0.0.0.0
http_access allow all

---

Squid ñòàðòóåò áåç îøèáîê, åñòü â ïðîöåññàõ, íî íå îòêðûâàåò 3128 ïîðò :(

Code:

---

$/opt/etc/init.d/S80squid start
Starting proxy server: done.
done.

---

Ïîäñêàæèòå ïîæàëóéñòà, ÷òî íå òàê â êîíôèãå:

Code:

---

visible_hostname SQUID_TEST
http_port 3128

auth_param basic program /opt/libexec/ncsa_auth /opt/etc/squid/users.acl
auth_param basic children 1
auth_param basic realm SQUID_TEST
auth_param basic credentialsttl 2 hours

cache_dir ufs /opt/var/squid/cache 20 16 256
pid_filename /opt/var/squid/logs/squid.pid
access_log /opt/var/squid/logs/access.log squid
icon_directory /opt/share/squid/icons
error_directory /opt/share/squid/errors/Russian-1251
coredump_dir /opt/var/squid/cache

cache_effective_user nobody
cache_effective_group nobody

cache_swap_low 97
cache_swap_high 100

acl all src 0.0.0.0/0.0.0.0
http_access allow all

---

Íóæåí _ìèíèìàëüíûé_ êîíôèã...
ACL ïîòîì áóäó ãåíåðèòü â êîíåö êîíôèãà (ñêðèïòîì èç îïèñàíèÿ).

 îáùåì ðàçîáðàëñÿ - íàäî ñêâèä ñ êëþ÷åì -D çàïóñêàòü, ÷òîáû îí íå äåëàë DNS òåñòîâ (íà íèõ è âàëèëñÿ).

Êàê óçíàë: ðàçâåäàë ýòî êîãäà çàïóñòèë â ðåæèìå debug (êëþ÷ -d1).

 îáùåì ðàçîáðàëñÿ - íàäî ñêâèä ñ êëþ÷åì -D çàïóñêàòü, ÷òîáû îí íå äåëàë DNS òåñòîâ (íà íèõ è âàëèëñÿ).

Êàê óçíàë: ðàçâåäàë ýòî êîãäà çàïóñòèë â ðåæèìå debug (êëþ÷ -d1).

Ê ñîæàëåíèþ íà êîíñîëü âñ¸ ðàâíî ïåðèîäè÷åñêè âàëÿòñÿ ñîîáùåíèÿ:

Code:

---

/opt/etc/init.d/FATAL: ipcache_init: DNS name lookup tests failed.
2009/01/06 06:51:20| Squid is already running!  Process ID 609

---

_Äåéñòâèòåëüíî_ ìèíèìàëüíûé êîíôèã:

Code:

---

visible_hostname SQUID_TEST
http_port 3128

cache_dir ufs /opt/var/squid/cache 20 16 256
pid_filename /opt/var/squid/logs/squid.pid
access_log /opt/var/squid/logs/access.log squid
icon_directory /opt/share/squid/icons
error_directory /opt/share/squid/errors/Russian-1251
coredump_dir /opt/var/squid/cache

cache_effective_user nobody
cache_effective_group nobody

cache_swap_low 97
cache_swap_high 100

acl all src 0.0.0.0/0.0.0.0
http_access allow all

---

À âîò ìèíèìàëüíûé êîíôèã ñ ïîääåðæêîé NCSA àâòîðèçàöèè ïîëüçîâàòåëåé:

Code:

---

visible_hostname SQUID_TEST
http_port 3128

auth_param basic program /opt/libexec/ncsa_auth /opt/etc/squid/users.acl
auth_param basic children 1
auth_param basic realm SQUID_TEST
auth_param basic credentialsttl 2 hours

cache_dir ufs /opt/var/squid/cache 20 16 256
pid_filename /opt/var/squid/logs/squid.pid
access_log /opt/var/squid/logs/access.log squid
icon_directory /opt/share/squid/icons
error_directory /opt/share/squid/errors/Russian-1251
coredump_dir /opt/var/squid/cache

cache_effective_user nobody
cache_effective_group nobody

cache_swap_low 97
cache_swap_high 100

acl all src 0.0.0.0/0.0.0.0

#
#      per-user ACLs
#
acl test_user proxy_auth user

#
#      per-user access rules
#
http_access allow test_user all

http_access deny all

---

Åñòåñòâåííî íå çàáûâàåì äîáàâëÿòü þçâåðåé â /opt/etc/squid/users.acl
êîìàíäîé (äîëæåí áûòü óñòàíîâëåí (íî íå îáÿçàòåëüíî çàïóñêàòü) àïà÷!):

Code:

---

htpasswd -b /opt/etc/squid/users.acl username password

---

À òàêæå â êîíôèã.

Åñòåñòâåííî ëó÷øå äëÿ äîáàâëåíèÿ íàïèñàòü ñêðèïò, ÷òîáû íå ïðàâèòü äëÿ êàæäîãî þçåðà 2 ôàéëà... ;)

Ïîñòàâèë ïîë ãîäà íàçàä ñêâèä
4 äîìàøíèõ þçåðà
1 Ãá USB flash äëÿ êåøà
Òàê âîò ïðè îòêðûòèå ñòðàíèö ÷óâñòâîâàëèñü òîðìîæåíèÿ,,
áûëî òàê: çàïðîñ ñòðàíèöû ïàóçà ïîòîì áàõ âñÿ ñòðàíèöà öåëèêîì îòêðûâàëàñü
ïîäîçðåâàþ, ÷òî ýòî èç çà òîãî, ÷òî ñêîðîñòü ÷òåíèÿ ôàëîâ ñ êåø äèñêà ìåäëåííå ÷åì çàãðóçêà èç ñåòè, ïîäîçðåâàþ ÷òî êýø ñåðâåð ñòîèò è ó ïðîâàéäåðà (àâàíãàðä) ïîýòîìó äîñòóï ê íåìó äîñòàòî÷íî áûñòðûé

Ïîïðàâüòå åñëè ÿ íå ïðàâ. Ñíåñ

Íà êîìïå ñ linux, êîòîðûé ñòîÿë äî ðîóòåðà â êà÷åñòâå øëþçà - âñå ðàáîòàëî õîðîøî è áûñòðî
Òàê ÷òî ïîäîçðåâàþ ÷òî íå ìåñòî ñêâèäó íà ðîóòåðå

Ñîáðàë íîâûé squid, à òî ÷òî-òî åãî ñîâñåì çàáðîñèëè..

Äîïîëíèòåëüíûå îïöèè:
--enable-storeio="ufs,null" \
--disable-carp \
--disable-wccp \
--disable-wccpv2 \
--enable-delay-pools \
--enable-kill-parent-hack \
--enable-err-language="Russian-1251" \
--enable-default-err-language="Russian-1251" \
--disable-ident-lookups \

Î÷åðåäíûå îáíîâëåíèÿ:

Ñîáðàë squid ñàìûé ïîñëåäíèé èç âåòêè 2.6 ñ îïöèÿìè:
--enable-storeio="ufs,null" \
--enable-delay-pools \
--enable-kill-parent-hack \
ò.å. ìàêñèìàëüíî óíèâåðñàëüíûé, äëÿ òåõ êòî õî÷åò ïðîäîëæàòü ïîëüçîâàòüñÿ âåòêîé 2.6


Äëÿ îñòàëüíûõ î÷åðåäíàÿ âåðñèÿ 2.7.6 ñ îïöèÿìè:
--enable-storeio="ufs,null" \
--disable-carp \
--disable-wccp \
--disable-wccpv2 \
--enable-delay-pools \
--enable-kill-parent-hack \
--enable-err-language="Russian-1251" \
--enable-default-err-language="Russian-1251" \
--disable-ident-lookups \

lightparser.pl ñèëüíî ãðóçèò ñèñòåìó!

Code:

---

Mem: 29248K used, 916K free, 0K shrd, 528K buff, 12056K cached
Load average: 3.38, 2.95, 2.43    (State: S=sleeping R=running,

  PID USER    STATUS  RSS  PPID %CPU %MEM COMMAND
30318 admin    R      1208 30316 88.3  4.0 lightparser.pl
25090 admin    R        244 20858  2.6  0.8 top
  172 nobody  S      5512  170  2.0 18.2 squid
  174 nobody  S        240  172  0.1  0.7 redirector.sh
30332 admin    S        124 30318  0.1  0.4 cat
  89 admin    SW        0    1  0.1  0.0 usb-storage-0
    3 admin    RWN        0    1  0.1  0.0 ksoftirqd_CPU0
  126 admin    S        324    1  0.0  1.0 nmbd
30316 admin    S        284 30314  0.0  0.9 sh
  176 nobody  S        244  172  0.0  0.8 redirector.sh
  179 nobody  S        244  172  0.0  0.8 redirector.sh
  173 nobody  S        240  172  0.0  0.7 redirector.sh
  175 nobody  S        240  172  0.0  0.7 redirector.sh
  177 nobody  S        240  172  0.0  0.7 redirector.sh
  178 nobody  S        240  172  0.0  0.7 redirector.sh
  180 nobody  S        240  172  0.0  0.7 redirector.sh
15947 admin    S        232    1  0.0  0.7 httpd
  94 admin    S        196    1  0.0  0.6 watchdog
  71 nobody  S        192    1  0.0  0.6 dnsmasq
  165 admin    S        176    1  0.0  0.5 lighttpd
  92 admin    S        172    1  0.0  0.5 pppd
  115 admin    S        124    1  0.0  0.4 dropbear
30314 admin    S        116  148  0.0  0.3 cron
  148 admin    S        112    1  0.0  0.3 cron
  65 admin    S        88    1  0.0  0.2 syslogd
  58 admin    S        72    1  0.0  0.2 telnetd
    1 admin    S        68    0  0.0  0.2 init
  208 nobody  S        60  172  0.0  0.1 unlinkd
13314 admin    S        56  128  0.0  0.1 smbd
  181 nobody  S        52  172  0.0  0.1 redirector.sh
20858 admin    S        52    58  0.0  0.1 sh
  182 nobody  S        48  172  0.0  0.1 redirector.sh
  183 nobody  S        48  172  0.0  0.1 redirector.sh
  184 nobody  S        48  172  0.0  0.1 redirector.sh
  185 nobody  S        48  172  0.0  0.1 redirector.sh
  187 nobody  S        48  172  0.0  0.1 redirector.sh
  188 nobody  S        48  172  0.0  0.1 redirector.sh
  189 nobody  S        48  172  0.0  0.1 redirector.sh
  190 nobody  S        48  172  0.0  0.1 redirector.sh
  191 nobody  S        48  172  0.0  0.1 redirector.sh
  192 nobody  S        48  172  0.0  0.1 redirector.sh
  193 nobody  S        48  172  0.0  0.1 redirector.sh
  124 admin    S        32    1  0.0  0.1 vsftpd
  66 admin    S        28    1  0.0  0.0 nas
  69 admin    S        28    1  0.0  0.0 klogd
  97 admin    S        16    94  0.0  0.0 ntp
  167 admin    S          8  165  0.0  0.0 php-fcgi
  152 admin    S          8    1  0.0  0.0 php-fcgi
  206 admin    S          8  167  0.0  0.0 php-fcgi
  207 admin    S          8  167  0.0  0.0 php-fcgi
  205 admin    S          8  152  0.0  0.0 php-fcgi
  128 admin    S          4    1  0.0  0.0 smbd
    4 admin    SW        0    1  0.0  0.0 kswapd
[admin@W500 root]$

---

è ïîäîëãó....
êàæäûå 5 ìèíóò ïðîöåññ åñò 80-90% ÖÏ îêîëî 2-õ ìèíóò!

Quote:

---

Originally Posted by serji

lightparser.pl ñèëüíî ãðóçèò ñèñòåìó!
...
è ïîäîëãó....
êàæäûå 5 ìèíóò ïðîöåññ åñò 80-90% ÖÏ îêîëî 2-õ ìèíóò!

---

À Âû ïîñòàâòå ðàç â ÷àñ, 5 èëè 24.
Âàì àêòóàëüíà ðèàë òàéì ñòàòèñòèêà?

Ï.Ñ. äëÿ òåêóùèõ ñîåäèíåíèé ìîæíî ïîëüçîâàòü sqstat 1.2 sqstat

ÿ ïîäóìàë, ÷åì ÷àùå îí ðàáîòàåò, òåì ìåíüøå îáúåì âû÷èñëåíèé åìó ïðîèçâîäèòü, òåì áûñòðåå (ìåíüøå ïî âðåìåíè) îí äîëæåí ðàáîòàòü

Quote:

---

Originally Posted by Less

1.
Çàìåíèòå íà

PHP Code:

---

acl QUERY urlpath_regex cgi-bin \?
no_cache deny QUERY 


---

2.
Âñòàâòå ýòè ñòðîêè

PHP Code:

---

acl MyHOST src IP/255.255.255.255
http_access allow MyHOST 


---

ïåðåä ñòðîêàìè

PHP Code:

---

http_access allow MY 


---

èëè æå Âàøåé ïåðâîé ñòðîêîé
http_access allow
ãäå âû ðàçðåøàåòå ïåðâîå ïðàâèëî êîòîðîìó íåîáõîäèìà àâòîðèçàöèÿ.

---

Óáðàë âîîáùå ïàðó ñòðîê, ïðî àâòîðèçàöèÿ.

[admin@WL500 share]$ cat /opt/etc/squid/squid.conf
#auth_param basic program /opt/libexec/ncsa_auth /opt/share/squid
#auth_param basic children 1
#auth_param basic realm Authorized users only. Squid PROXY
#auth_param basic credentialsttl 2 hours

#acl MY proxy_auth REQUIRED
acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl home src 172.16.0.0/255.255.0.0
acl to_localhost dst 172.16.0.0/16
acl Safe_ports port 21 # ftp
acl CONNECT method CONNECT
acl banners url_regex "/opt/share/squid/banners.acl"

http_access deny banners
http_access allow home

http_port 3128

hierarchy_stoplist cgi-bin ?

cache_dir ufs /opt/var/squid/cache 20 16 256

cache_swap_low 97
cache_swap_high 100

access_log /opt/var/squid/logs/access.log squid
pid_filename /opt/var/squid/logs/squid.pid

#acl QUERY urlpath_regex cgi-bin ?
#no_cache deny QUERY

refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern . 0 20% 4320

acl apache rep_header Server ^Apache
broken_vary_encoding allow apache

header_access From deny all
header_access Referer deny all
header_access User-Agent deny all
header_access Cache-Control deny all
header_access Via deny all
header_access X-Forwarded-For deny all

cache_effective_user nobody
cache_effective_group nobody

visible_hostname host

icon_directory /opt/share/squid/icons
error_directory /opt/share/squid/errors/Russian-1251

cachemgr_passwd 456789 all

coredump_dir /opt/var/squid/cache


Ðàáîòàåò, íî.
Åñëè ñäåëàòü.
iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-ports 3128
ðåäèðåêò, òî âûâàëèâàåòñÿ îøèáêà â web

ÎØÈÁÊÀ
Çàïðîøåííûé URL íå ìîæåò áûòü äîñòàâëåí.

--------------------------------------------------------------------------------

Âî âðåìÿ îáðàáîòêè çàïðîñà:

GET /index.php?option=com_planbafisha HTTP/1.1
Accept: */*
Referer: http://www.planbclub.ru/index.php?op...&id=5&Itemid=6
Accept-Language: ru
UA-CPU: x86
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 3.0.04506)
Host: www.planbclub.ru
Connection: Keep-Alive
Cookie: cfd9011911ae0506a6141b043a06539a=58d823048bba4197b 26ff0d570578983; fontSize=100


Ïðîèçîøëà ñëåäóþùàÿ îøèáêà:

Íåâåðíûé çàïðîñ.
Íåêîòîðûå àñïåêòû HTTP çàïðîñà íåïðàâèëüíû. Âîçìîæíûå ïðîáëåìû:

Îòñóòñòâóåò ëèáî íåèçâåñòåí ìåòîä çàïðîñà (GET, POST)
Îòñóòñòâóåò URL
Îòñóòñòâóåò HTTP èäåíòèôèêàòîð (HTTP/1.0)
Çàïðîñ ñëèøêîì âåëèê
Íå óêàçàí Content-Length äëÿ çàïðîñîâ POST èëè PUT
Íåäîïóñòèìûé ñèìâîë â èìåíè ñåðâåðà; ïîä÷åðêèâàíèÿ íåäîïóñòèìû

×òî ýòî ìîæåò áûòü.
Ïðàâèëüíî ëè âîîáùå óáèðàòü çàêîìåíòèðîâàííûå ó ìåíÿ ïàðàìåòðû.

Quote:

---

Originally Posted by StaREViL

Óáðàë âîîáùå ïàðó ñòðîê, ïðî àâòîðèçàöèÿ.

---

Ïîðûëñÿ â èíåòå. Íàøåë òîëüê ÷òî íàäî âêëþ÷èòü
httpd_accel_host virtual
httpd_accel_port 80
httpd_accel_with_proxy on
httpd_accel_uses_host_header on

Íî ñêâèä ðóãàåòñÿ, íå ïîíèìàåò ÷òî ýòî çà ñòðîêè òàêèå...

Äëÿ ïðîçðà÷íîãî ïðîêñèðîâàíèÿ íóæíî íå
http_port 3128
à
http_port 3128 transparent
Òîãäà ìîæíî ðåäèðåêò ñòàâèòü.