Ýêîíîìèÿ òðàôèêà (Privoxy + Polipo)

[oleg71]

À âîò òàê ìîæíî íàâåðíî ïîïðîáîâàòü:
http://ylsoftware.com/news/450

Ñïàñèáî (íî ýòî ÿ óæå íàõîäèë)! ×èòàë ÿ òàì, ÷èòàë...
Íî äóìàþ, êòî-òî æ ïîëüçóåòñÿ íà ðîóòå èëè íåò?
Òÿæåëî (ïîêà) ïåðåæåâûâàòü èíñòðóêöèè äëÿ ÁÁ è Á(L)
(Áîëüøîé Linux ) ïðèìåíèòåëüíî ê ðîóòó

×òî-òî ïðîñìàòðèâàåòñÿ â âèäå

Privoxy + Polipo

êòî ïîìîæåò ñ íàñòðîéêàìè?

Íå ïðîõîäèòå ìèìî!

Âåäü ïîëüçóåìñÿ, íàðîä... è ìîë÷èì

ïîäñêàæèòå, íàñòðîéêè îò ïðîã, ñòîÿùèõ íà ÁÁ (Linux)
ïîäîéäóò ëè äëÿ èñïîëüçîâàíèÿ íà ðîóòå ñ òåìè æå ïðîãàìè?

Ñ íàñòðîéêîé polipo ðàçîáðàëñÿ, âñå ðàáîòàåò, ÂÑÅÌ ñïàñèáî!

êàê ïðèêðóòèòü ê íåìó privoxy
èëè îòêîìïèëèòü bfilter

[oleg71]

Ïðîêñè äëÿ îôèñà ñ ýâðèñòè÷åñêèì ôèëüòðîì ðåêëàìû!
Íó íåóæåëè íèêòî íå ïîìîãåò

Äà ãóðó ñêîìïèëèòü ýòî äåëî, ïàðà ïóñòÿêîâ è íîâûé êîâðèê ìûøè

íó âèäèìî ïîêà íåêîãäà...

"Äà-äà. Èìåííî òàê. Ïðîãðàììû, ïîçâîëÿþùèå îòñåèâàòü ðåêëàìó èñïîëüçóÿ èíòåëëåêòóàëüíûå àëãîðèòìû âìåñòî ÷¸ðíûõ ñïèñêîâ ñóùåñòâóþò!"
BFilter - filtering web proxy
 îòëè÷èè îò äðóãèõ ðåøåíèé, â BFilter èñïîëüçóþòñÿ ýâðèñòè÷åñêèå ìåòîäû âûÿâëåíèÿ ðåêëàìíûõ áëîêîâ. Ìîæíî îòìåòèòü, íàëè÷èå âñòðîåííîãî javascript ïàðñåðà, àíàëèçàòîðà Flash ðîëèêîâ, ïîääåðæêó ñæàòîãî êîíòåíòà, âîçìîæíîñòü ïåðåñûëêè çàïðîñîâ ê äðóãîìó ïðîêñè ñåðâåðó."

[Nebulosa]

Ñêîìïèëèðîâàòü òî íàâåðíîå ìîæíî...
Ñóäÿ ïî http://bfilter.sourceforge.net/doc/build-unix.php äëÿ íå GUI âåðñèè íóæíû

• ACE >= 5.4
• libsigc++ 2.x.x - óðà! ýòî åñòü â ðåïàçèòîðèè
• zlib (any version) - òîæå óðà! òîæå åñòü â ðåïàçèòîðèè.

Ñëåäîâàòåëüíî íóæíî ñïåðâà ðàçîáðàòüñÿ ñ ACE -
http://www.dre.vanderbilt.edu/~schmi...#unix_autoconf

Ïîïðîáóåì...

[Ukaz]

Ñ íàñòðîéêîé polipo ðàçîáðàëñÿ, âñå ðàáîòàåò, ÂÑÅÌ ñïàñèáî!

êàê ïðèêðóòèòü ê íåìó privoxy
èëè îòêîìïèëèòü bfilter

Ïîäåëèòåñü êàê íàñòðîèëîñü âñå è åñòü ëè ðåàëüíàÿ ýêîíîìèÿ òðàôèêà çà ñ÷åò ïðîêñè. Ñïàñèáî

[oleg71]

Ïîäåëèòåñü êàê íàñòðîèëîñü âñå è åñòü ëè ðåàëüíàÿ ýêîíîìèÿ òðàôèêà çà ñ÷åò ïðîêñè. Ñïàñèáî

Êîãäà ÿ ñêàçàë çíàêîìîìó ëèíóêñîèäó ÷òî ó ìåíÿ íà ðîóòå
ñòîÿë squid ó íåãî áðîâè àæ íà ïðè÷åñêó çàëåçëè...
Ýòî ÿ ê òîìó, ÷òî squid ìíîãî æðåò ðåñóðñîâ è ïàìÿòè,
åãî âîîáùå-òî ïî-âçðîñëîìó âûäåëÿþò â îòäåëüíûé ñåðâåð

Ðàñêàçàòü êàê âñå íàñòðîèëîñü - âçÿë ñ ãóãëà êîíôèã polipo
à è íàñòðîèë .
ñ íèì ïîêà íèêàêîé ñòàòèñòèêè íåò.
Êîãäà ñòîÿë squid - ïðèòîðìàæèâàëî, íî ðàáîòàëî,
ñòðàíè÷êè ãðóçèëèñü áûñòðî (ìèëèñåêóíäíàÿ çàäåðæêà
è ïåðåä òîáîé âñÿ ñòðàíèöà, à íå ïî ÷àñòÿì ïîÿâëÿþùàÿñÿ
êýø ïîáîëüøå è ãóãë ïî îïòèìèçàöèè...
Èìõî, îáúÿñíÿòü ðàáîòó êýøèðóþùåãî ïðîêñè ãðàìîòíî âðÿäëè ñìîãó,
à âîò èç ïî÷èòàííîãî èçâåñòíî, ÷òî áðàòü èç êýøà íà ëîêàëüíîì
äèñêå âñåãäà äåøåâëå ÷åì èç èíåòà êàæäûé ðàç ïîñåùàÿ
îïðåäåëåííûé êðóã ñàéòîâ (èíòåðåñîâ), ïðè òàðèôå ïî òðàôèêó.

Ïðèìåð íàñòðîéêè polipo (îäèí èç ñîòåí ïî ãóãëó) âîò

[oleg71]

Ñêîìïèëèðîâàòü òî íàâåðíîå ìîæíî...
Ñóäÿ ïî http://bfilter.sourceforge.net/doc/build-unix.php äëÿ íå GUI âåðñèè íóæíû

• ACE >= 5.4
• libsigc++ 2.x.x - óðà! ýòî åñòü â ðåïàçèòîðèè
• zlib (any version) - òîæå óðà! òîæå åñòü â ðåïàçèòîðèè.

Ñëåäîâàòåëüíî íóæíî ñïåðâà ðàçîáðàòüñÿ ñ ACE -
http://www.dre.vanderbilt.edu/~schmi...#unix_autoconf

Ïîïðîáóåì...

Î÷åíü íàäåþñü , ÷òî ó âàñ, ñïåöîâ, âñå ïîëó÷èòñÿ!!!
ìîãó òîëüêî òåñòèðîâàòü (íàäåþñü, ÷òî ïîêà),
ýòî î÷åíü èíòåðåñíàÿ è íóæíàÿ ïðîãà â íàøåé êîïèëêå!

[Nebulosa]

Êîãäà ÿ ñêàçàë çíàêîìîìó ëèíóêñîèäó ÷òî ó ìåíÿ íà ðîóòå
ñòîÿë squid ó íåãî áðîâè àæ íà ïðè÷åñêó çàëåçëè...
Ýòî ÿ ê òîìó, ÷òî squid ìíîãî æðåò ðåñóðñîâ è ïàìÿòè,
åãî âîîáùå-òî ïî-âçðîñëîìó âûäåëÿþò â îòäåëüíûé ñåðâåð

Ììì... Íó à êðîìå òîãî ÷òî îí ïðîêñè polipo åùå ÷òî-íèáóäü óìååò? Ñàìîå ìèíèìàëüíîå õîòåëîñü áû ÷òîáû ïî ñïèñêó URL îí áëîêèðîâàë è âìåñòî íåãî îòïðàâëÿë íà äðóãîé óðë ñ ïóñòîé ãèôêîé.

À ïðî squid çðÿ âû òàê, âïîëíå ñåáå æèâåò íà ðîóòåðå íåâûêëþ÷àÿñü ìåñÿöàìè + åùå ïàðàëëåëüíî íèòè äëÿ ôèëüòðàöèè ðåêëàìû âèñÿò.. ìîãó öèôðû ïðèâåñòè êàêèå âàì íóæíî, ïî íàêëàäíûì ðàñõîäàì.. â ïîäïèñè ó ìåíÿ åñòü ññûëî÷êà âïðî÷åì..

Âû ýòîìó ëèíóêñîèäó ïðî nginx+php íå ðàññêàçûâàéòå (÷òî ïàðàëëåëüíî ñî ñêâèäîì íà ðîóòåðå ðàáîòàåò) à òî áîþñü åìó ñåðäöåì áóäåò ïëîõî. À ó ñåðâà÷êà òî.. 2000+ õîñòîâ (íå õèòîâ!) åæåñóòî÷íî..
Åùå ïðàâäà áûëà áàçà mysql, íî êàê ïîëüçîâàòåëåé ñòàëî áîëüøå 50 joomla ïîìèðàòü íà÷àëà.. ïåðåâåë åå íà "îòäåëüíûé ñåðâåð"

[oleg71]

http://www.pps.jussieu.fr/~jch/software/polipo/

[oleg71]

Ììì... Íó à êðîìå òîãî ÷òî îí ïðîêñè polipo åùå ÷òî-íèáóäü óìååò? Ñàìîå ìèíèìàëüíîå õîòåëîñü áû ÷òîáû ïî ñïèñêó URL îí áëîêèðîâàë è âìåñòî íåãî îòïðàâëÿë íà äðóãîé óðë ñ ïóñòîé ãèôêîé.

À ïðî squid çðÿ âû òàê, âïîëíå ñåáå æèâåò íà ðîóòåðå íåâûêëþ÷àÿñü ìåñÿöàìè + åùå ïàðàëëåëüíî íèòè äëÿ ôèëüòðàöèè ðåêëàìû âèñÿò.. ìîãó öèôðû ïðèâåñòè êàêèå âàì íóæíî, ïî íàêëàäíûì ðàñõîäàì.. â ïîäïèñè ó ìåíÿ åñòü ññûëî÷êà âïðî÷åì..

Âû ýòîìó ëèíóêñîèäó ïðî nginx+php íå ðàññêàçûâàéòå (÷òî ïàðàëëåëüíî ñî ñêâèäîì íà ðîóòåðå ðàáîòàåò) à òî áîþñü åìó ñåðäöåì áóäåò ïëîõî. À ó ñåðâà÷êà òî.. 2000+ õîñòîâ (íå õèòîâ!) åæåñóòî÷íî..
Åùå ïðàâäà áûëà áàçà mysql, íî êàê ïîëüçîâàòåëåé ñòàëî áîëüøå 50 joomla ïîìèðàòü íà÷àëà.. ïåðåâåë åå íà "îòäåëüíûé ñåðâåð"

Äà ðàáîòàåò òî îí (squid) - ðàáîòàåò, òîëüêî áåç ñâîïà ðîóòåð
ñðàçó çàãèáàëñÿ... È ó ìåíÿ îí ðàáîòàë (ïîâòîðÿþ).
À òî ÷òî íàñòàâèòü â ðîóò ìîæíî ïî ñàìûå óøè, ïîæàëóéñòà,
òîëüêî íå åõàòü ýòî áóäåò, à ïîëçòè íà ïðåäåëå...

Âîîáùå, ðàçãîâîð îá ýêîíîìèè òðàôèêà, âåñüìà ïîëåçåí
ïîëüçîâàòåëÿì òàðèôà ïî òðàôèêó, êîòîðûõ èìõî ñåé÷àñ ìåíüøå,
÷åì áåçëèìèò÷èêîâ , íî îíè ïîêà ÅÑÒÜ, "çíà÷èò íàñ áóäóò
ëþáèòü ïðîâàéäåðû, íàêðó÷èâàÿ òðàôèê, à ìû áóäåì ëþáèòü
ïðîâàéäåðîâ óñòàíàâëèâàÿ êýøèðóþùèå ïðîêñè è ðåçàêè ðåêëàìû"


"Ïðîãðàììû, ïîçâîëÿþùèå îòñåèâàòü ðåêëàìó èñïîëüçóÿ èíòåëëåêòóàëüíûå àëãîðèòìû âìåñòî ÷¸ðíûõ ñïèñêîâ ñóùåñòâóþò!"
polipo + BFilter - 1-é äåëàåò êýø, 2-é ðåæåò ðåêëàìó!
 îòëè÷èè îò äðóãèõ ðåøåíèé, â BFilter èñïîëüçóþòñÿ ýâðèñòè÷åñêèå ìåòîäû âûÿâëåíèÿ ðåêëàìíûõ áëîêîâ. Ìîæíî îòìåòèòü, íàëè÷èå âñòðîåííîãî javascript ïàðñåðà, àíàëèçàòîðà Flash ðîëèêîâ, ïîääåðæêó ñæàòîãî êîíòåíòà, âîçìîæíîñòü ïåðåñûëêè çàïðîñîâ ê äðóãîìó ïðîêñè ñåðâåðó."

[oleg71]

Ïîäñêàæèòå, ïðàâèëüíî ëè:
(åñëè êòî ïîìîæåò ñ ðåàëèçàöèåé (íàñòðîéêè) áóäó ìíîãî áëàãîäàðåí)

[vectorm]

Ïðàâèëüíî.
Òîëüêî äëÿ ïîäîáíûõ ñõåì ëó÷øå âíóòðåííèå êîííåêòû äåëàòü ïî ëîêàëõîñòó, íå ïî âíóòðèëàíîñêîìó IP - òàê â íàñòðîéêàõ ìîæåî âîîáùå âòîðîé ïðîêñè íà LAN íå ñâåòèòü.

Åùå âàðèàíòû (âñå ñ êëèåíòîâ ÷åðåç ïðîêñè ïðîïóñêàòü ïðèíóäèòåëüíî):
1. íà âõîäå òðàíñïàðåíò ïðîêñè, íà âûõîäå polipo (îí òðàíñïàðåíòîì ðàáîòàòü íå óìååò).
2. Âñå çàïðîñû íà èçâåñòíûå ïîðòû ðîóòèòü íà my.router:8123, ñìîòðÿùèé íà âòîðîé ïðîêñè.

ïåðâûé âàðèàíò äîëæåí áûòü ðàáî÷èì, âòîðîé íå ïðîáîâàë.

[oleg71]

Ïðàâèëüíî.
Òîëüêî äëÿ ïîäîáíûõ ñõåì ëó÷øå âíóòðåííèå êîííåêòû äåëàòü ïî ëîêàëõîñòó, íå ïî âíóòðèëàíîñêîìó IP - òàê â íàñòðîéêàõ ìîæåî âîîáùå âòîðîé ïðîêñè íà LAN íå ñâåòèòü.

Åùå âàðèàíòû (âñå ñ êëèåíòîâ ÷åðåç ïðîêñè ïðîïóñêàòü ïðèíóäèòåëüíî):
1. íà âõîäå òðàíñïàðåíò ïðîêñè, íà âûõîäå polipo (îí òðàíñïàðåíòîì ðàáîòàòü íå óìååò).
2. Âñå çàïðîñû íà èçâåñòíûå ïîðòû ðîóòèòü íà my.router:8123, ñìîòðÿùèé íà âòîðîé ïðîêñè.

ïåðâûé âàðèàíò äîëæåí áûòü ðàáî÷èì, âòîðîé íå ïðîáîâàë.

1. Ìîæåò èëè ýòî íå òî?

Can Polipo behave as a transparent proxy?
Polipo is transparent if you set the following in your config file:

maxAge = 0
maxExpiresAge = 0
# Ðàñêîììåíòèðóéòå, åñëè âû õîòèòå èñïîëüçîâàòü ðîäèòåëüñêèé ïðîêñè:
# ParentProxy = "squid.example.org: 3128"
# Ðàñêîììåíòèðóéòå, åñëè âû õîòèòå èñïîëüçîâàòü ðîäèòåëüñêèé ïðîêñè SOCKS:
# SocksParentProxy = "ëîêàëüíûé: 9050" # socksProxyType = socks5

Òîëüêî ñàì åùå íå ïðîáîâàë, ïîêà ðàçáèðàþñü.

Çíà÷èò, ðîäèòåëüñêèé ïðîêñè äëÿ polipo ïîñòàâèòü 127.0.0.1:3128 ?
à â êîíôèãóðàöèè privoxy ïîñòàâèòü èñïîëüçîâàòü äëÿ ñåáÿ
ip 127.0.0.1 ïîòð 3128 ?

Òàê?__________

Ìîæåò ÷òî-òî ÿ íåäîïîíèìàþ

Èç èíåòà òðàôèê ïîñòóïàåò íà privoxy ãäå
ðåæåòñÿ ðåêëàìà è ìóñîð, çàòåì èäåò íà polipo
ãäå êýøèðóåòñÿ íà äèñêå è ïîäàåòñÿ êëèåíòó.

Õîòåëîñü áû ÷òîá òàê ñðàáîòàëî, âîçìîæíî?

[vectorm]

Òàê?__________

Äà, òàê.

Ïîëíîñòüþ ïðîçðà÷íûé - èìååòñÿ â âèäó ýòî:

Can Polipo behave as an intercepting proxy?

No.

Interception proxying (sometimes confusingly called ‘‘transparent’’ proxying) is a technique that intercepts client connections at the network layer in order to redirect them at an application layer proxy.

Interception proxying is a fundamentally broken design (see for example this posting and RFC 3143, Section 2.2.2), and will not be supported by Polipo. If you want to use interception proxying in order to avoid manually configuring your clients, please use Netscape's proxy.pac auto-configuration (you don't need a separate web server, as a proxy.pac file can be served by Polipo's internal server). If you want to use interception proxying for any other reason, you're probably doing something wrong.

Èìåííî òàê óêàçàííàÿ êîíôèãóðàöèÿ è áóäåò ðàáîòàòü.

[oleg71]

Äà, òàê.

Ïîëíîñòüþ ïðîçðà÷íûé - èìååòñÿ â âèäó ýòî:

Can Polipo behave as an intercepting proxy?

No.

Interception proxying (sometimes confusingly called ‘‘transparent’’ proxying) is a technique that intercepts client connections at the network layer in order to redirect them at an application layer proxy.

Interception proxying is a fundamentally broken design (see for example this posting and RFC 3143, Section 2.2.2), and will not be supported by Polipo. If you want to use interception proxying in order to avoid manually configuring your clients, please use Netscape's proxy.pac auto-configuration (you don't need a separate web server, as a proxy.pac file can be served by Polipo's internal server). If you want to use interception proxying for any other reason, you're probably doing something wrong.

Èìåííî òàê óêàçàííàÿ êîíôèãóðàöèÿ è áóäåò ðàáîòàòü.

Ñïàñèáî !!!

Çàéìóñü íàñòðîéêàìè

[oleg71]

Òðåáóåòñÿ äëÿ privoxy libz.so.1.o
×òî ýòî è êàê ýòî ïîñòàâèòü???