NAS: Create your own caching proxy

[vinyols]

There you are, with that 1TB NAS and you surf mostly the same websites and in the process waste plenty of time waiting on downloads. So why not install your own Squid-proxy server on your NAS?

With the Synology and the pre-requisite of having ipkg installed - this takes no more than 10 minutes. In my example, my NAS IP is 172.16.0.97 and my IP range on my LAN is 172.16.0.0 - adjust this accordingly below:

1. Install squid: ipkg install squid
2.

Adjust Squid's config-file located in /opt/etc/squid/squid.conf:

code:

## SQUID CONFIG
cache_mgr Gerd@Naschenweng.info

## Those are the ports the proxy is going to listen to
http_port 172.16.0.97:3128
http_port 172.16.0.97:8080

# TAG: visible_hostname
# The host-name of the proxy-server. Can really be anything
visible_hostname MuffinStationProxy

# DISK CACHE OPTIONS
# -----------------------------------------------------------------------------
# Disk-cache options. Just adjust the cache-siz (in my case 20GB)
cache_replacement_policy lru
cache_dir ufs /opt/var/squid/cache/ 20000 16 256
minimum_object_size 0 KB
maximum_object_size 2097152 KB
maximum_object_size_in_memory 1024 KB

# MEMORY CACHE OPTIONS
# -----------------------------------------------------------------------------
# TAG: cache_mem (bytes)
cache_mem 8 MB
memory_replacement_policy lru

# ACCESS CONTROLS
# -----------------------------------------------------------------------------
acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl our_networks src 172.16.0.0/24 172.16.1.0/24
acl to_localhost dst 127.0.0.0/8

http_access allow manager localhost
http_access allow manager our_networks
http_access deny manager

# Allow all clients from my network
http_access allow our_networks

# And finally deny all other access to this proxy
http_access deny all

#Allow ICP queries from everyone
icp_access allow all



# LOG-FILES
# -----------------------------------------------------------------------------
access_log /opt/var/squid/logs/access.log squid

#cache_log none
#cache_log /opt/var/squid/logs/cache.log

#cache_access_log none
#cache_access_log /opt/var/squid/logs/access.log

#cache_store_log none
#cache_store_log /opt/var/squid/logs/store.log


# OPTIONS FOR TUNING THE CACHE
# -----------------------------------------------------------------------------

# TAG: cache
# A list of ACL elements which, if matched, cause the request to
# not be satisfied from the cache and the reply to not be cached.
# In other words, use this to force certain objects to never be cached.
#
# You must use the word 'DENY' to indicate the ACL names which should
# NOT be cached.
#
# Default is to allow all to be cached

#We recommend you to use the following two lines.
acl QUERY urlpath_regex cgi-bin \?
cache deny QUERY
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern . 0 20% 4320
refresh_pattern \.gif 1440 50% 40320 reload-into-ims
refresh_pattern \.jpg 1440 50% 40320 reload-into-ims
refresh_pattern \.tif 4320 50% 43200
refresh_pattern \.png 1440 50% 40320 reload-into-ims
refresh_pattern \.jpeg 1440 50% 40320 reload-into-ims
refresh_pattern ^http://*.google.*/.* 720 100% 4320

# refresh patterns to enable caching of MS windows update
refresh_pattern windowsupdate\.microsoft\.com/.*\.(cab|exe|psf) 4320 100% 120960 reload-into-ims
refresh_pattern update\.microsoft\.com/.*\.(cab|exe|psf) 4320 100% 120960 reload-into-ims
refresh_pattern office\.microsoft\.com/.*\.(cab|exe|psf) 4320 100% 120960
refresh_pattern windowsupdate\.com/.*\.(cab|exe|psf) 4320 100% 120960 reload-into-ims
refresh_pattern download\.microsoft\.com/.*\.(cab|exe|psf) 4320 100% 120960 reload-into-ims
refresh_pattern microsoft\.com 4320 100% 10080


pipeline_prefetch on
# Apache mod_gzip and mod_deflate known to be broken so don't trust
# Apache to signal ETag correctly on such responses
acl apache rep_header Server ^Apache
broken_vary_encoding allow apache

# Leave coredumps in the first cache dir
coredump_dir /opt/var/squid/cache

# Disable cachemgr password
cachemgr_passwd none all


-----------------------------------------------------------------------
3. Take note from my above config, that I chose a cache-size of 20(!) GB (cache_dir).
4. Validate your Squid configuration with squid -k parse
5. Create the Squid cache-directories with squid -z
6. Start Squid manually to check for errors: squid -NCd1
7. Create a symbolic link so that Squid starts automatically: ln -s /opt/etc/init.d/S80squid /usr/syno/etc/rc.d/
8. Once you restart the NAS, Squid should be started automatically (log files are in /opt/var/squid/logs)

Dummy error: Happened to me - if Squid starts and you don't notice any improvements in browsing speed, make sure that you have your browser's proxy settings adjusted ops:

IMPORTANT: As I have the caching server within a DMZ/Firewall, security-concerns are secondary. All users having access to the LAN and fall within the IP-range will automatically have access to the caching-proxy. The implementation of Squid was for improving the browsing/web-experience (speed has improved by almost 200% and average bandwidth consumption dropped by 30%).

Follow up:

If you get everything running, you should familiarise yourself with the statuses in Squid's access-log:

* TCP_HIT: A valid copy of the requested object was in the cache.
* TCP_MEM_HIT: A valid copy of the requested object was in the cache, AND it was in memory so it did not have to be read from disk.
* TCP_NEGATIVE_HIT: The request was for a negatively-cached object. Negative-caching refers to caching certain types of errors, such as "404 Not Found." The amount of time these errors are cached is controlled with the negative_ttl configuration parameter.
* TCP_MISS: The requested object was not in the cache.
* TCP_REFRESH_HIT: The object was in the cache, but STALE. An If-Modified-Since request was made and a "304 Not Modified" reply was received.
* TCP_REF_FAIL_HIT: The object was in the cache, but STALE. The request to validate the object failed, so the old (stale) object was returned.
* TCP_REFRESH_MISS: The object was in the cache, but STALE. An If-Modified-Since request was made and the reply contained new content.
* TCP_CLIENT_REFRESH: The client issued a request with the "no-cache" pragma.
* TCP_IMS_HIT: The client issued an If-Modified-Since request and the object was in thecache and still fresh.




thanks naschenweng.info

[wpte]

yeh, have been thinking of doing that myself... but
I was afraid you would get outdated pages
do you get any of those?

tell me, what kinda internet speed do you have? might try it myself

[wpte]

I tried installing everything, and everything is ok so far...
BUT
I get errors:

2008/12/05 11:36:02| NOTICE: maximum_object_size limited to 4194240 KB due to OS limitations
2008/12/05 11:36:02| Starting Squid Cache version 2.6.STABLE21 for mipsel-unknown-linux-gnu...
2008/12/05 11:36:02| Process ID 2546
2008/12/05 11:36:02| With 256 file descriptors available
2008/12/05 11:36:02| Using poll for the IO loop
2008/12/05 11:36:02| Performing DNS Tests...
2008/12/05 11:36:02| Successful DNS name lookup tests...
2008/12/05 11:36:02| DNS Socket created at 0.0.0.0, port 1123, FD 5
2008/12/05 11:36:02| Adding nameserver 192.168.1.1 from /etc/resolv.conf
2008/12/05 11:36:02| Adding nameserver 0.0.0.0 from /etc/resolv.conf
2008/12/05 11:36:02| WARNING: Squid does not accept 0.0.0.0 in DNS server specifications.
2008/12/05 11:36:02| Will be using 127.0.0.1 instead, assuming you meant that DNS is running on the same machine
2008/12/05 11:36:02| Unlinkd pipe opened on FD 10
2008/12/05 11:36:02| Swap maxSize 20480000 + 8192 KB, estimated 0 objects
2008/12/05 11:36:02| Target number of buckets: 78800
2008/12/05 11:36:02| Using 131072 Store buckets
2008/12/05 11:36:02| Max Mem size: 8192 KB
2008/12/05 11:36:02| Max Swap size: 20480000 KB
2008/12/05 11:36:02| Rebuilding storage in /mnt/cache/ (DIRTY)
2008/12/05 11:36:02| Using Least Load store dir selection
2008/12/05 11:36:02| chdir: /opt/var/squid/cache: (2) No such file or directory
2008/12/05 11:36:02| Current Directory is /tmp/local/root
2008/12/05 11:36:02| Loaded Icons.
2008/12/05 11:36:02| commBind: Cannot bind socket FD 11 to 172.16.0.97:3128: (126) Cannot assign requested address
2008/12/05 11:36:02| commBind: Cannot bind socket FD 11 to 172.16.0.97:8080: (126) Cannot assign requested address
FATAL: Cannot open HTTP Port
Aborted

I have my router on 192.168.2.1 instead 1.1
but what are these 172.16.0.87 adresses, it doesnt make any sense to me

[Serpent]

Just read this:

.....
In my example, my NAS IP is 172.16.0.97 and my IP range on my LAN is 172.16.0.0 - adjust this accordingly below:

2. Adjust Squid's config-file located in /opt/etc/squid/squid.conf:

code:

## SQUID CONFIG

## Those are the ports the proxy is going to listen to
http_port 172.16.0.97:3128
http_port 172.16.0.97:8080
.....

[wpte]

Just read this:

thanks serpent, I got it working now
it goes pretty fast actually

[vinyols]

thanks serpent, I got it working now
it goes pretty fast actually

Please paste your squid.conf
thank you

[wpte]

Please paste your squid.conf
thank you

Got it working now tho, but I'm trying to get dansguardian working with it, so you can block the pop-ups etc.

## SQUID CONFIG
cache_mgr myhidden@email-adress.com

## Those are the ports the proxy is going to listen to
http_port 192.168.2.1:3128
http_port 192.168.2.1:8081

# TAG: visible_hostname
# The host-name of the proxy-server. Can really be anything
visible_hostname router

# DISK CACHE OPTIONS
# -----------------------------------------------------------------------------
# Disk-cache options. Just adjust the cache-size
cache_replacement_policy lru
cache_dir ufs /mnt/cache/ 20000 16 256
minimum_object_size 0 KB
maximum_object_size 5000000 KB
maximum_object_size_in_memory 1024 KB

# MEMORY CACHE OPTIONS
# -----------------------------------------------------------------------------
# TAG: cache_mem (bytes)
cache_mem 8 MB
memory_replacement_policy lru

# ACCESS CONTROLS
# -----------------------------------------------------------------------------
acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl our_networks src 192.168.1.0/24 192.168.2.0/24
acl to_localhost dst 127.0.0.0/8

http_access allow manager localhost
http_access allow manager our_networks
http_access deny manager

# Allow all clients from my network
http_access allow our_networks

# And finally deny all other access to this proxy
http_access deny all

#Allow ICP queries from everyone
icp_access allow all



# LOG-FILES
# -----------------------------------------------------------------------------
access_log /opt/var/squid/logs/access.log squid

#cache_log none
#cache_log /opt/var/squid/logs/cache.log

#cache_access_log none
#cache_access_log /opt/var/squid/logs/access.log

#cache_store_log none
#cache_store_log /opt/var/squid/logs/store.log


# OPTIONS FOR TUNING THE CACHE
# -----------------------------------------------------------------------------

# TAG: cache
# A list of ACL elements which, if matched, cause the request to
# not be satisfied from the cache and the reply to not be cached.
# In other words, use this to force certain objects to never be cached.
#
# You must use the word 'DENY' to indicate the ACL names which should
# NOT be cached.
#
# Default is to allow all to be cached

#We recommend you to use the following two lines.
acl QUERY urlpath_regex cgi-bin \?
cache deny QUERY
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern . 0 20% 4320
refresh_pattern \.gif 1440 50% 40320 reload-into-ims
refresh_pattern \.jpg 1440 50% 40320 reload-into-ims
refresh_pattern \.tif 4320 50% 43200
refresh_pattern \.png 1440 50% 40320 reload-into-ims
refresh_pattern \.jpeg 1440 50% 40320 reload-into-ims
refresh_pattern ^http://*.google.*/.* 720 100% 4320

# refresh patterns to enable caching of MS windows update
refresh_pattern windowsupdate\.microsoft\.com/.*\.(cab|exe|psf) 4320 100% 120960 reload-into-ims
refresh_pattern update\.microsoft\.com/.*\.(cab|exe|psf) 4320 100% 120960 reload-into-ims
refresh_pattern office\.microsoft\.com/.*\.(cab|exe|psf) 4320 100% 120960
refresh_pattern windowsupdate\.com/.*\.(cab|exe|psf) 4320 100% 120960 reload-into-ims
refresh_pattern download\.microsoft\.com/.*\.(cab|exe|psf) 4320 100% 120960 reload-into-ims
refresh_pattern microsoft\.com 4320 100% 10080


pipeline_prefetch on
# Apache mod_gzip and mod_deflate known to be broken so don't trust
# Apache to signal ETag correctly on such responses
acl apache rep_header Server ^Apache
broken_vary_encoding allow apache

# Leave coredumps in the first cache dir
coredump_dir /opt/var/squid/cache

# Disable cachemgr password
cachemgr_passwd none all 

I have my routers IP at 192.168.2.1
chose 8081 for port because I have port 8080 for the webadmin page.

I found out that 5gb is more than enough tho, still don't have it filled after a few days, only like 100mb or something

[Gizmo1007]

Any luck getting dansguarding to work with squid?

[wpte]

Any luck getting dansguarding to work with squid?

currently I'm not working on that anymore

I had some compiling problems I remember...
maybe I should try the crosscompiler once

there is also squidguard: http://www.squidguard.org/