CONTENT FILTERING #1
This week, I was given the task of filtering all gaming and video streaming in our network. After doing some research, I find out this is not as straight forward as it seems. Of course, I do not want to spend any money on purchasing any expensive software or hardware.
The general direction I took is to look for light weight proxy that can be installed on our existing OpenWRT router. Initially, I tried tinyproxy and dansguardian. This seems to be a promising solution. However after few hours of testing, users complained that lots of sites throws this error message "ERR_CONTENT_DECODING_FAILED". After a hard search, I cannot find out the course of this error. I decided to ditch tinyproxy and try privoxy. I noticed privoxy while trying to solve the error message.
Anyway, since I went into the trouble of getting tinyproxy and dansguardian up and running. Here are the steps I took to get it running.
| Router Name | OpenWrt |
| Router Model | Buffalo WZR-HP-G450H |
| Firmware Version | OpenWrt Attitude Adjustment 12.09 / LuCI 0.11.1 Release (0.11.1) |
| Kernel Version | 3.3.8 |
ssh into your router.
Install the software
opkg update opkg install tinyproxy luci-app-tinyproxy dansguardian
Delete luci cache
rm /tmp/luci-indexcache
Configure tinyproxy
You can use either tinyproxy config file or luci to configure tinyproxy. Luci config is at Service->Tinyproxy. I used the config file.
My router LAN IP is 192.168.1.1. Tinyproxy port is 3128.
root@OpenWrt:~# cat /etc/config/tinyproxy config tinyproxy option User 'nobody' option Group 'nogroup' option Port '3128' option Listen '127.0.0.1' option Timeout '600' option DefaultErrorFile '/usr/share/tinyproxy/default.html' option StatFile '/usr/share/tinyproxy/stats.html' option MaxClients '100' option MinSpareServers '5' option MaxSpareServers '20' option StartServers '10' option MaxRequestsPerChild '0' option ViaProxyName 'tinyproxy' list ConnectPort '443' list ConnectPort '563' option enabled '1' option FilterExtended '1' option FilterURLs '1' option LogLevel 'Connect' option Syslog '1' option Allow '127.0.0.1' option Allow '192.168.1.0/24'
Configure firewall for transparent proxy
config redirect
option name 'transparent proxy'
option src 'lan'
option proto 'tcp'
option src_dport '80'
option dest_port '3128'
option src_dip '!192.168.1.1'
option dest_ip '192.168.1.1'
Initialise tinyproxy's log file
root@OpenWrt:~# touch /var/log/tinyproxy.log root@OpenWrt:~# chown nobody.nogroup /var/log/tinyproxy.log
Restart tinyproxy every night to refresh memory
root@OpenWrt:~# crontab -e< 0 22 * * * root /etc/init.d/tinyproxy restart
Initialise dansguardianty's log files
root@OpenWrt:~# mkdir /var/log/dansguardian root@OpenWrt:~# touch /var/log/dansguardian/access.log root@OpenWrt:~# touch /var/log/dansguardian/stats
Configure dansguardian
root@OpenWrt:~# root@OpenWrt:~# grep ^[^#] /etc/dansguardian/dansguardian.conf reportinglevel = 2 languagedir = '/usr/share/dansguardian/languages' language = 'ukenglish' loglevel = 2 logexceptionhits = 2 logfileformat = 1 loglocation = '/var/log/dansguardian_access.log' statlocation = '/var/log/dansguardian_stats' filterip = filterport = 8888 proxyip = 127.0.0.1 proxyport = 3128 #this have to match tinyproxy port accessdeniedaddress = 'http://YOURSERVER.YOURDOMAIN/cgi-bin/dansguardian.pl' nonstandarddelimiter = on usecustombannedimage = off custombannedimagefile = '/usr/share/dansguardian/transparent1x1.gif' filtergroups = 1 filtergroupslist = '/etc/dansguardian/lists/filtergroupslist' bannediplist = '/etc/dansguardian/lists/bannediplist' exceptioniplist = '/etc/dansguardian/lists/exceptioniplist' showweightedfound = on weightedphrasemode = 2 urlcachenumber = 1000 urlcacheage = 900 scancleancache = on phrasefiltermode = 2 preservecase = 0 hexdecodecontent = off forcequicksearch = off reverseaddresslookups = off reverseclientiplookups = off logclienthostnames = off createlistcachefiles = on maxuploadsize = -1 maxcontentfiltersize = 256 maxcontentramcachescansize = 2000 maxcontentfilecachescansize = 20000 filecachedir = '/tmp' deletedownloadedtempfiles = on initialtrickledelay = 20 trickledelay = 10 downloadmanager = '/etc/dansguardian/downloadmanagers/fancy.conf' downloadmanager = '/etc/dansguardian/downloadmanagers/default.conf' contentscannertimeout = 60 contentscanexceptions = off recheckreplacedurls = off forwardedfor = off usexforwardedfor = off logconnectionhandlingerrors = on logchildprocesshandling = off maxchildren = 120 minchildren = 8 minsparechildren = 4 preforkchildren = 6 maxsparechildren = 32 maxagechildren = 500 maxips = 0 ipcfilename = '/tmp/.dguardianipc' urlipcfilename = '/tmp/.dguardianurlipc' ipipcfilename = '/tmp/.dguardianipipc' nodaemon = off nologger = off logadblocks = off loguseragent = off daemonuser = 'root' daemongroup = 'root' softrestart = off
root@OpenWrt:# grep ^[^#] /etc/dansguardian/dansguardianf1.conf groupmode = 1 groupname = 'group_one' bannedphraselist = '/etc/dansguardian/lists/bannedphraselist' weightedphraselist = '/etc/dansguardian/lists/weightedphraselist' exceptionphraselist = '/etc/dansguardian/lists/exceptionphraselist' bannedsitelist = '/etc/dansguardian/lists/bannedsitelist' greysitelist = '/etc/dansguardian/lists/greysitelist' exceptionsitelist = '/etc/dansguardian/lists/exceptionsitelist' bannedurllist = '/etc/dansguardian/lists/bannedurllist' greyurllist = '/etc/dansguardian/lists/greyurllist' exceptionurllist = '/etc/dansguardian/lists/exceptionurllist' exceptionregexpurllist = '/etc/dansguardian/lists/exceptionregexpurllist' bannedregexpurllist = '/etc/dansguardian/lists/bannedregexpurllist' picsfile = '/etc/dansguardian/lists/pics' contentregexplist = '/etc/dansguardian/lists/contentregexplist' urlregexplist = '/etc/dansguardian/lists/urlregexplist' blockdownloads = off exceptionextensionlist = '/etc/dansguardian/lists/exceptionextensionlist' exceptionmimetypelist = '/etc/dansguardian/lists/exceptionmimetypelist' bannedextensionlist = '/etc/dansguardian/lists/bannedextensionlist' bannedmimetypelist = '/etc/dansguardian/lists/bannedmimetypelist' exceptionfilesitelist = '/etc/dansguardian/lists/exceptionfilesitelist' exceptionfileurllist = '/etc/dansguardian/lists/exceptionfileurllist' headerregexplist = '/etc/dansguardian/lists/headerregexplist' bannedregexpheaderlist = '/etc/dansguardian/lists/bannedregexpheaderlist' naughtynesslimit = 160 categorydisplaythreshold = 0 embeddedurlweight = 0 enablepics = off bypass = 0 bypasskey = '' infectionbypass = 0 infectionbypasskey = '' infectionbypasserrorsonly = on disablecontentscan = off deepurlanalysis = off
Run tinyproxy and dansguardian
root@OpenWrt:~# /etc/init.d/tinyproxy enable root@OpenWrt:~# /etc/init.d/tinyproxy start root@OpenWrt:~# /etc/init.d/firewall restart
For dansguardian, I realised there is no startup scripts is installed. I just run it from the command by typing dansguardian. Reload it by dansguardian -r. For dansguardian I get this error: "Error reading custom image file: /usr/share/dansguardian/transparent1x1.gif" I just ignore this. It runs fine. If both tineyproxy and dansguardian is running, you should see in ps. There should be lots of messages in their log files.
Content filtering
All the content filtering files are in /etc/dansguardian/lists. You can edit them to suit your requirements.
Debugging
If you want to see what the tinyproxy and dansguardian is doing, check their log files in /var/log.
root@OpenWrt:~# tinyproxy -d -c /var/etc/tinyproxy.conf
References:
- http://www.farville.com/home-networks-a-transparent-proxy-to-monitor-kids/
- http://wiki.openwrt.org/doc/howto/proxy.tinyproxy
- https://forum.openwrt.org/viewtopic.php?id=40631
- https://wiki.gentoo.org/wiki/Content_Filter
- http://alien.slackbook.org/dokuwiki/doku.php?id=slackware:proxy
