OpenWRT - site and content filtering #1 tinyproxy and dansguardian

CONTENT FILTERING #1

This week, I was given the task of filtering all gaming and video streaming in our network. After doing some research, I find out this is not as straight forward as it seems. Of course, I do not want to spend any money on purchasing any expensive software or hardware. 

The general direction I took is to look for light weight proxy that can be installed on our existing OpenWRT router. Initially, I tried tinyproxy and dansguardian. This seems to be a promising solution. However after few hours of testing, users complained that lots of sites throws this error message "ERR_CONTENT_DECODING_FAILED". After a hard search, I cannot find out the course of this error. I decided to ditch tinyproxy and try privoxy. I noticed privoxy while trying to solve the error message. 

Anyway, since I went into the trouble of getting tinyproxy and dansguardian up and running. Here are the steps I took to get it running. 
Router NameOpenWrt
Router ModelBuffalo WZR-HP-G450H
Firmware VersionOpenWrt Attitude Adjustment 12.09 / LuCI 0.11.1 Release (0.11.1)
Kernel Version3.3.8

ssh into your router.

Install the software

opkg update
opkg install tinyproxy luci-app-tinyproxy dansguardian

Delete luci cache 

rm /tmp/luci-indexcache

Configure tinyproxy

You can use either tinyproxy config file or luci to configure tinyproxy. Luci config is at Service->Tinyproxy. I used the config file.
My router LAN IP is 192.168.1.1. Tinyproxy port is 3128.
root@OpenWrt:~# cat /etc/config/tinyproxy 

config tinyproxy
 option User 'nobody'
 option Group 'nogroup'
 option Port '3128'
 option Listen '127.0.0.1'
 option Timeout '600'
 option DefaultErrorFile '/usr/share/tinyproxy/default.html'
 option StatFile '/usr/share/tinyproxy/stats.html'
 option MaxClients '100'
 option MinSpareServers '5'
 option MaxSpareServers '20'
 option StartServers '10'
 option MaxRequestsPerChild '0'
 option ViaProxyName 'tinyproxy'
 list ConnectPort '443'
 list ConnectPort '563'
 option enabled '1'
 option FilterExtended '1'
 option FilterURLs '1'
 option LogLevel 'Connect'
 option Syslog '1'
 option Allow '127.0.0.1'
 option Allow '192.168.1.0/24'

Configure firewall for transparent proxy

 config redirect                                        
    option name 'transparent proxy'                    
    option src 'lan'                                   
    option proto 'tcp'                                 
    option src_dport '80'                              
    option dest_port '3128'                            
    option src_dip '!192.168.1.1'                      
    option dest_ip '192.168.1.1'

Initialise tinyproxy's log file

root@OpenWrt:~# touch /var/log/tinyproxy.log 
root@OpenWrt:~# chown nobody.nogroup /var/log/tinyproxy.log                        

Restart tinyproxy every night to refresh memory

root@OpenWrt:~# crontab -e<
0 22 * * * root /etc/init.d/tinyproxy restart

Initialise dansguardianty's log files

root@OpenWrt:~# mkdir /var/log/dansguardian
root@OpenWrt:~# touch /var/log/dansguardian/access.log
root@OpenWrt:~# touch /var/log/dansguardian/stats

Configure dansguardian

root@OpenWrt:~# root@OpenWrt:~# grep ^[^#] /etc/dansguardian/dansguardian.conf 
reportinglevel = 2
languagedir = '/usr/share/dansguardian/languages'
language = 'ukenglish'
loglevel = 2
logexceptionhits = 2
logfileformat = 1
loglocation = '/var/log/dansguardian_access.log'
statlocation = '/var/log/dansguardian_stats'
filterip =
filterport = 8888
proxyip = 127.0.0.1
proxyport = 3128   #this have to match tinyproxy port  
accessdeniedaddress = 'http://YOURSERVER.YOURDOMAIN/cgi-bin/dansguardian.pl'
nonstandarddelimiter = on
usecustombannedimage = off
custombannedimagefile = '/usr/share/dansguardian/transparent1x1.gif'
filtergroups = 1
filtergroupslist = '/etc/dansguardian/lists/filtergroupslist'
bannediplist = '/etc/dansguardian/lists/bannediplist'
exceptioniplist = '/etc/dansguardian/lists/exceptioniplist'
showweightedfound = on
weightedphrasemode = 2
urlcachenumber = 1000
urlcacheage = 900
scancleancache = on
phrasefiltermode = 2
preservecase = 0
hexdecodecontent = off
forcequicksearch = off
reverseaddresslookups = off
reverseclientiplookups = off
logclienthostnames = off
createlistcachefiles = on
maxuploadsize = -1
maxcontentfiltersize = 256
maxcontentramcachescansize = 2000
maxcontentfilecachescansize = 20000
filecachedir = '/tmp'
deletedownloadedtempfiles = on
initialtrickledelay = 20
trickledelay = 10
downloadmanager = '/etc/dansguardian/downloadmanagers/fancy.conf'
downloadmanager = '/etc/dansguardian/downloadmanagers/default.conf'
contentscannertimeout = 60
contentscanexceptions = off
recheckreplacedurls = off
forwardedfor = off
usexforwardedfor = off
logconnectionhandlingerrors = on
logchildprocesshandling = off
maxchildren = 120
minchildren = 8
minsparechildren = 4
preforkchildren = 6
maxsparechildren = 32
maxagechildren = 500
maxips = 0
ipcfilename = '/tmp/.dguardianipc'
urlipcfilename = '/tmp/.dguardianurlipc'
ipipcfilename = '/tmp/.dguardianipipc'
nodaemon = off
nologger = off
logadblocks = off
loguseragent = off
daemonuser = 'root'
daemongroup = 'root'
softrestart = off

root@OpenWrt:# grep ^[^#] /etc/dansguardian/dansguardianf1.conf 
groupmode = 1
groupname = 'group_one'
bannedphraselist = '/etc/dansguardian/lists/bannedphraselist'
weightedphraselist = '/etc/dansguardian/lists/weightedphraselist'
exceptionphraselist = '/etc/dansguardian/lists/exceptionphraselist'
bannedsitelist = '/etc/dansguardian/lists/bannedsitelist'
greysitelist = '/etc/dansguardian/lists/greysitelist'
exceptionsitelist = '/etc/dansguardian/lists/exceptionsitelist'
bannedurllist = '/etc/dansguardian/lists/bannedurllist'
greyurllist = '/etc/dansguardian/lists/greyurllist'
exceptionurllist = '/etc/dansguardian/lists/exceptionurllist'
exceptionregexpurllist = '/etc/dansguardian/lists/exceptionregexpurllist'
bannedregexpurllist = '/etc/dansguardian/lists/bannedregexpurllist'
picsfile = '/etc/dansguardian/lists/pics'
contentregexplist = '/etc/dansguardian/lists/contentregexplist'
urlregexplist = '/etc/dansguardian/lists/urlregexplist'
blockdownloads = off
exceptionextensionlist = '/etc/dansguardian/lists/exceptionextensionlist'
exceptionmimetypelist = '/etc/dansguardian/lists/exceptionmimetypelist'
bannedextensionlist = '/etc/dansguardian/lists/bannedextensionlist'
bannedmimetypelist = '/etc/dansguardian/lists/bannedmimetypelist'
exceptionfilesitelist = '/etc/dansguardian/lists/exceptionfilesitelist'
exceptionfileurllist = '/etc/dansguardian/lists/exceptionfileurllist'
headerregexplist = '/etc/dansguardian/lists/headerregexplist'
bannedregexpheaderlist = '/etc/dansguardian/lists/bannedregexpheaderlist'
naughtynesslimit = 160
categorydisplaythreshold = 0
embeddedurlweight = 0
enablepics = off
bypass = 0
bypasskey = ''
infectionbypass = 0
infectionbypasskey = ''
infectionbypasserrorsonly = on
disablecontentscan = off
deepurlanalysis = off

Run tinyproxy and dansguardian

root@OpenWrt:~# /etc/init.d/tinyproxy enable
root@OpenWrt:~# /etc/init.d/tinyproxy start
root@OpenWrt:~# /etc/init.d/firewall restart
For dansguardian, I realised there is no startup scripts is installed. I just run it from the command by typing dansguardian. Reload it by dansguardian -r. For dansguardian I get this error: "Error reading custom image file: /usr/share/dansguardian/transparent1x1.gif" I just ignore this. It runs fine. If both tineyproxy and dansguardian is running, you should see in ps. There should be lots of messages in their log files.

Content filtering

All the content filtering files are in /etc/dansguardian/lists. You can edit them to suit your requirements. 

Debugging

If you want to see what the tinyproxy and dansguardian is doing, check their log files in /var/log.
root@OpenWrt:~# tinyproxy -d -c /var/etc/tinyproxy.conf

References: