Re: [squid-users] Caching issue with http_port when running in transparent mode

Eliezer Croitoru wrote:

> On 05/06/2012 17:22, Hans Musil wrote:
> > Eliezer wrote:
> >
> >> one important thing to be aware of is that if you are using the same
> box
> >> as a gateway and squidbox it's better to use the "redirect" instead of
> >> DNAT.
> >>
> >> you can always try to use:
> >> http://nocat.net/downloads/NoCatSplash/
> >>
> >> or to write your own helper.
> >> it can be pretty simple to build such an helper and you will just need
> >> to use some NAT chains\tables on iptables that will redirect any
> >> connection to the world into the webserver with a login page that
> >> connected to a script that will do some stuff in the iptables "allow"
> >> table.
> >>
> >> do you need to apply some username and password mechanism\auth or just
> >> splash screen to agree some rules\agreement ?
> >>
> >> Eliezer
> >>
> >
> > Thanks again, Eliezer. The hint for the REDIRECT target is a good point.
> >
> > NoCatSplash does not work for my as I need more control. Not only that
> users need to login, they also need to logout when done. Furthermore, I need
> to trigger a traffic quotation system from the login/out script. Also, web
> traffic needs to be logged. NoCatSplash seems not to be flexible enough.
> >
> > Hans
> >
>
> well.
> nocatspalash can be updated a bit to fit a login\logout.
> i know that there was a tool for billing and quota
> and it depends on the traffic quota basis you want.
> if you want to supply internet traffic based only on web quota or also
> based on other network services quota.
>
> i have implemented long ago a traffic meter using iptables and wrote it
> to a mysql DB.
> and can be added to it a quota limit based on user\ ip.
>
> if you want the idea then its':
> create a web page to login logout with cgi based on passwd and user in db.
> add to it a quota status if you want(preferably to yes).
>
> in iptables rules you should create specific talbes for quota meter.
> so the iptables should:
> allow all users in lan traffic to the gw machine web site.
> have tables that count traffic for each ip that will be added by the web
> scritps.
> have an helper that runs every 30 sec and dumps ip tables stats and
> reset the counter.
> then parses the data from the file into db by users.
> then checks if quota exceeded and set the proper iptables tables\rules
> and db flags for that user and.
>
> i would run the helper every 30 secs for grace time but will run a
> specific login\logout script\program that will change the proper flags
> and counters in db for the user\ip.
>
> this is a tutorial specific for iptablbes counter
> http://www.catonmat.net/blog/traffic-accounting-with-iptables/
> i have seen the thing with the DB here:
> http://wiki.openvz.org/Traffic_accounting_with_iptables
>
> you can use use snmp to pull the data from db using a script
>
> to get a specific table data (like a custom one you can use)
> iptables --line-number -xnvL FORWARD
>
> iptables --line-number -xnvL FORWARD |gawk '{print $1 " " $3 " " $10}'
> this will give you the bytes statistics for each IP.
>
> just remember that if you are using a proxy server you will also need to
> count the redirected\intercepted traffic in a intercept table.
>
> i have found this nice thing to use snmp for monitoring:
> http://www.nativenet.ch/content/view/28/51
>
> and also this:
>
> http://forums.cacti.net/viewtopic.php?t=8091&highlight=iptables
>
>
> as for the exact way to measure clients traffic quota i'm sure there is
> a more "way forward" way then parsing the iptables stats.
>
> but it's one of the best tools in linux world.
>
> there is also the quota module of iptalbes but im not sure it's for this
> case.
>
> so any way it's a big thing quota and users by itself.
>
> i think it's doable if you will custom the iptables structure\schema for
> this specific use.
> every time you check the current counter you can zero it specifically.
>
>
> if you are up to the task of combining a psudo code for the whole
> process with me i will be happy to sit on it some time in from the next
> week.
>
> Eliezer
>
>
> <SNIP>

Thank you, Eliezer, for this very detailed description. Some months ago, I already did play around with quotation and traffic shaping. And I think I have found a reasonable way to manage this things.

As you have mentioned, iptables has a quota module witch is very helpful. In contrary to the traffic measuring tutorial you have linked, my goal is not to meassure arbitrary traffic, but to set a pre-defined quota. After this quota is exceeded, the traffic for this user will be throttled by traffic control rules. iptables' task is just to mark packets that exceed the quota. The rest is done by the traffic control tool tc.

Unfortunately, I have not yet digged out all my old stuff. Thus, I'm not yet able to send you a working example code, but just some key lines:

In iptables:

2 simple chains that mark a packet and return. 10 is normal traffic, 11 is traffic that will be throttled:

iptables -t mangle -A mrkret10 -j MARK --set-xmark 0xa/0xffffffff
iptables -t mangle -A mrkret10 -j RETURN
iptables -t mangle -A mrkret11 -j MARK --set-xmark 0xb/0xffffffff
iptables -t mangle -A mrkret11 -j RETURN

Now, the quotation using the quota module:

iptables -t mangle -A chkiquota -d $usrIp -m quota --quota $usrIquota -g mrkret10
iptables -t mangle -A chkiquota -d $usrIp -g mrkret11

As long as the quota is not yet exceeded, the first rule counts down the remaining quota and goes to chain mrkret10 (without returning). If the quota is exceeded, the package goes to chain mrkret11, also without returning.

When a user loggs out (or wants to know his remaining traffic volume), we have to read out the remaining quota from iptables. This is done by

iptables -t mangle -S chkiquota

and a simple parsing.

This way, I do not need any daemon job. Only, perhaps, for an auto logout mechanism in case some user forgets to logout.

Hans

-- 
NEU: FreePhone 3-fach-Flat mit kostenlosem Smartphone!                                  
Jetzt informieren: http://mobile.1und1.de/?ac=OM.PW.PW003K20328T7073a