Limiting Bandwidth Usage By VirtualHost [HOWTO]

[sliqua]

Alot of people ask me how I can host clients as large as a certain few that I do efficiently. This quick tutorial explains how to limit bandwidth consumption on files over ___ kb/mb in order to keep overall kb/s or mb/s bandwidth usage to a minimum in order to maintain high download speeds for webpages, but slow down connections for large files and/or file farms (download sites).

There are two different choices for accomplishing this task, I currently utilize both Apache modules but for different reasons, one allows you to specify specific file types (.wav, .mp3, .avi, etc.) and one limits overall usage by each virtualhost.

In order to limit specific file types, i'd recommend the mod_bandwidth port for Apache 2, called bw_mod which is avaliable at http://www.ivn.cl/apache/

This module is relatively easy to get up and running and configured on an Interworx server as the process is the same for any version of Apache :-)

Step One:
Download the latest version from the website mentioned above.

Step Two:
Extract the tarball by executing tar -zxf bw_mod-0.6.tgz
Open the folder of the bw_mod files.

Step Three:
/usr/sbin/apxs -i -a -c bw_mod-0.6.c (change the version respectively)

Step Four:
Restart Apache

Step Five:
Edit the respective virtualhost files in /etc/httpd/conf.d to include the options you want, in my case the following configuration is used in order to limit files over 500kb to be downloaded at roughly 30kb/s:
BandWidthModule On
LargeFileLimit * 500 30240

Step Six:
Restart Apache

Step Seven:
Enjoy faster page loads, while efficiently allowing users to run their sites with large downloadable files.

...The second solution is called mod_cband which is avaliable at http://cband.linux.pl, the install process is almost the exact as the one mentioned above, but always check the documentation avaliable on their site, it has a lot more functionality but doesn't include limiting by file type.

Hope this has been a helpful tutorial and helps you maintain efficient Interworx servers :cool:

[IWorx-Tim]

Thanks for your contribution, ALex. I'm sure there are a lot of people here who could use this.

[Matt]

Perhaps this sort of thing could be included on a global and a per-siteworx basis in a future release. Shouldn't be that hard to do considering it's based on individual vhost files.

[IWorx-Tim]

Perhaps this sort of thing could be included on a global and a per-siteworx basis in a future release. Shouldn't be that hard to do considering it's based on individual vhost files.

We'll take it under consideration. Server security us a priority for the InterWorx team as long as it does not interfere with server functionality.

Tim

[djkiruban]

[root@tamilxtreme bw_mod-0.6]# /usr/sbin/apxs -i -a -c bw_mod-0.6.c
/bin/sh /usr/lib/apr/build/libtool --silent --mode=compile gcc -prefer-pic -O2 -g -pipe -m32 -march=i386 -mtune=pentium4 -DAP_HAVE_DESIGNATED_INITIALIZER -DLINUX=2 -D_REENTRANT -D_GNU_SOURCE -pthread -I/usr/include/apr-0 -I/usr/include/httpd -c -o bw_mod-0.6.lo bw_mod-0.6.c && touch bw_mod-0.6.slo
/usr/lib/apr/build/libtool: line 1092: gcc: command not found
apxs:Error: Command failed with rc=65536

can anyone help me ?

[IWorx-Chris]

yum install gcc


Chris

[djkiruban]

thank you very much

now in step 5 can you explain more please sir

Step Five:
Edit the respective virtualhost files in /etc/httpd/conf.d to include the options you want, in my case the following configuration is used in order to limit files over 500kb to be downloaded at roughly 30kb/s:
BandWidthModule On
LargeFileLimit * 500 30240


give me a example where to put this :

BandWidthModule On
LargeFileLimit * 500 30240

[sliqua]

Hello,

You put this in the individual virtualhosts on a domain-basis. So you'd put it in example.com.conf in conf.d

-Alex

[djkiruban]

hi is work fine

i want to know if i can also limit the maximum download in same time ?

[sliqua]

Yes, it is possible. I'd recommend actually reading the documentation before posting a question here though :-)

http://www.ivn.cl/apache/bw_mod-0.6.txt

[mpcomputers]

Great post...I was able to limit my http traffic perfectly..

I have another question which is driving me crazy!!

My T1 line is capping out when users send larger files via email. I know I can limit the size of the emails but I would rather limit the speed at which they are transfered.

Is ther a way for me to limit the bandwidth usage for qmail just like we did with http traffic?

Any other ideas to slow down max transfer rates for email would be appreicated. Thanks!

[FusionHosting]

Is there a way to make this mod effective server wide for all vhosts by default rather than having to edit each one?

I'd also second adding some of the popular apache mods like this as optional modules included with Interworx. I think it's a lot of value to the product (just like the load balancing did) to have the front end configuration and built-in plug 'n play availability of these solutions out of the box.

It just makes my job that easier and saves so much time rather than having to manually install or compile these.

[sliqua]

Honestly, I think the only thing wrong with InterWorx is how generating the rrd graphs and things is handled - php uses ridiculous amounts of memory when it is occuring that makes some php applications inaccessible on servers with large sites and lots of logs to be processed.

In regards to server wide - i'm sure it could be done, but limiting by vhost ensures that each user recieves the resources that they pay for, and that sites that don't typically use that much transfer get their fair share when needed.

In regards to email, I have the same type of issue with users that pay to send out 30-40 thousand e-mails a week. I decided to charge them a flat rate based on the amount of subscribers they have rather than having to try to calculate resource usage and further tie down server resources. I do agree that there needs to be something done about how to calculate the bandwidth used and limit it - e-mail is as important as websites these days.

[FusionHosting]

I managed to get mod_cband going on my cluster manager to test it out. It kept erroring on the "CBandClassRemoteSpeed googlebot_class 20kb/s 2 3" line so I commented that out for the time being.

I also disabled limitipconn_module, seems redundant now. As with your example I had to edit the individual .conf file for the vhost. Since this is primarily a single app server I would have preferred having it default for all vhosts or server wide on apache but couldn't figure a way to do that.

Has anyone compared cband to mod_bandwidth?

[jerricho]

/usr/sbin/apxs -i -a -c bw_mod-0.6.c
Can't exec "/usr/bin/apr-config": No such file or directory at /usr/sbin/apxs line 49.
Use of uninitialized value in scalar chomp at /usr/sbin/apxs line 54.
apxs:Error: /usr/bin/apr-config not found!.

[IWorx-Chris]

yum install apr-devel apr-util-devel

[FusionHosting]

You may have better luck compiling from actual source code instead of apxs, I had to do that with geo_ip and some other apache mods.

[Gimly]

This contribution help me because I want to use mod_bandwith, thank you for the how to :)

[shinji]

Thanks for providing this howto. Now my site has a nice bandwidth cap in place to prevent possible download hogs in the future. If you want to do it for all files however you have to place this in the config as well or it won't work.

ForceBandWidthModule On

Mine looks like this:

BandWidthModule On
ForceBandWidthModule On
LargeFileLimit * 5120 20480

[pascal]

Is it possible to have this config elsewhere than in a <virtualhost> ?
It could be great to apply this rule to ALL vhosts (at least the rule limiting BW for LargeFile). In the doc they only talk about vhost


Pascal

[dwhs com]

Is it possible to have this config elsewhere than in a <virtualhost> ?
It could be great to apply this rule to ALL vhosts (at least the rule limiting BW for LargeFile). In the doc they only talk about vhost


Pascal

I have been toying with this as well but have no solution.

There has to be a way to run this server wide I just can't figure it out.

Basically giving apache a MPS limit server wide.

[shinji]

What I did is put it in the base vhost config file so that when a new site is generated then it will automatically be placed in the config for it. I'm sure there is a way to get it to be global among all sites but in the meantime this is the best I could come up with. You just have to manually add it to the pre-existing sites. The nice thing is that you can go and manually adjust it later to something else or eliminate it entirely if you find that it wouldn't be useful for that particular site.

[dwhs com]

What I did is put it in the base vhost config file so that when a new site is generated then it will automatically be placed in the config for it. I'm sure there is a way to get it to be global among all sites but in the meantime this is the best I could come up with. You just have to manually add it to the pre-existing sites. The nice thing is that you can go and manually adjust it later to something else or eliminate it entirely if you find that it wouldn't be useful for that particular site.

That is good, my problem is that I wanted to limit the collective amount of the server. otherwise each site basically could use it's limit and the server could use up too much bandwidth as a whole.

But thanks!!!

[djkiruban]

yum install apr-devel apr-util-devel

how to do when we dont have fedora core ?

actually i have ubuntu

:mad: help me please

[sliqua]

Errr, it's quite obvious that Ubuntu doesn't use an RPM system by default, which is also a reason that InterWorx doesn't support it. Ubuntu is based on Debian, which runs on apt. Use 'man apt' for information on how to use it. Common uses are:

apt-get update
apt-get upgrade
apt-get install packagename
apt-cache search program

[djkiruban]

cant found package

with apt-get install apr-devel apr-util-devel

[bear]

trying to install but get this each time?

[root@******* mod_bw-0.7]# /usr/sbin/apxs -i -a -c mod_bw-0.7.c
/usr/lib/apr/build/libtool --silent --mode=compile gcc -prefer-pic -O2 -g -pipe -m32 -march=i386 -mtune=pentium4 -I/usr/kerberos/include -DAP_HAVE_DESIGNATED_INITIALIZER -DLINUX=2 -D_REENTRANT -D_GNU_SOURCE -pthread -I/usr/include/apr-0 -I/usr/include/httpd -c -o mod_bw-0.7.lo mod_bw-0.7.c && touch mod_bw-0.7.slo
gcc: mod_bw-0.7.c: No such file or directory
gcc: no input files
apxs:Error: Command failed with rc=65536


any idea please ?

thank you

[brooky]

I'm no expert but I'm pretty sure it is because you need gcc installed.

In the terminal run:

yum install gcc

... then try again.

[bear]

gcc was already installed i chcked, but thank you for you reply.

started again and went ok this time, not sure what i did wrong mind.

Thank you for the tutorial