htaccess Tips and Tricks
(This is an older article and some options might not still be relevant)
After many frustrating months of learning how to protect my website, I decided to help some other webmasters out as well. There’s really only one reason we need to worry about this stuff: People like to steal. Every increase in my site’s security has been brought on by someone hacking into it and let me tell you, there have been many increases.
I have separated this page into several sections:
1) Stopping hackers
2) Stopping site snagging (offline viewing)
3) Stopping Hotlinking
4) Multiple Domain Names: Shared Members Areas
Section 1) Stopping hackers
The most common way of protecting your members-only area is with, as I’m sure you know, a file named .htaccess sitting in your server’s member’s folder. This file is used by your server to pop up a little box and force people to enter a username and password. It then checks that against a password file located on your server to see if the info is valid. If it is, access is given.
There are, however, many lines that you can add to your .htaccess file that most webmasters don’t really know about. I’ll go through them one by one as well as show you completed .htaccess files that you can start using immediately.
NOTE: You only need to copy the text below that is in black. It is also important that you use a very basic text editor to save the file. Use Notepad or NoteTab to do it. DO NOT use MS Word! The file will not save correctly! You should also realize that a .htaccess file is just a plain text file with a funny name. The complete file name really is .htaccess, with a period in front, and all.
Here is the basic .htaccess file that most people use:
AuthUserFile /server/path/to/your/password/file/.htpasswd
AuthGroupFile /dev/null
AuthName “Members Area”
AuthType Basic
<limit GET PUT POST>
require valid-user
</limit>
This file, when placed in your members-only folder will protect all of the subfolders under it. There are however some holes here. Once inside the member’s area, they can still poke around for things you may not want them to see by being creative and typing in the URL. Most of the time this is no big deal. However, they really don’t need to be poking around in there.
Another problem is that some password security programs have to be accessed directly or in a very specific way to work. An older version of the security program I used required a file called index.cgi to be placed in the member’s only folder. When you link to http://your-main-domain.com/members/ it would do two things. First, the .htaccess file would check the username and password to see if they were valid. Second, if approved, it would run my security program to see how many people have used that username and password. If that checked out, they would be sent to the opening page of my member’s area which was actually http://your-main-domain.com/members/welcome.htm.
That worked fine as long as no one tried to go directly to the welcome.htm page. Guess what, hackers are smart. By posting a simple link on a password trading site, they could bypass the security program and gain access in one easy step. The link would look like this:
http://username:password@your-main-domain.com/members/welcome.htm
Look familiar? If you’ve ever been password traded (and you will) it should look familiar. After that, I learned of some code that would stop this and force everyone to use one page to gain access to the member’s area.
AuthUserFile /server/path/to/your/password/file/.htpasswd
AuthGroupFile /dev/null
AuthName “Members Area”
AuthType Basic
<limit GET PUT POST>
require valid-user
</limit>
RewriteEngine On
RewriteCond %{HTTP_REFERER} !^http://([a-z0-9-]+.)*your-main-domain.com/ [NC]
RewriteRule /* http://www.your-main-domain.com/login.htm [L, R]
The new section activated the RewriteEngine feature of your .htaccess file. This will now only allow access to the member’s area of your site through a link on your page. If they don’t use an actual link on your site they cannot get in. Any URL that you manually type into the address bar of your browser will show up in your log file as having no referrer and will not pass. The only way to satisfy the RewriteCond of this updated .htaccess file is to use a link on your site.
Using this example, you will need a new little web page named login.htm in your free area. On that page, you will need a link to your member’s area. Whatever link will allow your security program to work right?
The main thing I like about using this is that it keeps people from messing around inside the member’s area. Since I update with new pics every week, I can upload several sets at a time to the server and have them waiting. I don’t have to worry about anyone finding them before I link to them.
Now, remember, if you don’t have any software in place to monitor how many times your usernames and passwords are being used, this won’t help you at all. This method won’t stop shared usernames and passwords from being used. It is only here to channel people into your password-sharing software. I personally recommend using the Password Sentry. It’s a one-time charge and they give you lifetime upgrades and support. It was also not very expensive. I haven’t found any program out there that I liked any better, at any price. You can find them at monster-submit.com/sentry/
Remember Web Host Pro has FREE password sharing and brute force protection!
I actually use their newest version which can stop people from hammering your site with username and password combinations until they get one that works. I was getting at least one person a day running one of those programs on my site trying to get in. I still use a .htaccess file in my member’s area, but it no longer checks for a username and password. It looks for a temporary cookie that is placed on their system if they are approved by my security program. It’s just as secure but blocks those password-hammering programs completely.
——————————————————————————–
2) Stopping Site Snagging
This one pisses me off. There are many programs out there designed for “offline viewing” of websites. These programs allow a person to download everything on your site to their computer. It works wonderfully in the free area, however, if they have a username and password to your site, they can also download your entire member’s area.
If you don’t have any software protecting you from password traders, this one could be devastating. Not only could everyone in the world get into your member’s area for free, but they could also download everything in there in a hurry. If you have 200 MB of stuff on your site and 1000 people get in for free and decide to use one of these programs, you’re looking at 200 gigabytes of transfer in as short as one day. Can you afford that? Those numbers are kind too. Many of you have much more than 200 MB of stuff. I’ve also been traded in the past and was receiving 4500 people per hour into the member’s area for free. That could put you out of business in a hurry.
If you don’t think that these programs are a problem check your stats. Many stats programs will tell you the different web browsers that are visiting your site. I have programs like Teleport Pro and Offline Explorer in my top 10 web browsers every single day.
Since we have to pay for bandwidth, which can get expensive as your site grows, this can turn into a major problem. I was surprised at how much bandwidth I saved after adding these lines to a .htaccess file.
Here’s the best part. You can place this .htaccess file in your root public directory. Put it in the same folder as your site’s opening index file and it will protect your entire site.
You’ll notice one major difference in this file. It doesn’t require usernames and passwords to get in. Those lines have simply been removed from the file. It will also not have any effect on the .htaccess file in your member’s folder. That one will check passwords, and this one will stop people from snagging your site.
There are actually 3 sections to the file below.
The first section allows you to block specific users’ IP addresses. I have two blocked here. Some users tried hammering my site with around 20,000 username and password combos. This part is optional since most people have a new IP each time they log on. However, if they are using a cable modem they will keep the same IP all of the time like the two in my example. If I were you I would definitely leave that guy in there.
The second section is related to error 404s. This works well with the way many search engines work. I don’t know how many of them are still linking to pages on my site that no longer exist. If someone clicks on a link from that search engine that is no longer any good, they just get that blank error page. The error document line below forwards those people to another page. I forward them to my opening page. That way, if they come to my site using a link that no longer is valid, they end up at my opening page never realizing that the link was bad.
The third section stops the programs that will try and download your site. Since I’m finding more all the time the list keeps growing. If you discover more, just add them to it. If the program is actually two words, Teleport Pro, for example, you only need to include one word to block them. Notice below that I have a line including Teleport, but not Teleport Pro. I’ve downloaded the program and tested it. This method works perfectly.
The very last line, the RewriteRule, is where violators will be sent to https://webhostpro.com it’s a generic page from our hosting service that will take the bandwidth of the person getting kicked off.
<Limit GET>
order allow,deny
deny from 24.128.16.113
allow from all
</Limit>
errordocument 404 https://webhostpro.com/index.php?rp=/login
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} ^.*WebZIP.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Iria.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Stripper.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Offline.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Copier.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Crawler.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Snagger.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Teleport.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Reaper.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Wget.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Grabber.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Sucker.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Downloader.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Siphon.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Collector.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Mag-Net.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Widow.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Pockey.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*DA.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Snake.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*BackWeb.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*gotit.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Vacuum.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*SmartDownload.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Pump.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*HMView.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Ninja.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*HTTrack.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*JOC.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*likse.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Memo.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*pcBrowser.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*SuperBot.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*leech.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Mirror.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Recorder.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*GrabNet.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Likse.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Navroad.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*attach.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Magnet.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Surfbot.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Bandit.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Ants.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Buddy.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Whacker.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*DISCoPump.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Drip.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*EirGrabber.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*ExtractorPro.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*EyeNetIE.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*FlashGet.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*GetRight.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Gets.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Go!Zilla.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Go-Ahead-Got-It.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Grafula.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*IBrowse.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*InterGET.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*InternetNinja.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*JetCar.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*JustView.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*MIDowntool.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*MisterPiX.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*NearSite.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*NetSpider.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*OfflineExplorer.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*PageGrabber.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*PapaFoto.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Pockey.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*ReGet.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Slurp.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*SpaceBison.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*SuperHTTP.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Teleport.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebAuto.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebcamWatcher.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebCopier.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebFetch.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebReaper.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*FreeLoader.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Clint’sWebcam.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebCamSpy.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*CamEVU.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*iCamMaster.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*CamChaserPro.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*FlashIT.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebSauger.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebStripper.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebWhacker.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebZIP.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebImageCollector.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebSucker.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Webster.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Wget.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*eCatch.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*ia_archiver.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*lftp.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*tAkeOut.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*FileHound.*$
RewriteRule /* http://your-main-domain.com [L,R]
If you decide to redirect them somewhere else be sure to leave the “[L, R]” at the end of the line. It’s rather important.
Remember to always check your site immediately after uploading a new .htaccess file to your server. If there are any errors in your file, your site will most likely not load at all. In that case, quickly delete the file from the server until you figure out what went wrong!
——————————————————————————–
3) Stopping Hotlinking
I think I see some of you smiling already. Yes, you can use a .htaccess file to stop people from hotlinking images off of your site. I recently discovered several of my pictures being posted on a messageboard. They had a little message and then my picture would pop up in the message. It was loading directly off of my server with absolutely nothing pointing back to me. I was pissed.
The .htaccess file to prevent this is very similar to some of the ones above. It’s just much shorter since it only performs one function, to stop hotlinking. It does this by checking the referrer. In other words, where the hit is coming from.
I have actually moved all of my images, graphics, games, you name it into a subfolder in the free area. I then just place this .htaccess file into that folder.
I DON’T recommend adding these lines into the .htaccess file above that protects your entire site. Why? Well, when you sign up on someone else’s friends page you have to enter the URL of your ID picture. If you block everything then all of your ID pictures on all of those friends pages you signed up for will not load. Your ID picture will be a very sexy little red x.
You can stop people from hotlinking your ID pictures if you want, just think it through first. I have my banner farm protected to stop new sign-ups from hotlinking. However, I still have a few pictures in unprotected areas too. That way I can sign up for new friends and links pages. You also don’t want to block everything if you purposely post pictures on picture-post pages. If you block your entire site, none of those picture posts will load.
Similar to some of the above files, this one will allow the picture to load if the referring site starts with webhostpro/ only. Do not include the www . in here. That’s what all of the crap in front of webhostpro/ is for. The referrer can end with anything it likes, as long as it has webhostpro/ in it.
RewriteEngine Onhttps://webhostpro.com/index.php?rp=/login
RewriteCond %{HTTP_REFERER} !^http://([a-z0-9-]+.)*memberssite.com/ [NC]
RewriteRule /* [L,R]
——————————————————————————–
Multiple Domain Names: Shared Members Areas
Here’s a fun one. Many of you may have several websites but only one credit card account and one password file. How do you get everyone to have access to all of your site’s member areas but only use one account? Easy. Use .htaccess files. This can get a little bit tricky so pay attention.
Let’s say you have three sites: webhost1.pro, webhost21.pro, and webhost3.pro
Let’s also say that you want anyone joining one site to have access to all three.
Pick one site to house the main entry page. Just like in the above examples, create a page called https://webhostpro.com/index.php?rp=/login in the free area of that site. You can call it whatever you want. Use that page as the entry page for all of your websites. Just put a link on there saying “Click here to enter the member’s area” or something.
Now everywhere on webhost2.pro and webhost3.pro that says “member’s entrance” should point to https://webhostpro.com/index.php?rp=/login Understand? Only one entrance page and only one password file. Everyone must enter from the same place.
Now, you’ll need to add the following lines to your .htaccess file in the member’s only folder of webhost1.pro.
RewriteEngine Onhttps://webhostpro.com/index.php?rp=/login
RewriteCond %{HTTP_REFERER} !^http://([a-z0-9-]+.)*webhost.pro/ [NC]
RewriteCond %{HTTP_REFERER} !^http://([a-z0-9-]+.)*webhost2.pro/members/ [NC]
RewriteCond %{HTTP_REFERER} !^http://([a-z0-9-]+.)*webhost3.pro/members/ [NC]
RewriteRule /* [L,R]
This will allow entry only from either your main page’s entry page or from the member’s area of your other sites. Is part is tricky to think about but very important.
Your new webhost1.pro member’s only folder .htaccess file will most likely look like this:AuthUserFile /server/path/to/your/password/file/.htpasswd
AuthGroupFile /dev/null
AuthName “Members Area”
AuthType Basic
<limit GET PUT POST>
require valid-user
</limit>
RewriteCond %{HTTP_REFERER} !^http://([a-z0-9-]+.)*webhost.pro/ [NC]
RewriteCond %{HTTP_REFERER} !^http://([a-z0-9-]+.)*webhost2.pro/members/ [NC]
RewriteCond %{HTTP_REFERER} !^http://([a-z0-9-]+.)*webhost3.pro/members/ [NC]
RewriteRule /* https://webhostpro.com/index.php?rp=/login [L, R]
Now here’s the fun part. The member’s areas of webhost2.pro and webhost3.pro will no longer check for a valid username and password. They will only check out where the person is coming from. If they aren’t coming from one of three places they will be routed to the login.htm page on webhost1.pro.
This .htaccess file is very small and should be placed in the members-only folder at webhost2.pro and webhost3.pro.
You must include lines for all of your sites in every .htaccess file.
The .htaccess files at webhost2.pro/members and webhost3.pro/members should look like this:
RewriteCond %{HTTP_REFERER} !^http://([a-z0-9-]+.)*webhost1.pro/members/ [NC]https://webhostpro.com/index.php?rp=/login
RewriteCond %{HTTP_REFERER} !^http://([a-z0-9-]+.)*webhost2.pro/members/ [NC]
RewriteCond %{HTTP_REFERER} !^http://([a-z0-9-]+.)*webhost3.pro/members/ [NC]
RewriteRule /* [L,R]
That’s it. They’re very short files but they will do the job. These new .htaccess files at webhost1 and webhost2 will only allow people access if they’re coming from the members-only area of one of the other sites. They don’t need to check usernames and passwords too.
I made a new page in my member’s area that links to all three of my sites. Once they are validated webhost.pro they end up on this one page. It’s sort of a “Welcome inside. What site do you want to visit?” type of thing. It works very, very well and allows me to use one password file for as many sites as I want.
This method can also be used to allow two very different sites to share a member. Each site can be owned and operated by two different people using two different login pages, generating their own revenue, but sharing a members area.
Just allow access from either your own site or the members-only folder of the other site.