It's important to make sure your server is properly protected by a firewall, but configuring IPtables rulesets by hand can be confusing and it's easy to make mistakes.
One easy solution is to use APF (Advanced Policy Firewall), along with it's companion application BFD (Brute Force Detection). APF allows you to manage IPtables using some simple configuration files, while BFD checks the server logs for failed login attempts - blocking the source IPs once a pre-determined limit is reached (25 failed logon attempts is the default).
To install them, download the latest stable versions:
wget http://www.rfxn.com/downloads/apf-current.tar.gz
wget http://www.r-fx.ca/downloads/bfd-current.tar.gz
Unpack the files using 'tar -xzf' and then run the install.sh script for each.
APF:
You won't be able to see APF running as a process (running ps or top), so to check the status use this command:
/usr/local/sbin/apf-st (or –status)
If APF hasn't started automatically, start it by running:
/etc/init.d/apf start
To start adding rules, open the main config file:
vim /etc/apf/conf.apf
To allow specific TCP and UDP ports, you'll need to add them one of four sections within the config file:
Common inbound (ingress) TCP ports
Common inbound (ingress) UDP ports
Common outbound (egress) TCP ports
Common outbound (egress) UDP ports
Add any ports you want to allow, using an underscore to separate ranges, e.g. -
# Common inbound (ingress) TCP ports
IG_TCP_CPORTS="21,25,80,110,143,222,443,587,43,2077_2087,"
This will allow inbound access to the TCP ports 21,25,80,110,143,222,443,587,43 and the range 2077 to 2087.
You can also enabled third-party block lists, such as DROP and DShield, by editing the Remote Rule Imports section. Each list entry starts with a DLIST_[ListName] option set to '0', Change this to '1' to enable it.
Host-based rules can exist in one of two states - allow or deny - each of which has it's own config file:
/etc/apf/allow_hosts.rules
/etc/apf/deny_hosts.rules
You grant an IP address unrestricted access, just add it to the allow_hosts.rules file and then restart apf.
To grant the IP address access only to specific ports, add a ruling using this syntax:
protocol:flow[s/d]=port:[s/d]=ip(/mask)
Note: s = source, d = destination, flow = packet flow in/out
For example, assume we've blocked access to port 110 (by NOT listing it in conf.apf) and want to allow POP downloads only from the specific IP address 10.0.0.1, you would add a rule like this:
tcp:in:d=110:s=10.0.0.1
tcp:out:d=110:s=10.0.0.1
This would allow inbound and outbound access to port 110 from 10.0.0.1. IP ranges can be allowed by adding a netmask in CIDR notation after the IP address, e.g. 10.0.0.1/16.
To block IP addresses, add them to the deny_hosts.rules file and restart apf.
You can also block access to specific ports, using the same syntax described above to allow ports. The example given, if added to deny_hosts.rules, would block access to port 110 from the IP address 10.0.0.1, instead of granting access.
Note: Always remember to restart apf after adding any rules to make sure they take effect. You can also force a reload using:
/etc/apf/apf -r
You can also check for changes and errors in the apf log:
tail -f /var/log/apf_log
BFD:
The main BFD config file is:
/usr/local/bfd/conf.bfd
There are several options, but the main two you might want to edit are:
1) TRIG="25"
This option allows you to set the number of failure events per IP address before it is blocked. The default is usually 25, but you can add a lower or higher value as required.
2) EMAIL_ADDRESS="root"
By default, all alerts are sent to the root user. If you want them to be sent to an external e-mail address, add the address here. Multiple addresses can be added by comma separating them.
If BFD blocks an IP address it will be written to the deny_hosts.rules file, blocking all access to any port from that IP address. The service which initiated the block will be listed next to the IP address, e.g. {bfd.pure-ftpd} would indicate that more than 25 failed FTP login attempts have originated from the blocked IP address.
To allow access again, remove IP address from deny_hosts.rules and restart apf.
For more information, see the README files at:
http://www.rfxn.com/appdocs/README.apf
http://www.rfxn.com/appdocs/README.bfd
Wednesday, May 26, 2010
Thursday, May 20, 2010
Identifying Disk Usage and Reclaiming Space
As web sites become larger, fuelled by the availability of faster and faster internet connections, disk space can rapidly become depleted. Anything from excessive FTP uploads, to large log files, can quickly start to eat into your available disk space, so it's useful to know how to identify the cause and know how to reclaim that space back.
The df command (an old Unix command that stands for Disk Free) can be used to get a quick overview of the server's disk usage. Run it and you will see something like this:
: chrish@delli@12:56 ; df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 13G 12G 1.6G 88% /
tmpfs 980M 0 980M 0% /lib/init/rw
varrun 980M 384K 980M 1% /var/run
varlock 980M 0 980M 0% /var/lock
udev 980M 2.8M 977M 1% /dev
tmpfs 980M 164K 980M 1% /dev/shm
/dev/sda1 236M 31M 194M 14% /boot
/dev/sda5 19G 13G 5.9G 68% /mnt/storage1
Note: By default, df will report the partition sizes in kilobytes or half-kilobytes. If you use the -h switch, as shown above, it will print sizes in a human readable format (e.g., 1K, 234M, 2G).
>From the listing above, you can see the / partition is starting to run low on space. While 1.6 gigabytes of space free isn't critical, let's look into what is using this and see if we can free up some more space.
For this we will use the du (Disk Usage) command. Rather than checking how much of the disk space is being used, it reports how large files on the disk are.
The usual format for using this command would be something like : du -sch /*
We've added a few switches to the command, which are explained below:
-s Without this option the command will spider any directories that it takes as arguments and give you the size on very file inside them (this is the equivalent to setting --max-depth 0).
-c Provides a total at the end of list, useful to ensure that what you're checking matches the directory totals.
-h Like df, this option displays the sizes in a more readable format.
When you have many large filesystems mounted to directories on disks that are running out of space, it's good practise to stop du from traversing into them - to accomplish this use the -x switch. This forces du to stay on the filesystem(s) that your path arguments correspond to. Because of this, if you wanted to run the command "du -sch /*" you would need to rewrite it if any of the directories in / are on separate filesystems (like /boot). To achieve the same information, but just reporting:
du -hx --max-depth 1 /
If you're wondering why "/*" would include filesystems mounted in directories inside / it is because this type of argument makes use of shell 'globbing'. Globbing is where the shell (i.e. Bash) fills in the blanks and passes them across to the program (in this case du).
If you're still a little confused, try running the following command.
echo /*
Before moving onto how you might want to deal with these files we'll take a quick look at the find command. The find command lists files in a certain place that meet a certain criteria. If you were to run the following command it would eventually list all files that are on your system:
find /
To help us find files that we might want to consider removing, the following two commands may be of use.
find /directroy/containing/uploads -type f -size +100M -ls
find /directory/containing/uploads -type f -mtime +90 -ls
Both of these commands are looking inside the directory (and subdirectories of) /directory/containing/uploads. The '-type f' part tells find to only return files (rather than directories, for instance), as we are only really interested in files. The first of these commands will list all of the files more than 100 megabytes in size, and the second lists all files that haven't been modified in the last 90 days. The -ls argument at the end of these provides us with more information about the files, rather than just their names.
If you are happy to move the files that it flags up, you can make find do this also. Using the -exec flag you can request that find runs a command on each file. Here's an example that would move the files larger than 100 megabytes to another directory. Be sure not to ommit the \; punctuation at the end.
find /directory/containing/uploads -type f -size +100M -exec mv '{}'
/mnt/large_external_harddrive/ \;
Now that you have an idea of what is using the space let's cover the basic short term fixes available. Here I'll briefly cover moving, removing and compressing data.
Please bear in mind that by removing the wrong files and directories you risk not only losing valuable data, you also put the integrity of your system at risk. If in doubt, always check before removing a file or a directory.
If you have enough space available on another filesystem, and you do not want to remove what you have isolated as the most obvious cause, feel free to move (mv) the files or directories to another location.
mv /var/log/big.log /mnt/large_external_harddrive/
If you want to you can create symbolic link from the old location to where it now resides; this way it will continue to be accessible from the old path.
ln -s /mnt/large_external_harddrive/big.log /var/log/big.log
To simply truncate a file (reduce to nothing in size), simply redirect nothing into it. Like so:
echo > /var/log/big.log
Or, more simply:
> /var/log/big.log
This saves you the hassle of recreating it with the correct name and file permissions after simply removing it.
Finally, to compress data use the gzip command simply point it at a file. For instance the following command will remove the file it is pointed at, and in its place a compressed version (/var/log/big.log.gz) will exist.
gzip /var/log/big.log
This isn't of much use when the filesystem is already full though - because there is insufficient space to create the compressed file, prior to removing the original.
However, we can use gzip to compress the file to a different place if there isn't enough space to move the file and then compress it. The following example creates the file
/mnt/large_external_harddrive/big.log.gz. Bear in mind that when using gzip with the -c switch, it does not remove the original file.
gzip -c /var/log/big.log > /mnt/large_external_harddrive/big.log.gz
Finally, a note concerning log files, if you remove a log file without restarting the program that is writing to it you may find that 'df' doesn't report the space as free. This is because although link to the file might not exist in the filesystem now, however while a process is still referencing the file the blocks associate with it are still marked as used.
Therefore, if you have to remove a current Apache log file, for instance, be sure to restart Apache afterwards.
The df command (an old Unix command that stands for Disk Free) can be used to get a quick overview of the server's disk usage. Run it and you will see something like this:
: chrish@delli@12:56 ; df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 13G 12G 1.6G 88% /
tmpfs 980M 0 980M 0% /lib/init/rw
varrun 980M 384K 980M 1% /var/run
varlock 980M 0 980M 0% /var/lock
udev 980M 2.8M 977M 1% /dev
tmpfs 980M 164K 980M 1% /dev/shm
/dev/sda1 236M 31M 194M 14% /boot
/dev/sda5 19G 13G 5.9G 68% /mnt/storage1
Note: By default, df will report the partition sizes in kilobytes or half-kilobytes. If you use the -h switch, as shown above, it will print sizes in a human readable format (e.g., 1K, 234M, 2G).
>From the listing above, you can see the / partition is starting to run low on space. While 1.6 gigabytes of space free isn't critical, let's look into what is using this and see if we can free up some more space.
For this we will use the du (Disk Usage) command. Rather than checking how much of the disk space is being used, it reports how large files on the disk are.
The usual format for using this command would be something like : du -sch /*
We've added a few switches to the command, which are explained below:
-s Without this option the command will spider any directories that it takes as arguments and give you the size on very file inside them (this is the equivalent to setting --max-depth 0).
-c Provides a total at the end of list, useful to ensure that what you're checking matches the directory totals.
-h Like df, this option displays the sizes in a more readable format.
When you have many large filesystems mounted to directories on disks that are running out of space, it's good practise to stop du from traversing into them - to accomplish this use the -x switch. This forces du to stay on the filesystem(s) that your path arguments correspond to. Because of this, if you wanted to run the command "du -sch /*" you would need to rewrite it if any of the directories in / are on separate filesystems (like /boot). To achieve the same information, but just reporting:
du -hx --max-depth 1 /
If you're wondering why "/*" would include filesystems mounted in directories inside / it is because this type of argument makes use of shell 'globbing'. Globbing is where the shell (i.e. Bash) fills in the blanks and passes them across to the program (in this case du).
If you're still a little confused, try running the following command.
echo /*
Before moving onto how you might want to deal with these files we'll take a quick look at the find command. The find command lists files in a certain place that meet a certain criteria. If you were to run the following command it would eventually list all files that are on your system:
find /
To help us find files that we might want to consider removing, the following two commands may be of use.
find /directroy/containing/uploads -type f -size +100M -ls
find /directory/containing/uploads -type f -mtime +90 -ls
Both of these commands are looking inside the directory (and subdirectories of) /directory/containing/uploads. The '-type f' part tells find to only return files (rather than directories, for instance), as we are only really interested in files. The first of these commands will list all of the files more than 100 megabytes in size, and the second lists all files that haven't been modified in the last 90 days. The -ls argument at the end of these provides us with more information about the files, rather than just their names.
If you are happy to move the files that it flags up, you can make find do this also. Using the -exec flag you can request that find runs a command on each file. Here's an example that would move the files larger than 100 megabytes to another directory. Be sure not to ommit the \; punctuation at the end.
find /directory/containing/uploads -type f -size +100M -exec mv '{}'
/mnt/large_external_harddrive/ \;
Now that you have an idea of what is using the space let's cover the basic short term fixes available. Here I'll briefly cover moving, removing and compressing data.
Please bear in mind that by removing the wrong files and directories you risk not only losing valuable data, you also put the integrity of your system at risk. If in doubt, always check before removing a file or a directory.
If you have enough space available on another filesystem, and you do not want to remove what you have isolated as the most obvious cause, feel free to move (mv) the files or directories to another location.
mv /var/log/big.log /mnt/large_external_harddrive/
If you want to you can create symbolic link from the old location to where it now resides; this way it will continue to be accessible from the old path.
ln -s /mnt/large_external_harddrive/big.log /var/log/big.log
To simply truncate a file (reduce to nothing in size), simply redirect nothing into it. Like so:
echo > /var/log/big.log
Or, more simply:
> /var/log/big.log
This saves you the hassle of recreating it with the correct name and file permissions after simply removing it.
Finally, to compress data use the gzip command simply point it at a file. For instance the following command will remove the file it is pointed at, and in its place a compressed version (/var/log/big.log.gz) will exist.
gzip /var/log/big.log
This isn't of much use when the filesystem is already full though - because there is insufficient space to create the compressed file, prior to removing the original.
However, we can use gzip to compress the file to a different place if there isn't enough space to move the file and then compress it. The following example creates the file
/mnt/large_external_harddrive/big.log.gz. Bear in mind that when using gzip with the -c switch, it does not remove the original file.
gzip -c /var/log/big.log > /mnt/large_external_harddrive/big.log.gz
Finally, a note concerning log files, if you remove a log file without restarting the program that is writing to it you may find that 'df' doesn't report the space as free. This is because although link to the file might not exist in the filesystem now, however while a process is still referencing the file the blocks associate with it are still marked as used.
Therefore, if you have to remove a current Apache log file, for instance, be sure to restart Apache afterwards.
Thursday, May 13, 2010
Fighting SPAM
A survey carried out by Microsoft in 2009 suggested that 97% of all emails sent over the internet are unwanted. While definitive spam statistics can be hard to confirm, it certainly amounts to billions of messages a day - with upwards estimates of around 100 billion spam messages sent per day, suggested by some sources. Even using much more conservative estimates, it all adds up to a huge amount of unwanted email worldwide. And some days it can feel like it's all ending up in your mailbox...
While spam may seem like little more than a nuisance, even it's simplest effects can have much more dramatic consequences when considered globally. While it may only take an individual user a few minutes each day to delete unwanted mail, when multiplied by all of the users in the world, the scale of the problem quickly becomes apparent. A recent study suggested that $130 billion was lost by businesses in 2009 due to spam - from the time spent by employees reading and deleting the messages, to recovering from virus infection, financial fraud from phishing sites, and the cost of the extra bandwidth incurred handling the incoming spam.
Much has been done to try and educate users about the dangers of spam - e.g. not opening attachments from unknown/untrusted senders, not confirming bank details via email, etc - but Microsoft's survey still found around 46% of users, even when warned of the risks, would still open executable files, and enter personal details on unvalidated websites. So, clearly, it's best to try and ensure as much spam as possible is stopped before it gets anywhere near a users personal email account.
Being saddled with a name derived from a Monty Python sketch probably hasn't helped spam to be taken very seriously, but if you would like to start getting serious about the problem, and help reduce your users burden of responsibility a little bit, here are some suggestions:
Spamassassin:
The most popular open-source, server-side anti-spam application is Spamassassin. This checks all email against a set of rules and assigns a score to each. If the result exceeds a pre-set limit, the email is tagged as spam and then processed (deleted/quarantined/etc) by a final set of rules.
Installing Spamassassin with a default ruleset can result in a dramatic reduction in spam. It's probably best not to try writing your own rules, unless you're familiar with how Spamassassin works - however, you might want to whitelist and blacklist certain known good/mail addresses or whole domains. For changes to a specific user's email account, you would edit the ~/.spamassassin/user_prefs file, for global changes you would edit the /etc/mail/spamassassin/local.cf file.
The basic format for whitelisting a specific email address is:
whitelist_from my.test@example.co.uk
To whitelist everything from the domain example.co.uk, we would use a wildcard instead:
whitelist_from *@example.co.uk
To blacklist an address or domain, use the directive blacklist_from instead.
To get the best results from Spamassassin, it's a good idea to install MailScanner too. MailScanner is an open-source email security system, which provides an easy to use configuration file (MailScanner.conf) where spam scores and rules can be configured. It also has support for other plugins - the two most useful being:
* Anti-virus - Usually this means using the free Clam Anti-virus package, as there are several options to specific to ClamAV available, but around 25 other packages are supported. The latest virus definitions are updated daily (you can configure it to check more frequently, if required), and positive detections can either be delivered, quarantened or automatically deleted.
* Phishing emails - Checks all URLs against a list of known phishing sites and marks the email's subject line with the warning: {Fraud?} Note: On busy mail servers this option can increase the load considerably so, once enabled, it's best to monitor the server for any sign that performance is being adversely affected.
To make monitoring MailScanner much easier, you can install MailWatch - a web-based front-end, written in PHP. This can be used to display mail queue statistics (full support for this function is only available if you're using Sendmail or Exim), as well as various spam/virus statistics, e.g. total messages, spam/viruses blocked. It also includes Quarantine management, allowing you to release, delete or run sa-learn (a Spamassassin tool for training it's Bayesian filter) across any quarantined messages.
Catchalls:
Normally, you would setup individual email addresses for each user, and any mail sent to a non-existant email address would be rejected. This is the best method to use.
If you use a catchall, it will "catch all" of these emails for the domain, that are sent to non-existant addresses, and accept them for delivery. This makes them a nice, easy target for spammers, as they don't have to use valid email addresses to get their spam through to your server. This then forces the server to accept and process a much higher volume of mail, pushing the load up and eating up resources, as it churns through all this excess mail. The result is a hugely bloated mailbox, full of spam, which usually ends up being ignored as no one can be bothered to go through all the messages to locate any legitimate ones.
The simple solution to this problem is: catchalls are evil, so DON'T use them!
Dictionary Harvest Attack:
This is a form of brute force attack used by spammers to collect valid email addresses, which will then be used as sender addresses for spam. This involves running through various permutations of common addresses, sending a tiny message - usually just a helo/ehlo - to avoid triggering anti-spam software. Any addresses that are accepted for delivery are assumed to be valid and are added to a list of addresses. You can usually tell this happening by checking your mail logs, where you will a large volume of alphabetically sequential email addresses logged and rejected, such as:
jack@mydomain.com
jill@mydomain.com
john@mydomain.com
john.a@mydomain.com
john.b@mydomain.com
The easiest way to spot this sort of behaviour is to install and configure an application such as Logwatch, which will email you a summary of the logs each day. If the connections come from the same IP address(es), or particular IP ranges, you can then block them with your firewall.
SPF Records:
This won't reduce the amount of spam received by your server, but it will reduce the amount of spoofed mail received by other servers.
Why should this matter to you? Well, the 'From' address of an email can easily be modified by the sender, allowing a spammer to make the email appear to have been sent from a different address. If they decide to fake, or 'spoof', addresses using domains hosted on your server, it could cause you problems as many mail servers/services will assume you are the source of the spam. This could result in you being black-listed, or in extreme circumstances even subjected to a retaliatory attack by disgruntled server admins or hackers, such as a DDOS attack. So using SPF records is equal parts a kindness to your fellow internet users, and part self-interest in maintaining a trustworthy reputation on the net.
SPF (or Sender Policy Framework) uses DNS entries to publish the IP addresses of the server(s) that are allowed to send mail for a domain. If a mail server then receives a message from the domain, that doesn't come from one of these valid addresses, it's fairly safe to assume it will be spam and can be deleted or quarantined.
The ultimate goal would be for every domain in the world to start validating their mail using SPF, or similar methods. If it's ever achieved, this would effectively mean the end for spam (at least in its current form), but we're a long way from that at the moment, with only around 5% of .com and .net domains currently implementing SPF records.
HELO/EHLO:
Spam is often sent from poorly configured mail servers, so you can set your server to reject any connections that issue an invalid HELO or EHLO message. This is a basic greeting all email servers issue when connecting, so it can allow you to block spammers before they've even connected to your server. However, it can also result in valid email being lost too, as there's no way to guarantee that all legitimate mail servers will be properly configured. This option is probably only worth using if you only want to handle mail sent from a set group of known senders, where you can ensure their server's greetings are valid.
Email Addresses Management:
Use multiple email addresses. When mailing clients or trusted business contacts, use your real email address. When signing up to anything online, or posting on forums, use a different email address. This will allow you to protect your real email address from spam, as there's often no way of knowing if your address details will be treated with respect and not passed on to spammers.
Even unwanted mail from legitimate businesses is spam, so it's not just adult or other 'dodgy' sites that can result in an increase in spam. Many people notice a dramatic increase in spam after signing up to social networking sites, for example. If you suspect signing up to a certain website might result in a lot of spam, create a temporary email just for that particular sign-up. Spamassassin/MailScanner can then be set to apply much more stringent checks to these addresses, and they can be removed if they prove to nothing but a spam trap.
Conclusion:
While these points are far from definitive, they should be enough to start you thinking about how to tackle the problem of spam. Even a few simple changes could make a huge difference, without too much investment of time and effort being required. For further reading, see the links below:
ClamAV - http://www.clamav.net
MailScanner - http://www.mailscanner.info
MailWatch - http://mailwatch.sourceforge.net
Spamassassin - http://spamassassin.apache.org
SPF - http://www.openspf.org
While spam may seem like little more than a nuisance, even it's simplest effects can have much more dramatic consequences when considered globally. While it may only take an individual user a few minutes each day to delete unwanted mail, when multiplied by all of the users in the world, the scale of the problem quickly becomes apparent. A recent study suggested that $130 billion was lost by businesses in 2009 due to spam - from the time spent by employees reading and deleting the messages, to recovering from virus infection, financial fraud from phishing sites, and the cost of the extra bandwidth incurred handling the incoming spam.
Much has been done to try and educate users about the dangers of spam - e.g. not opening attachments from unknown/untrusted senders, not confirming bank details via email, etc - but Microsoft's survey still found around 46% of users, even when warned of the risks, would still open executable files, and enter personal details on unvalidated websites. So, clearly, it's best to try and ensure as much spam as possible is stopped before it gets anywhere near a users personal email account.
Being saddled with a name derived from a Monty Python sketch probably hasn't helped spam to be taken very seriously, but if you would like to start getting serious about the problem, and help reduce your users burden of responsibility a little bit, here are some suggestions:
Spamassassin:
The most popular open-source, server-side anti-spam application is Spamassassin. This checks all email against a set of rules and assigns a score to each. If the result exceeds a pre-set limit, the email is tagged as spam and then processed (deleted/quarantined/etc) by a final set of rules.
Installing Spamassassin with a default ruleset can result in a dramatic reduction in spam. It's probably best not to try writing your own rules, unless you're familiar with how Spamassassin works - however, you might want to whitelist and blacklist certain known good/mail addresses or whole domains. For changes to a specific user's email account, you would edit the ~/.spamassassin/user_prefs file, for global changes you would edit the /etc/mail/spamassassin/local.cf file.
The basic format for whitelisting a specific email address is:
whitelist_from my.test@example.co.uk
To whitelist everything from the domain example.co.uk, we would use a wildcard instead:
whitelist_from *@example.co.uk
To blacklist an address or domain, use the directive blacklist_from instead.
To get the best results from Spamassassin, it's a good idea to install MailScanner too. MailScanner is an open-source email security system, which provides an easy to use configuration file (MailScanner.conf) where spam scores and rules can be configured. It also has support for other plugins - the two most useful being:
* Anti-virus - Usually this means using the free Clam Anti-virus package, as there are several options to specific to ClamAV available, but around 25 other packages are supported. The latest virus definitions are updated daily (you can configure it to check more frequently, if required), and positive detections can either be delivered, quarantened or automatically deleted.
* Phishing emails - Checks all URLs against a list of known phishing sites and marks the email's subject line with the warning: {Fraud?} Note: On busy mail servers this option can increase the load considerably so, once enabled, it's best to monitor the server for any sign that performance is being adversely affected.
To make monitoring MailScanner much easier, you can install MailWatch - a web-based front-end, written in PHP. This can be used to display mail queue statistics (full support for this function is only available if you're using Sendmail or Exim), as well as various spam/virus statistics, e.g. total messages, spam/viruses blocked. It also includes Quarantine management, allowing you to release, delete or run sa-learn (a Spamassassin tool for training it's Bayesian filter) across any quarantined messages.
Catchalls:
Normally, you would setup individual email addresses for each user, and any mail sent to a non-existant email address would be rejected. This is the best method to use.
If you use a catchall, it will "catch all" of these emails for the domain, that are sent to non-existant addresses, and accept them for delivery. This makes them a nice, easy target for spammers, as they don't have to use valid email addresses to get their spam through to your server. This then forces the server to accept and process a much higher volume of mail, pushing the load up and eating up resources, as it churns through all this excess mail. The result is a hugely bloated mailbox, full of spam, which usually ends up being ignored as no one can be bothered to go through all the messages to locate any legitimate ones.
The simple solution to this problem is: catchalls are evil, so DON'T use them!
Dictionary Harvest Attack:
This is a form of brute force attack used by spammers to collect valid email addresses, which will then be used as sender addresses for spam. This involves running through various permutations of common addresses, sending a tiny message - usually just a helo/ehlo - to avoid triggering anti-spam software. Any addresses that are accepted for delivery are assumed to be valid and are added to a list of addresses. You can usually tell this happening by checking your mail logs, where you will a large volume of alphabetically sequential email addresses logged and rejected, such as:
jack@mydomain.com
jill@mydomain.com
john@mydomain.com
john.a@mydomain.com
john.b@mydomain.com
The easiest way to spot this sort of behaviour is to install and configure an application such as Logwatch, which will email you a summary of the logs each day. If the connections come from the same IP address(es), or particular IP ranges, you can then block them with your firewall.
SPF Records:
This won't reduce the amount of spam received by your server, but it will reduce the amount of spoofed mail received by other servers.
Why should this matter to you? Well, the 'From' address of an email can easily be modified by the sender, allowing a spammer to make the email appear to have been sent from a different address. If they decide to fake, or 'spoof', addresses using domains hosted on your server, it could cause you problems as many mail servers/services will assume you are the source of the spam. This could result in you being black-listed, or in extreme circumstances even subjected to a retaliatory attack by disgruntled server admins or hackers, such as a DDOS attack. So using SPF records is equal parts a kindness to your fellow internet users, and part self-interest in maintaining a trustworthy reputation on the net.
SPF (or Sender Policy Framework) uses DNS entries to publish the IP addresses of the server(s) that are allowed to send mail for a domain. If a mail server then receives a message from the domain, that doesn't come from one of these valid addresses, it's fairly safe to assume it will be spam and can be deleted or quarantined.
The ultimate goal would be for every domain in the world to start validating their mail using SPF, or similar methods. If it's ever achieved, this would effectively mean the end for spam (at least in its current form), but we're a long way from that at the moment, with only around 5% of .com and .net domains currently implementing SPF records.
HELO/EHLO:
Spam is often sent from poorly configured mail servers, so you can set your server to reject any connections that issue an invalid HELO or EHLO message. This is a basic greeting all email servers issue when connecting, so it can allow you to block spammers before they've even connected to your server. However, it can also result in valid email being lost too, as there's no way to guarantee that all legitimate mail servers will be properly configured. This option is probably only worth using if you only want to handle mail sent from a set group of known senders, where you can ensure their server's greetings are valid.
Email Addresses Management:
Use multiple email addresses. When mailing clients or trusted business contacts, use your real email address. When signing up to anything online, or posting on forums, use a different email address. This will allow you to protect your real email address from spam, as there's often no way of knowing if your address details will be treated with respect and not passed on to spammers.
Even unwanted mail from legitimate businesses is spam, so it's not just adult or other 'dodgy' sites that can result in an increase in spam. Many people notice a dramatic increase in spam after signing up to social networking sites, for example. If you suspect signing up to a certain website might result in a lot of spam, create a temporary email just for that particular sign-up. Spamassassin/MailScanner can then be set to apply much more stringent checks to these addresses, and they can be removed if they prove to nothing but a spam trap.
Conclusion:
While these points are far from definitive, they should be enough to start you thinking about how to tackle the problem of spam. Even a few simple changes could make a huge difference, without too much investment of time and effort being required. For further reading, see the links below:
ClamAV - http://www.clamav.net
MailScanner - http://www.mailscanner.info
MailWatch - http://mailwatch.sourceforge.net
Spamassassin - http://spamassassin.apache.org
SPF - http://www.openspf.org
Subscribe to:
Posts (Atom)
