Firstly, each entry in your log files will have a date stamp. You can use the following 'sed' command to find out the start and end dates of the log:
root@server [~]> sed -n -e '1p' -e '$p' /var/log/messages.3
Jul 4 04:02:14 170094 syslogd 1.4.1: restart.
Jul 11 04:01:42 170094 yum: net-tools-1.60-81.el5.x86_64: 100
root@server [~]>
By performing a directory listing these date stamps can be used to show you when a log file was last modified:
root@server [~]> ls -l /var/log/messages*
-rw------- 1 root root 2244927 Jul 30 15:20 /var/log/messages
-rw------- 1 root root 4663646 Jul 25 04:02 /var/log/messages.1
-rw------- 1 root root 7281102 Jul 18 04:00 /var/log/messages.2
-rw------- 1 root root 2362032 Jul 11 04:01 /var/log/messages.3
-rw------- 1 root root 3647411 Jul 4 04:01 /var/log/messages.4
Here we can see that the file messages.4 is almost guaranteed to go back to June 27. But when you encounter a single log file without any pattern of rotations it is often useful to grab the first few entries of it to determine when it begins. This is where the command 'head' comes in useful. By default it will take the first 10 lines from a file, or standard input if you're piping to it, otherwise you can set the number of lines like in the example below.
root@server [~]> head -5 /tmp/zabbix_agentd.log
5578:20100730:090202 zabbix_agentd started. ZABBIX 1.4.6.
5579:20100730:090202 zabbix_agentd collector started
5581:20100730:090202 zabbix_agentd listener started
5583:20100730:090202 zabbix_agentd listener started
5585:20100730:090202 zabbix_agentd listener started
Similar to head, the command 'tail' allows you to display the bottom lines from files. Tail also includes a switch that allows it to 'follow' a file as it is being written to. This is useful when you're troubleshooting a running service - such as the Apache while a PHP script is failing to work correctly - as it allows you to see the logging as it is generated. To use the follow feature simply add the -f switch when using tail, like in the command below:
root@server [~]> tail -f error.log
Sometimes it is useful to makes copies of sections of a log file when investigating a problem. Below is an example of how you would take the log entries from July 29 to July 30 from an Apache error log and copy them into another file:
root@server [~]> sed -n '/\[Thu Jul 29/,/\[Fri Jul 30/p' error.log > /tmp/errorLog.text
If you've ever had an obscene number of requests from the same IP or subnet hitting your machine you might be familiar with the necessity of extracting proof from the log files in order to submit an abuse claim to the IP provider. This is very straight forward and simply involves searching for the IP address, or the common part of the subnet, and directing the output to another file.
root@server [~]> egrep '10.0(\.[0-2]?[0-9]{1,2}){2}' access.log > /tmp/badAccessLog.excerpt
Furthermore, to simply search for an IP address use the following regular expression - this can be useful when searching for excessive connections. Here I've included a hat (^) at the beginning of the match, now it will only match possible IP address that appear at the start of a line - useful when browser revision version numbers can take the appearance of IP addresses:
root@server [~]> egrep -o '^([0-2]?[0-9]{1,2}\.){3}[0-2]?[0-9]{1,2}' access.log
The following command will provide you with an ordered list of the biggest hitters:
root@server [~]> egrep -o '^([0-2]?[0-9]{1,2}\.){3}[0-2]?[0-9]{1,2}' access.log | sort | uniq -c | sort -n
3 10.0.0.17
10 127.0.0.1
89 10.0.0.2
Sometimes you'll find that it's useful to reverse the output, which can be done by piping the output into 'tac'. Another use for tac is when you're grep'ing for something that you know will appear near the end of a very large log file. To speed up the retrieval of the appropriate entries, tac the file, then grep for the string you#re looking for - then tee direct it into a file (this way you'll know when you've caught it). Finally tac the output file to see the output in correct order. Here's an example.
root@server [~]> tac /var/log/maillog | grep 'me@lampguru' | tee /tmp/file.tmp
Jul 30 16:22:58 pulse sendmail[6446]: o6UFMtQu006444: to=
Jul 30 16:00:15 pulse sendmail[5381]: o6UF0Cl4005378: to=
root@server [~]> tac /tmp/file.tmp
Jul 30 16:00:15 pulse sendmail[5381]: o6UF0Cl4005378: to=
Jul 30 16:22:58 pulse sendmail[6446]: o6UFMtQu006444: to=
Finally, when using the 'less' command to view large files include the -n switch to turn off line numbering as this will improve performance.
0 comments:
Post a Comment