What do you guys do with your log files?

[Prion]

I'm curious as to what you guys do with your system log files (/var/log).

Do you examine them periodically? If so how do you go about it? (i.e. do you simply go through them with a text editor or parse them with grep for specific keywords?)

Do you change their settings so that they are set to append-only so that they can't be overwritten? (to prevent something like: cat /dev/null > /var/log/<filename>). If so how do you go about dealing with cron jobs that automatically archive/rotate your log files with logrotate?

Do you copy your log files onto a different server? Or send them to a specific tty terminal?

Even then isn't there a way for ordinary users to spoof the entries in the log files by using logger? (For example, to make it look as though another user is trying to brute force their way through your root password).

Basically, what's the best policy to have with log files?

[blammo]

I got an old RH7.3 box that acts as a logging server for my network and all logs are pumped to it. I have snort running on all Linux boxes and Eventslog on Windows. Everything works nicely and I just tail the messages on a monitor where I can see them from my desk. The only time I'll really look at the monitor is when snort starts beeping and highlighting in red any possible exploits. On this box I also have Nagios and Nessus installed checking services and vulnerabilities on all machines. I don't really look at the logs for trends or anything. I do have it set where I get a call and/or email if anything goes wrong. That's basically it.

[Jman]

I let them be rotated automatically with logrotate.

If I have problems or concerns I load them up in a text editor and search. Maybe I should be more security conscious but I mostly use them for debugging problems.

The rpmpkgs log is handy for checking for installed packages without a slow rpm -qa query.

[Uhlix]

I got an old RH7.3 box that acts as a logging server for my network and all logs are pumped to it. I have snort running on all Linux boxes and Eventslog on Windows. Everything works nicely and I just tail the messages on a monitor where I can see them from my desk. The only time I'll really look at the monitor is when snort starts beeping and highlighting in red any possible exploits. On this box I also have Nagios and Nessus installed checking services and vulnerabilities on all machines. I don't really look at the logs for trends or anything. I do have it set where I get a call and/or email if anything goes wrong. That's basically it.

That sounds like a pretty sweet set up. :D

[crackers]

I actually do not recommend looking at log files with a text editor, unless you specify "read only" - and this is especially true on the "active" (not rotated) files. Remember, those files are live and are constantly being updated. If you open them in an editor, two things can happen:

1) you get a file lock and they can't be updated (syslogd would probably throw a hairball)
2) you accidentally make a change and over-write the now-changed file and you just lost all the recent entries

Using a non-editing text viewer is recommended - less works quite well in a terminal.

[blammo]

That sounds like a pretty sweet set up. :D
I got a little perl script that colorizes these logs for me so that every hostname has its own color and that services are defined simalarly. Sendmail=blue, FTP=green, Doman=cyan, Snort=red w/console beep, etc. The thing is that even though hundreds of messages scroll by during the day, I stilll get a "feel" for the network. I'll know that if I get an attack-response alert from snort every morning from the same host on the network, it will be just Joe Regular visiting his favorite misconfigured site. The logs all follow a pattern and if something ain't right, I'll notice it right away.

Other benefits to remote logging are management, auditing, and the look on the face of Mr. Hacker who has just realized that his break-in and transactions have all been logged to a remote server. The cons are the time and pain of setting it all up, but after you do, it pretty much manages itself.

[RedFedora]

I usually let logoate handle them.. On slow days I'll skim through the logs, looking for
anything unusual. Then I eat them. Log files go great with chocolate milk.

[Mutsumi]

I usually let logoate handle them.. On slow days I'll skim through the logs, looking for
anything unusual. Then I eat them. Log files go great with chocolate milk.

Jejeje LOL