Fedora 16 machine
$ uname -a
Linux localhost.localdomain 3.3.0-8.fc16.i686 #1 SMP Thu Mar 29 18:33:55 UTC 2012 i686 i686 i386 GNU/Linux
$ find /proc/$(pidof rsyslogd)/task/*/fd/* -type l -exec ls -l {} \;
lr-x------. 1 root root 64 Apr 7 07:58 /proc/1164/task/1164/fd/0 -> /dev/null
l-wx------. 1 root root 64 Apr 7 07:58 /proc/1164/task/1164/fd/1 -> /dev/null
l-wx------. 1 root root 64 Apr 7 07:58 /proc/1164/task/1164/fd/2 -> /dev/null
lrwx------. 1 root root 64 Apr 7 07:58 /proc/1164/task/1164/fd/3 -> socket:[7047]
lr-x------. 1 root root 64 Apr 7 07:58 /proc/1164/task/1164/fd/4 -> /proc/kmsg
lr-x------. 1 root root 64 Apr 7 07:58 /proc/1164/task/1169/fd/0 -> /dev/null
l-wx------. 1 root root 64 Apr 7 07:58 /proc/1164/task/1169/fd/1 -> /dev/null
l-wx------. 1 root root 64 Apr 7 07:58 /proc/1164/task/1169/fd/2 -> /dev/null
lrwx------. 1 root root 64 Apr 7 07:58 /proc/1164/task/1169/fd/3 -> socket:[7047]
lr-x------. 1 root root 64 Apr 7 07:58 /proc/1164/task/1169/fd/4 -> /proc/kmsg
lr-x------. 1 root root 64 Apr 7 07:58 /proc/1164/task/1170/fd/0 -> /dev/null
l-wx------. 1 root root 64 Apr 7 07:58 /proc/1164/task/1170/fd/1 -> /dev/null
l-wx------. 1 root root 64 Apr 7 07:58 /proc/1164/task/1170/fd/2 -> /dev/null
lrwx------. 1 root root 64 Apr 7 07:58 /proc/1164/task/1170/fd/3 -> socket:[7047]
lr-x------. 1 root root 64 Apr 7 07:58 /proc/1164/task/1170/fd/4 -> /proc/kmsg
lr-x------. 1 root root 64 Apr 7 07:58 /proc/1164/task/1171/fd/0 -> /dev/null
l-wx------. 1 root root 64 Apr 7 07:58 /proc/1164/task/1171/fd/1 -> /dev/null
l-wx------. 1 root root 64 Apr 7 07:58 /proc/1164/task/1171/fd/2 -> /dev/null
lrwx------. 1 root root 64 Apr 7 07:58 /proc/1164/task/1171/fd/3 -> socket:[7047]
lr-x------. 1 root root 64 Apr 7 07:58 /proc/1164/task/1171/fd/4 -> /proc/kmsg
Some things I tried to do to fix it already...
1. remove/reinstall rsyslog with yum
2. systemctl restart rsyslog.service -f
systemctl restart syslog.socket -f
systemctl restart syslog.target -f
3. I created /var/log/{wtmp,lastlog,secure,messages} (with the same permissions) and restarted the daemon after I noticed rsyslog stopped logging things.
4. logrotate -f /etc/logrotate.conf
After numerous reboots I still cant get anything. When the system boots up I get an error, Console System Logger Failed, but when the machine starts `service rsyslog status` return `active (running)` and the `logger` command still exists (but it goes to /dev/null like everything else, not even emergency messages show up any more).
I'm fairly certain the inode for /var/log and the files within is/was referenced/saved somewhere and after deleting /var/log (i know, i know) rsyslog has started sending everything to /dev/null instead.
`wtmp`, `yum.log` and Xorg have since logged things to /var/log, but I'm not sure they came through rsyslog.
Any ideas?
---------- Post added at 01:11 PM ---------- Previous post was at 01:08 PM ----------
By the way, here's the output of that find command on a machine with functioning rsyslog
$ find /proc/$(pidof rsyslogd)/task/*/fd/* -type l -exec ls -l {} \;
lr-x------ 1 root root 64 Apr 7 07:15 /proc/1964/task/1964/fd/0 -> /dev/null
l-wx------ 1 root root 64 Apr 7 07:15 /proc/1964/task/1964/fd/1 -> /dev/null
l-wx------ 1 root root 64 Apr 7 07:15 /proc/1964/task/1964/fd/2 -> /dev/null
lrwx------ 1 root root 64 Apr 7 07:15 /proc/1964/task/1964/fd/3 -> socket:[912]
lr-x------ 1 root root 64 Apr 7 07:15 /proc/1964/task/1964/fd/4 -> /proc/kmsg
l-wx------ 1 root root 64 Apr 7 07:15 /proc/1964/task/1964/fd/5 -> /var/log/messages
l-wx------ 1 root root 64 Apr 7 07:15 /proc/1964/task/1964/fd/6 -> /var/log/boot.log
l-wx------ 1 root root 64 Apr 7 07:15 /proc/1964/task/1964/fd/7 -> /var/log/cron
l-wx------ 1 root root 64 Apr 7 07:15 /proc/1964/task/1964/fd/8 -> /var/log/secure
l-wx------ 1 root root 64 Apr 7 07:15 /proc/1964/task/1964/fd/9 -> /var/log/maillog
lr-x------ 1 root root 64 Apr 7 07:15 /proc/1964/task/1981/fd/0 -> /dev/null
l-wx------ 1 root root 64 Apr 7 07:15 /proc/1964/task/1981/fd/1 -> /dev/null
l-wx------ 1 root root 64 Apr 7 07:15 /proc/1964/task/1981/fd/2 -> /dev/null
lrwx------ 1 root root 64 Apr 7 07:15 /proc/1964/task/1981/fd/3 -> socket:[912]
lr-x------ 1 root root 64 Apr 7 07:15 /proc/1964/task/1981/fd/4 -> /proc/kmsg
l-wx------ 1 root root 64 Apr 7 07:15 /proc/1964/task/1981/fd/5 -> /var/log/messages
l-wx------ 1 root root 64 Apr 7 07:15 /proc/1964/task/1981/fd/6 -> /var/log/boot.log
l-wx------ 1 root root 64 Apr 7 07:15 /proc/1964/task/1981/fd/7 -> /var/log/cron
l-wx------ 1 root root 64 Apr 7 07:15 /proc/1964/task/1981/fd/8 -> /var/log/secure
l-wx------ 1 root root 64 Apr 7 07:15 /proc/1964/task/1981/fd/9 -> /var/log/maillog
lr-x------ 1 root root 64 Apr 7 07:15 /proc/1964/task/1982/fd/0 -> /dev/null
l-wx------ 1 root root 64 Apr 7 07:15 /proc/1964/task/1982/fd/1 -> /dev/null
l-wx------ 1 root root 64 Apr 7 07:15 /proc/1964/task/1982/fd/2 -> /dev/null
lrwx------ 1 root root 64 Apr 7 07:15 /proc/1964/task/1982/fd/3 -> socket:[912]
lr-x------ 1 root root 64 Apr 7 07:15 /proc/1964/task/1982/fd/4 -> /proc/kmsg
l-wx------ 1 root root 64 Apr 7 07:15 /proc/1964/task/1982/fd/5 -> /var/log/messages
l-wx------ 1 root root 64 Apr 7 07:15 /proc/1964/task/1982/fd/6 -> /var/log/boot.log
l-wx------ 1 root root 64 Apr 7 07:15 /proc/1964/task/1982/fd/7 -> /var/log/cron
l-wx------ 1 root root 64 Apr 7 07:15 /proc/1964/task/1982/fd/8 -> /var/log/secure
l-wx------ 1 root root 64 Apr 7 07:15 /proc/1964/task/1982/fd/9 -> /var/log/maillog
lr-x------ 1 root root 64 Apr 7 07:15 /proc/1964/task/1983/fd/0 -> /dev/null
l-wx------ 1 root root 64 Apr 7 07:15 /proc/1964/task/1983/fd/1 -> /dev/null
l-wx------ 1 root root 64 Apr 7 07:15 /proc/1964/task/1983/fd/2 -> /dev/null
lrwx------ 1 root root 64 Apr 7 07:15 /proc/1964/task/1983/fd/3 -> socket:[912]
lr-x------ 1 root root 64 Apr 7 07:15 /proc/1964/task/1983/fd/4 -> /proc/kmsg
l-wx------ 1 root root 64 Apr 7 07:15 /proc/1964/task/1983/fd/5 -> /var/log/messages
l-wx------ 1 root root 64 Apr 7 07:15 /proc/1964/task/1983/fd/6 -> /var/log/boot.log
l-wx------ 1 root root 64 Apr 7 07:15 /proc/1964/task/1983/fd/7 -> /var/log/cron
l-wx------ 1 root root 64 Apr 7 07:15 /proc/1964/task/1983/fd/8 -> /var/log/secure
l-wx------ 1 root root 64 Apr 7 07:15 /proc/1964/task/1983/fd/9 -> /var/log/maillog
---------- Post added at 01:15 PM ---------- Previous post was at 01:11 PM ----------
ls /var/log -latsri
total 592
136252 0 -rw-r--r--. 1 root root 0 Apr 7 00:06 boot.log-20120407
158117 0 -rw-r--r--. 2 root root 0 Apr 7 02:56 boot.log
132609 4 drwxrwx--T. 2 root gdm 4096 Apr 7 02:57 gdm
164644 8 -rw-r--r--. 1 root utmp 7680 Apr 7 02:57 wtmp
156668 0 -rw-r--r--. 1 root root 0 Apr 7 02:57 pm-powersave.log
8193 4 drwxr-xr-x. 20 root root 4096 Apr 7 04:48 ..
157138 0 -rw-r--r--. 1 root root 0 Apr 7 05:13 messages-20120407
157172 100 -rw-r-----. 1 root root 101785 Apr 7 05:32 lynis-report.dat
143405 328 -rw-r-----. 1 root root 332638 Apr 7 05:32 lynis.log
157337 0 -rw-r--r--. 1 root root 0 Apr 7 05:37 spooler-20120407
157324 0 -rw-r--r--. 1 root root 0 Apr 7 05:37 secure-20120407
157325 0 -rw-r--r--. 1 root root 0 Apr 7 05:37 maillog-20120407
157336 0 -rw-r--r--. 1 root root 0 Apr 7 05:37 cron-20120407
157170 0 -rw-r--r--. 1 root root 0 Apr 7 05:43 nodejs
134838 4 drwx------. 2 root lp 4096 Apr 7 05:49 cups
157476 0 -rw-r--r--. 1 root root 0 Apr 7 05:49 secure
157474 0 -rw-r--r--. 1 root root 0 Apr 7 05:49 messages
157916 0 -rw-r--r--. 1 root root 0 Apr 7 05:49 maillog
158094 0 -rw-r--r--. 1 root root 0 Apr 7 05:49 spooler
158171 0 -rw-r--r--. 1 root root 0 Apr 7 05:49 cron
162070 0 -rw-r--r--. 1 root root 0 Apr 7 05:49 wtmp-20120407
164944 0 -rw-rw-r--. 1 root root 0 Apr 7 06:00 lastlog
170403 0 -rw-r--r--. 1 root root 0 Apr 7 06:00 btmp-20120407
156667 4 -rw-------. 1 root root 815 Apr 7 06:52 yum.log-20120407
136249 48 -rw-r--r--. 1 root root 46908 Apr 7 07:47 Xorg.0.log.old
157171 44 -rw-r--r--. 1 root root 42575 Apr 7 07:47 Xorg.1.log
138635 44 -rw-r--r--. 1 root root 44200 Apr 7 07:57 Xorg.0.log
174745 0 -rw-------. 1 root root 0 Apr 7 08:06 yum.log
174746 0 -rw-r--r--. 1 root utmp 0 Apr 7 08:06 btmp
132797 4 drwxr-xr-x. 4 root root 4096 Apr 7 08:06 .
Linear Mode
