Systemctl error no space left on device

[willc0de4food]

Hello,

I've recently started to see this error message when I try to use sytemctl to start a service (any service).

[root@localhost ~]# systemctl start httpd.service
Error: No space left on device
^C
[root@localhost ~]# systemctl start postgresql.service
Error: No space left on device
Job failed. See system journal and 'systemctl status' for details.
[root@localhost ~]# systemctl status postgresql.service
postgresql.service - PostgreSQL database server
Loaded: loaded (/usr/lib/systemd/system/postgresql.service; disabled)
Active: failed (Result: exit-code) since Wed, 22 Aug 2012 10:21:23 -0400; 32s ago
Process: 31448 ExecStartPre=/usr/bin/postgresql-check-db-dir ${PGDATA} (code=exited, status=1/FAILURE)
CGroup: name=systemd:/system/postgresql.service

[root@localhost ~]# df -h
Filesystem Size Used Avail Use% Mounted on
rootfs 146G 28G 117G 20% /
devtmpfs 2.9G 0 2.9G 0% /dev
tmpfs 2.9G 2.9M 2.9G 1% /dev/shm
tmpfs 2.9G 2.3M 2.9G 1% /run
/dev/mapper/VolGroup-lv_root 146G 28G 117G 20% /
tmpfs 2.9G 0 2.9G 0% /sys/fs/cgroup
tmpfs 2.9G 0 2.9G 0% /media
/dev/sda1 485M 113M 347M 25% /boot
/dev/mapper/VolGroup-lv_home 306G 141G 149G 49% /home

I've read people mention SELinux, I've tried the above both with SELinux disabled & permissive. I have no issue with free space, so I'm not sure what the true problem is here :| so any help would be appreciated.

[DBelton]

You may be running out of inodes instead of actual disk space.

Try running df -i and see what your inode usage is.

[george_toolan]

Did you enable quotas?

[willc0de4food]

Originally Posted by DBelton

You may be running out of inodes instead of actual disk space.

Try running df -i and see what your inode usage is.

[root@localhost ~]# df -i
Filesystem Inodes IUsed IFree IUse% Mounted on
rootfs 9682944 483049 9199895 5% /
devtmpfs 743466 490 742976 1% /dev
tmpfs 746381 29 746352 1% /dev/shm
tmpfs 746381 595 745786 1% /run
/dev/mapper/VolGroup-lv_root 9682944 483049 9199895 5% /
tmpfs 746381 12 746369 1% /sys/fs/cgroup
tmpfs 746381 1 746380 1% /media
/dev/sda1 128016 320 127696 1% /boot
/dev/mapper/VolGroup-lv_home 20316160 425427 19890733 3% /home

Originally Posted by george_toolan

Did you enable quotas?

I've done nothing with quotas.

[DBelton]

my next thought is a corrupted filesystem.

reboot, and when the grub menu comes up, hit "e" to edit your kernel line. On the end of the line that starts with linux /vmlinux- just add the parameter fsck.mode=force then boot your machine. It should force a filesystem check when it boots, before remounting your root filesystem read/write.

[willc0de4food]

Adding the fsck check to the kernel line did run the check, but didn't fix the problem :[
However, I have had this exact same issue for quite some time.. http://forums.fedoraforum.org/showthread.php?t=277866 >.> I've just always been to Ctrl+D to continue out of it and the system has seemed to work fine. In finding that thread, though, I *believe* I was able to fix that issue and will see if the problem has been resolved (there were TONS of errors :-X )

Update: the problem persists. :[

[kruton]

This is probably due to inotify running out of resources to watch directories.

I can't post the solution to the forum because the variable name contains a banned word, but look at the Crashplan support page.