Fedora Linux Support Community & Resources Center
  #1  
Old 20th March 2017, 11:05 PM
cuttenweiler Offline
Registered User
 
Join Date: Mar 2017
Location: USA - Virginia
Posts: 2
macossafari
Unhappy [SOLVED] Need actual text LOG files - Journald & Syslog-ng not talking to each other

Journalctl / journald logs aren't going to work for me because I need individual log files that can sent to different areas of our infrastructure for processing / archiving and a ton of other activities. Basically it is a non-starter to change the way the workflow around these logs work.

I have syslog-ng installed because the back-end systems us it to centralize files. I can't change that either - so syslog-ng is what I have to use.

It is professed that syslog-ng can read journald directly with the system() source, but its just not happening. I see the logging information in journalctl, but can't get it to simply WRITE to a disk location to save my life. Most of the work going on is Perl, executed in cron - this works fine on other machines not running systemd/journald. It can't be this hard - what am I missing?

Thanks in advance - Please see my configs below

(journald.conf is default - no lines uncommented.. and Fedora 25's systemd doesn't seem to honor the ForwardToSyslog switch anyway.)

syslog-ng.conf
Code:
@version:3.9
@include "scl.conf"

# syslog-ng configuration file.
#
# This should behave pretty much like the original syslog on RedHat. But
# it could be configured a lot smarter.
#
# See syslog-ng(8) and syslog-ng.conf(5) for more information.
#
# Note: it also sources additional configuration files (*.conf)
#       located in /etc/syslog-ng/conf.d/

options {
    flush_lines (0);
    time_reopen (10);
    log_fifo_size (1000);
    chain_hostnames (off);
    use_dns (no);
    use_fqdn (no);
    create_dirs (no);
    keep_hostname (yes);
};

source s_sys {
    system();
    internal();
    # udp(ip(0.0.0.0) port(514));
};

destination d_cons { file("/dev/console"); };
destination d_mesg { file("/var/log/messages"); };
destination d_auth { file("/var/log/secure"); };
destination d_mail { file("/var/log/maillog" flush_lines(10)); };
destination d_spol { file("/var/log/spooler"); };
destination d_boot { file("/var/log/boot.log"); };
destination d_cron { file("/var/log/cron"); };
destination d_kern { file("/var/log/kern"); };
destination d_mlal { usertty("*"); };

filter f_kernel     { facility(kern); };
filter f_default    { level(info..emerg) and
                        not (facility(mail)
                        or facility(authpriv) 
                        or facility(cron)); };
filter f_auth       { facility(authpriv); };
filter f_mail       { facility(mail); };
filter f_emergency  { level(emerg); };
filter f_news       { facility(uucp) or
                        (facility(news) 
                        and level(crit..emerg)); };
filter f_boot   { facility(local7); };
filter f_cron   { facility(cron); };
filter f_appservers { facility(local4); host("(^jenkins|app$|app_d$)"); };

# Custom additions to support application
destination d_speciallogs { file("/var/log/perl_logs/${PROGRAM}.log" create_dirs(yes) owner("automagic") group("automagic") perm(0660) dir-owner("automagic") dir-group("automagic") dir-perm(770) ); };
destination d_DEBUG { file("/var/log/perl_logs/DEBUG.log" create_dirs(yes) owner("automagic") group("automagic") perm(0660) dir-owner("automagic") dir-group("automagic") dir-perm(770) ); };
destination d_INFO { file("/var/log/perl_logs/SYSINFO.log" create_dirs(yes) owner("automagic") group("automagic") perm(0660) dir-owner("automagic") dir-group("automagic") dir-perm(770) ); };
destination d_ERROR { file("/var/log/perl_logs/ERROR.log" create_dirs(yes) owner("automagic") group("automagic") perm(0660) dir-owner("automagic") dir-group("automagic") dir-perm(770) ); };
destination d_central_syslog { syslog("logcentral.server" transport("tcp")); };

#log { source(s_sys); filter(f_kernel); destination(d_cons); };
log { source(s_sys); filter(f_kernel); destination(d_kern); };
log { source(s_sys); filter(f_default); destination(d_mesg); };
log { source(s_sys); filter(f_auth); destination(d_auth); };
log { source(s_sys); filter(f_mail); destination(d_mail); };
log { source(s_sys); filter(f_emergency); destination(d_mlal); };
log { source(s_sys); filter(f_news); destination(d_spol); };
log { source(s_sys); filter(f_boot); destination(d_boot); };
log { source(s_sys); filter(f_cron); destination(d_cron); };


# Modified for custom Application
log { source(s_sys); filter(f_appservers); destination(d_central_syslog); destination(d_speciallogs); };
log { source(s_sys); filter(f_appservers); filter( f_DEBUG ); destination(d_DEBUG); };
log { source(s_sys); filter(f_appservers); filter( f_INFO ); destination(d_INFO); };
log { source(s_sys); filter(f_appservers); filter( f_ERROR ); destination(d_ERROR); };

filter f_DEBUG { level(debug..emerg); };
filter f_INFO { level(info..emerg); };
filter f_ERROR { level(err..emerg); };

# Source additional configuration files (.conf extension only)
@include "/etc/syslog-ng/conf.d/*.conf"


# vim:ft=syslog-ng:ai:si:ts=4:sw=4:et:

Last edited by cuttenweiler; 22nd March 2017 at 05:52 PM. Reason: Marked SOLVED
Reply With Quote
  #2  
Old 21st March 2017, 01:55 AM
PabloTwo Offline
"Registered User" T-Shirt Winner
 
Join Date: Mar 2007
Location: Seville, FL
Posts: 7,410
linuxchrome
Re: Need actual text LOG files - Journald & Syslog-ng not talking to each other

You can have the traditional plain text /var/log/messages* files by installing the rsyslog package.
Reply With Quote
  #3  
Old 21st March 2017, 12:45 PM
frobert Offline
Registered User
 
Join Date: Mar 2017
Location: Hungary
Posts: 2
linuxubuntufirefox
Re: Need actual text LOG files - Journald & Syslog-ng not talking to each other

Hi,

Try the following, and see if it helps:
* Try temporarily disable Selinux/apparmor. If it helps, adjust your selinux/apparmor settings to allow syslog-ng to write the log files.
* Try to remove the host filter from the f_appservers filter. I'm not sure that the host information is available in the journal.
* The f_DEBUG and other filters are defined after they are used. It might not be an issue, but move them above the log statements.

HTH,

Robert
Reply With Quote
  #4  
Old 21st March 2017, 02:25 PM
PabloTwo Offline
"Registered User" T-Shirt Winner
 
Join Date: Mar 2007
Location: Seville, FL
Posts: 7,410
linuxchrome
Re: Need actual text LOG files - Journald & Syslog-ng not talking to each other

I guess I didn't pay enough attention to this statement:
Quote:
(journald.conf is default - no lines uncommented.. ...
The default is:
Code:
grep ToSyslog /etc/systemd/journald.conf 
#ForwardToSyslog=no
Reply With Quote
  #5  
Old 21st March 2017, 02:38 PM
cuttenweiler Offline
Registered User
 
Join Date: Mar 2017
Location: USA - Virginia
Posts: 2
macossafari
Re: Need actual text LOG files - Journald & Syslog-ng not talking to each other

Robert for the win!

The issue was that the journald does not handle the host entry in the f_appservers filter!

So I changed this:

Code:
filter f_appservers { facility(local4); host("(^jekins|app$|app_d$)"); };
To this:

Code:
filter f_appservers { facility(local4); };
... and it all started working.

Right before you replied I noticed that everything was going to the /var/log/messages, just not the specified destinations, so I was going to " syslog-ng -Fevd" to trace the debug once I noticed that, but your suggestion helped keep me out of the weeds.

Also, apparently syslog-ng.conf is evaluated as a whole, not in sequenced order. I inherited this file, but I'll be re-arranging it to be more human readable as well.

I really can't thank you enough Robert!

Cheers!


Quote:
Originally Posted by frobert View Post
Hi,

Try the following, and see if it helps:
* Try temporarily disable Selinux/apparmor. If it helps, adjust your selinux/apparmor settings to allow syslog-ng to write the log files.
* Try to remove the host filter from the f_appservers filter. I'm not sure that the host information is available in the journal.
* The f_DEBUG and other filters are defined after they are used. It might not be an issue, but move them above the log statements.

HTH,

Robert
Reply With Quote
  #6  
Old 22nd March 2017, 03:51 PM
frobert Offline
Registered User
 
Join Date: Mar 2017
Location: Hungary
Posts: 2
linuxubuntufirefox
Re: Need actual text LOG files - Journald & Syslog-ng not talking to each other

I'm really glad that it's working now, let us know if you need further help with syslog-ng!
Reply With Quote
Reply

Tags
actual, files, journald, log, syslogng, talking, text

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Will Un-installing VMware Player remove my VMDK Files??? my actual OS VM's? cyberdome Using Fedora 2 19th February 2016 06:02 PM
F19 Logrotate of syslog files not working cmaxeiner Using Fedora 3 13th May 2014 09:27 AM
journald and syslog question? fsckPotthead Using Fedora 1 9th May 2014 03:45 AM
SVG Files - no text. Adunaic Using Fedora 2 1st November 2010 04:18 PM
finding files with wildcards in the actual filename kyle775 Using Fedora 2 5th September 2010 05:26 AM


Current GMT-time: 02:28 (Tuesday, 28-03-2017)

TopSubscribe to XML RSS for all Threads in all ForumsFedoraForumDotOrg Archive
logo

All trademarks, and forum posts in this site are property of their respective owner(s).
FedoraForum.org is privately owned and is not directly sponsored by the Fedora Project or Red Hat, Inc.

Privacy Policy | Term of Use | Posting Guidelines | Archive | Contact Us | Founding Members

Powered by vBulletin® Copyright ©2000 - 2012, vBulletin Solutions, Inc.

FedoraForum is Powered by RedHat