The general format of a Syslogd2 input command is:

--<keyword> = <primary-parameter>,<option-list>

The primary parameter is the 'source' of the data (a filename, socket specification, named-pipe filepath, etc). There are two threadpools that use a 'null' (empty) primary parameter (the kernel and journal input threadpools). In the case of the kernel threadpool, either input comes from a file specified in the kernel-headers or from a system-call, making a primary parameter meaningless. In the case of the journal threadpool, input comes from the systemd-journal API which (again) makes the concept of a primary parameter meaningless.

The option-list is a comma-separated list of options, each of which is a keyword (and therefore parsed as non-case-sensitive). The options in this list will be a mixture of boolean values, integers, strings, keywords, lists and strings depending on their purpose. The order in which options are listed does not matter. If an option is duplicated, the first entry (left-to-right) will be used.

This is a good time to remember that Syslogd2 provides access to a number of 'extra' facilities (32 by default: 'extra0' through 'extra31') that can be used as clean 'scratchpads' to keep incoming data-streams clear of the more traditional facilities that might otherwise 'corrupt' the incoming data.

The number of these 'extra' facilities defaults to 32, but can range from 0 [zero] through 1000 [one thousand] when set at compile-time. There are no run-time confiuration options for these 'extra' facilities, but they can be accessed and used at run-time as you will see throughout this webpage.

Rules for setting boolean values

All boolean values in Syslogd2 primary configuration files (excluding secondary files such as filter-files, cache-file, etc) take an optional parameter. This is because of the difficulty around specifiying that DNS should be disabled for a connection after it has been globally enabled. (There is simply no way to provide a syntax that says to positively return a boolean value to its default state after it has been toggled, then inherited.)
The parameter to enable any boolean value is one of '1', 'y', or 'yes'. Omitting the parameter will also set the booelan value. To disable the booelan value, use '0', 'n' or 'no'.

... boolean, boolean = y, boolean = 1, boolean = yes ## these settings enable the value 'boolean'
... boolean = n, boolean = 0, boolean = no ## these settings disable the value 'boolean'

Note that the options 'y' and 'n' allow for better readability and positive settings (either enable or disable) when the value for boolean may be inherited as either setting.

Rules for setting boolean values using --enable and --disable

The rules for using --enable and --disable (which are themselves boolean in nature) combine with the positive-settings provided by the boolean parameters in predictable ways. Syslogd2 considers that --enable is 'positive' and --disable is 'negative'. Likewise, 'y' or simply setting the value is 'positive' while 'n' is 'negative. Thus:

~ --enable boolean, boolean = y, boolean = 1 ## '=' omitted. these values are enabled.
~ --disable boolean, boolean = y, boolean = 1 ## '=' omitted. these values are disabled.
~ --enable boolean = n, boolean = 0 ## '=' omitted. these values are disabled.
~ --disable boolean = n, boolean = 0 ## '=' omitted. these values are enabled.
<Not mentioning the boolean variable at all> ## boolean retains its default (inherited) state (usually disabled).

Essentially, two 'wrongs' DO make a 'right' (double-negatives cancel each other out).

About Input Options

The option list associated with each input specification in Syslogd2 is a comma-separated list of options from multiple categories that may be freely mixed in any desired sequence. Some of these option categories are unique to specific input methods and are covered in the section that covers that input method. The rest of the options are generic in the sense that they can be used with any input method. These options are covered (by category) below.

Identity Options

Identity options depend entirely upon the connection used to read a particular input source. These options are less about operational settings than they are about distinguishing one connection from another (Examples: port (an inteer) and selection of UDP or TCP (booleans)). The scope of an identity option is limited to a single connection. These options will be covered under individual sections devoted to each threadpool type.

Threadpool-Type-Specific Options

Threadpool-type-specific options depend entirely upon the threadpool-type that is supporting a given input specification. The scope of a threadpool-type-specific option is a single threadpool. Threadpool-type-specific options may be set globally using the --defaults command or per-threadpool using the --ThreadMaps command.

Threadpool-type-specific settings may also (often) be set by individual connection specifications (Examples: pollinterval (an integer) and poll (a boolean)). If this method of setting threadpool parameters is chosen, one of two scenarios will be executed:

For boolean values, only the first connection seen by the parser (lowest line number) for each threadpool will actually set the option (or leave it at default values). Subsequent connections that use the same threadpool will ignore the setting.

For integer values, a one-way 'ratchet' is implemented. Buffer-line and thread counts may only be increased. Timeout values (including poll-intervals) may only decrease. The reasoning behind this is that a particular source may have minimum resource needs, so it will 'bump' the parameters of whichever threadpool that connection is assigned to in order to increase resources to 'minimum acceptable' values.

Threadpool-type-specific options are covered under the individual sections devoted to each threadpool type.

Threadpool Selection and Configuration

Input sources in Syslogd2 may be grouped (subject to compatbilities below) into threadpools with each different group (of compatible connections) 'serviced' by a unique threadpool resource set of (1) code-algorithm and (2) multiple, shared execution-threads. Given the variety of inputs supported, there may be up to one (1) kernel and one (1) journal threadpool, but multiple of each of socket/pipe and text-file threadpools -- all operating simultaneouly with different thread-counts, (and if enabled) different worker-type threadpools each with different buffer-sizes, worker-threads and queueing characteristics.

Individual connections specify the threadpool they will become a component (client) of using the 'threadpoolid = n' or 'id = n' option. The default value of 'n' is always 0, regardless of threadpool type. Unless otherwise stated, this puts all (compatible) inputs into a minimum number of threadpools where they share threads and other resources.

The allowed values for 'ThreadpoolId' ('id') are the complete range of non-negative integers. Each of the following threadpool-types has its own separate series of 'id' numbers: socket/pipe/tailfile, kernel, journal, user, worker, output and housekeeping. Of these, kernel, journal, housekeeping and user threadpool types are all single-instance, one-off threadpools with their threadpool-id ignored (set to zero). The socket/pipe/tailfile, worker and output threadpool types may each have a many threadpool instances as desired (and supported by the underlying hardware). The restrictions mentioned above concerning the 'grouping' of connection-types into threadpools are:

The 'Kernel' and 'Journal' type threadpools are unique and 'one-off'. There will never be more than one of these threadpools per executing instance of Syslogd2 (since there is only one kernel or journal to read from).
Socket and name-pipe inputs may be freely mixed together in the same threadpools since they share the same read-algorithm (the code that each input-threadpool executes). If a threadpool consists of a single named-pipe connection, it will be automatically 'elevated' to a 'dedicated' (high-speed) algorithm as will any threadpool containing only a single datagram (udp or Linux datagram) socket connection.
Text-file threadpools are not allowed to share threadpool-ids with sockets/pipes because they use a different code-algorithm which is fundamentally incompatible with sockets/named-pipes. However, as a (less desirable) alternative to creating a kernel threadpool, a text-file threadpool CAN be used to read the /proc/kmsg file. The use of the actual, purpose-designed kernel-threadpool type is recommended for reading kernel data, however. There is no 'high-speed' threadpool option for text-file input.

Syslogd2 performs some sleight-of-hand during startup when it encounters a text-file (tailfile) 'connection' specifying 'threadpool-id 0 [zero]'. It silently changes the threadpool-id number to an (otherwise illegal) value of -2 in order to de-conflict the setting with sockest/named-pipes that also default to threadpool-id 0.

The full set of (optional) threadpool-selection/configuration options available to all inputs (not counting threadpool-specific options) at the connection level is listed below. All threadpoolid values default to zero as do all queue values. Unless CAP_WORKERTHREADS was declared at compile-time, the 'workers', 'lines' and 'queue' options will be non-valid.

ThreadpoolId (id): [default: 0 [zero]]: Specifies the threadpool-id to assign this input connection to. If the threadpool does not exist, it will be created using global-default parameters that can be over-ridden by the parameters contained in this connection's option-list.
Readers (r): [default: inherited]: Specifies the minimum number of threads in the input threadpool. If the previous value was less than this value, it is raised to this value.
Workers (w): [default: inherited]: Specifies the minimum number of threads in the worker-threadpool specified by the 'Queue' option. If the previous value was less than this value, it is raised to this value. THIS OPTION REQUIRES THAT CAP_WORKERTHREADS WAS DEFINED AT COMPILE-TIME.
Lines (l[ell]): [default: inherited]: Specifies the minimum number of buffer-lines (messages) that can be stored in the input FIFO queue of the worker-threadpool specified by the 'Queue' option. If the previous value was less than this value, it is raised to this value. THIS OPTION REQUIRES THAT CAP_WORKERTHREADS WAS DEFINED AT COMPILE-TIME.
Queue (q): [default: 0 [zero]]: Specifies the worker-threadpool that will process the messages read by this input threadpool. THIS OPTION REQUIRES THAT CAP_WORKERTHREADS WAS DEFINED AT COMPILE-TIME.

--<keyword> = ... , id=1, readers=3, workers=4, lines = 50, queue=1, ... ## with CAP_WORKERTHREADS defined
--<keyword> = ... , id=0, readers=3, ... ## without CAP_WORKERTHREADS defined

Facility, Priority, and Hostname Options

These options are inter-related and important enough to deserve their own grouping. These are 'base' options (meaning there are no compile-time requirements to use them). These options work by over-riding the inherited values set at the global level (or compiled into the code). Any input string received that does not contain an imbedded 'facility/priority' (facpri) field, will use these faciltiy/priority values if set. If not set, the incoming message will be assigned the global value for kernelfacility (or kern.notice if not set) for messages from the kernel or the global value userfacility (or user.notice if not set).

The full list of options in this grouping are:

Facility (Fac): [default: inherited] This value can be set as a single string ('user') or a combined string ('user.notice'). This value will be used ONLY if there is no 'facpri' field detected in the raw data string. If the combined form of the parameter is used, it sets both the facility and the priority at the same time.

Priority (Pri): [default: inherited] This value can be set as a single string ('notice') or a combined string ('user.notice'). This value will be used ONLY if there is no 'facpri' field detected in the raw data string. If the combined form of the parameter is used, it sets both the facility and the priority at the same time.

Hostname (Host): [default: inherited] This value will over-ride the global values for 'HostName' and 'DomainName' (which are automatically initialized via 'gethostname' system-call at startup. Any message that is determined not to have a hostname will use this value (if set) as it's hostname-field.

Ignore: [default: <empty string>]: The parameter to this keyword is a semi-colon-separated list of keywords from the set: [facility, priority, hostname]. Syslogd2 interprets each element in the keyword-list as follows:
- If the corresponding field in the raw data string is missing, it the 'default' value (either from the option-list or the global settings or the compile-time values) is used.
- If the corresponding field in the raw data string is present, it is parsed, then ignored and the default value (as above) is used instead.
This has the effect of 'over-riding' the current contents of the incoming data field as if the 'modified' ('default') content had been included in the original message.

NoHeader (NH): [default: off (auto-detected)] This boolean value acts as a hint (instruction) to Syslogd2 that there is no 'header' (consisting of a time-field + a host-field) in the content from this host. This can save CPU cycles on high-volume hosts because Syslogd2 will not attempt to parse a time-field. (According to the syslog protocol specifications, if a time-field is found, the hostname is 'blindly' accepted as the first character-string following the time-field. No time-field, => no host field). The use of NoHeader is common with network-devices that do not include time or host-fields in their raw syslog strings and with text files.

NoHost: [default: off] This boolean value acts as a hint (instruction) to Syslogd2 that the input format from this source contains a valid (recognizable) time- field, but no host field. (Though this does not following syslog protocol guidance, the format of text files is never required to be in strict syslogg format. It also turns out that systemd messages forwarded via their syslog socket contains a readable time-field, but no host field, so you will find this option in common usage with Syslogd2 and systemd.

$h and $d: [default: unused] These two variables may be used in conjunction with either the hostname option in the option-list or with the primary parameter when specifying IP sockets. The $h string is replaced with the system's default hostname and $d is replaced with the system'd default domainname. These strings will be the initialized values from gethostname or the global variable parameter values if modified by the system administrator.
Example: a host named 'sallyhost.domain.com' defines a text-file input with the option 'hostname=logfile.$h.$d'. The resultingg hostname field in the output will be 'logfile.sallyhost.domain.com'. To force this hostname to be used for all inputs from that source (whether the original string contains a host field or not), add 'ignore = hostname'.

~ --<keyword> = ... , facility = extra0, priority = notice, hostname = callcenter.$h.$d, \
ignore = facility; priority; hostname, noheader, ...

Other Generic Input Options

The last 'grouping' of input-options is the 'other' group. This is comprised of miscellaneous options that don't fit any of the other categories.

DNS: [default: Inherited] This boolean parameter allows the admin to over-ride the global setting of the same name for this connection only.

Filter: [default: <empty string>] The parameter to this option is a filepath (either absolute or relative). If a relative filepath, it is relative to ConfigDir (the default input-files directory for Syslogd2 (/etc/syslog.d).

NameCache (Cache): [default: Inherited] This boolean parameter allows the admin to over-ride the global setting of the same name for this connection only.

Network (Net): [default: Any] The parameter to this option is the keyword 'Any' or a semi-colon-separated list of keywords from the set: [Down, local, Other]. This parameter specifies the network states in which this connection will be valid. Syslogd2 knows that when the network state is 'Down', (regardless of this setting), no IP connections can be established OR resolved. When the network state is 'Local', only those connections on the loopback interface will be accepted or initialized. As implied at the link, this option is a work-in-progress, but is already paying dividends. Most people will not need to use this option for input. It MAY have uses for controlling output spooling, however (especially for collecting and spooling data while the network is down to be forwarded to a remote host once the network is up and available.

NoForcePrintable (NFP): [default: Inherited] This boolean parameter allows the admin to over-ride the global setting of the same name for this connection only.

Configuring Socket Input

Socket input in Syslogd2 is the only input-type in the 'base' feature-set' (meaning it requires no compile-time options).

By default, Syslogd2 will open one Linux datagram socket at /dev/log (or the location of system header files if ported to a non-linux machine). The user may over-ride the location of this default socket via global variables (--defaults logsocket=) at run-time. The user may also disable the creation of this default input socket (--disable syslog) at runtime. The default definition of this socket may also be over-ridden simply by re-declaring it (the default declaration carries a line-number higher than the number of lines in the file, so it will always lose a first-come-first-serve conflict).

If the IP service is enabled (--enable inet or -i or -r), and at least one address-family is available to use, Syslogd2 will also open UDP/IP port 514 (defined syslog) on the address '*' (all interfaces). If -f (--enable forwarding) is also present, Syslogd2 will allow traffic received from an IP interface to be forwarded to a remote host over IP as well. This action mirrors that of traditional syslog processors (even to using the -i and -f or -r command-line parameter to enable IP services). If the IP service is enabled, the default IP port can still be disabled using --disable defaultip. Like the default Linux socket above, the default IP input carries a line-number higher than the number of lines in the file, so it too, will always lose a first-come-first-serve conflict.

The keywords used to specify socket input(s) are --socket= and --input=. These keywords are aliases of each other. The primary parameter to these keywords is either an absolute pathname (to define a Linux socket) or an IP address or IP hostname (to define an IP socket). The primary parameter format (does it start with '/' ?) determines Linux vs IP.

Identity options are used to specify the protocol (and for IP the port to connect to). (Reminder: all keywords are non-case-sensitive)

Datagram (dgram, UDP, U, d: [default: on] This boolean option positively selects the default protocol of 'datagram' for this input connection.

Stream (TCP, T, s: [default: off] This boolean option selects the alternate socket protocol of 'stream' for this input connection. Stream/TCP support requires that CAP_STREAMIN be included in the configuration at run-time. Datagram and Stream are mutually exclusive options.

Port (p): [default: 514 for UDP, undefined for TCP] This positive integer value specifies the IP port to listen on for this connection.

Version (ver): [default: 46] This keyword takes a parameter from the set: [4, 6, 46]. This option specifies the address-family (ies) that this connection should listen on. If an IP-address is given, only the address-family matching that address format will be used and the version setting will be internally adjusted to match. If a hostname is given, the version setting will specify which socket-family (IPv4 or IPv6) will be opened. Setting version=46 (the default) with an IP hostname may result in multiple IP sockets being opened if that hostname resolves to both an IPv4 and IPv6 address. No more than one listening socket per address-family will be opened and the defined 'canonical address' (whether from cache or DNS) will be used for that hostname.

There are no threadpool-specific keywords used with socket input threadpools.

Connection-specific keywords that affect Linux sockets (and also apply to Name-Pipes) define the file-system parameters of the device that is created when the listening connections are opened. These parameters are valid on Linux sockets and Named pipes only:

Uid (u): [default: 0] Parameter is a username (string) or a userid (integer) that should be set in the filesystem as the owner of this socket or pipe device.
Gid (g): [default: 0] Parameter is a groupname (string) or a groupid (integer) that should be set in the filesystem as the group-owner of this socket or pipe device.
Mode (m): [default: 666] Parameter is a filesystem permission value (octal integer) that should be set in the filesystem as the permission mask for this socket or pipe device.

Global keywords that affect socket operations include:

--systemd (--enable systemd): [default: disabled] Tells Syslogd2 to adjust the default Linux socket definition from '/dev/log' to '/run/systemd/journal/syslog, nohost' to accomodate the requirements of systemd.
--Remote (--enable remote, -r): [default: disabled] Enables both inet (ip services) and forwarding (...of traffic received over IP to a remote IP host).
--ApplicationMode (--enable ApplicationMode): [default: disabled] A macro that 'reconfigures' Syslogd2 to run as an application (versus running as the primary system logging service). The purpose of this macro is to help insure that all innate conflicts with an external syslog logging service are resolved by disabling the potentially conflicting interfaes in Syslogd2. It is equivalent to:

~ --disable UserLogging, UserThreads, KernelLogging, KernelThreads, Syslog, DefaultIP, Console, Tty
~ --enable AllMessagges, NameCache, HouseKeeping, Inet, Forwarding

Examples of socket input definitions for:

A hypothetical network input (via IP) (Assumes that CAP_WORKERTHREADS is enabled).
This input definition also forces all incoming switch/router traffic to the unused facility 'extra2' to avoid conflict with standard syslog facilities.
It also uses the high-speed algorithm by creating (specifying) threadpool number 1 (one) that is comprised of only one datagram socket.

~ --enable inet, forwarding
~ --input 192.168.2.40, noforceprintable, noheader, id=1, readers=5, workers=15, lines=100, \
queue=1, filter=filters/switchrouter.input.filter, facility=extra2, ignore=facility

A Syslogd2 command input (via streaming Linux socket)

~ --input /dev/logstream, stream, command, mode=600 ## only root should now be able to use this socket.

Adding a filter to the default Linux socket definition on a systemd-based Linux host.

~ --systemd ## Tell Syslogd2 that systemd is in use and to move the 'default' Linux input port accordingly
~ --disable syslog ## disable the default syslog port. (makes the above line redundent)
~ --input /run/systemd/journal/syslog, nohost, filter=filters/defLinux.input.filter ## new definition (with filter)

Configuring Systemd Journal Input

[Top of page]

The --Journal Command
The --JournalField Command
Other Recommended Settings

The --Journal Command

The keyword to confiure Syslogd2 to obtain input from the Linux systemd journal subsystem is '--journal'. There should only ever be one instance of the '--journal' keyword in any given configuration (there is only one threadpool to confiure and only one journal to read from).

The keyword '--journalfield' is used to configure individual journal fields.

The systemd journal is best thought of as a database that Syslogd2 can sequentially traverse (query) using a systemd-provided Application Programming Interface (API). With each step a series of data-fields is made available to Syslogd2 who then combines the available data into a raw syslog string that is passed to the processing phase as if it had been read from a file, socket, kernel or pipe.

It must be stressed that the systemd journal is NOT syslog (nor is it even based on syslog). A fairly detailed discussion of configuring Syslogd2 to 'read' from a systemd journal can be found here. There is little sense in copying the detailed information from the link to here, so I'll just briefly review the '--journal' and '--journalfield' command options for the sake of completeness of this page.
The --journal keyword requires that the CAP_JOURNALTHREADS compiler-symbol was included at compile-time.

There should never be more than one instance of the '--journal' keyword in any configuration.

There is one (1) global configuration setting that affects sysemd journal input operation:

JournalThreads (Journal): [disabled] This global boolean value must --enable the journal threadpool code before it will be active for use (--enable journal).

There are no threadpool-specific configuration parameters for the journal threadpool.

There are several connection-specific configuration parameters for the journal threadpool:

Transport (Soruce, Src): [defualt: 'All'] This keyword specifies the data to include based on the sources (transport) from which the journal receieved the data. This keyword takes either the keyword 'All' or a semi-colon-separated list of keywords from the set:
- audit: The data came from the kernel's audit subsystem.
- driver: The data came from internally-driven (internal to systemd) sources.
- syslog: The data came from the system-default syslog socket at '/var/log' using the syslog protocol.
- journal: The data came was received by the journal via the systemd's native 'journal' protocol.
- stdout: The data was read from a system-service's stdout or stderr data streams.
- kernel: The data was read from the local system kernel.
Each keyword in the list above may be preceded by either '!' [exclamation point] or '~' [tilde] to indicate 'not'.
~ --journal = src = audit;!driver;syslog;journal;~stdout ## results in 'audit', 'syslog' and 'journal' only.

Append: [default: disabled] This boolean value tells Syslogd2 to start reading from the end of the journal instead of from the beginning. Since the journal on my Ubuntu system goes back about a month, starting from the beginning is a significant amount of data (much of which is likely duplicative).

DefaultMaxLength (MaxLength, Length): [default: 80] This non-negative integer sets a default limit on the length of any journal field inserted into the syslog message. Some of the journal fields can get very long if not length-limited.

IncludeAllFields (AllFields): [default: disabled] This boolean value sets the default policy toward including additioanl journal fields as name=value parameters in the 'sdstring' component of the syslog message. Enable this value to add all available fields by default. (Individual fields can be disabled using --journalfield later). In the default case (disabled), no addiational data-fields are inserted into the raw syslog data-string prior to submission to the processing phase.

NoSDLength: [default: disabled] This boolean value (when enabled) disables the creation of an extra field added by Syslogd2 providing the overall length of the sd-string portion of the message. When present, this value (named SDStringLength) includes the length of the 'SDStringLength field itself. This field is created as an aid to buffer-management for those that wish to pass journal fields through syslog to downstream processes (such as DBD2 for management applications.

NoTagField: [default: disabled] This boolean value (when enabled) disables the creation of a 'tag field' (comprised of the process's application-identifier and pid (Example: 'kernel [12345]') followed by ": " that precedes the free-form portion of the msg string). Not all the information from the journal originates from syslog. Therefore, Syslogd2 chooses to use syslog-generated data where available and journal-generated data when no syslog-generated fields are prsent. The following options (as well as the timestamp field discussed later) allow the user to modify this behavior.
- TagName (AppName): [default: 's'] This boolean value (with allowed values of 's' and 'j') selects the field Syslogd2 will use to represent the application name in the 'tag field' (if it is built). By default, (value of 's') the field 'SYSLOG_IDENTIFIER' is used (this is the name the journal stripped out of the original syslog message it received). Setting this value to 'j' causes Syslogd2 to use the '_COMM' field (the name of the program that produced the message).
  
  These two fields may vary for various reasons. For example, a shell script launched by 'crontab' may have a 'SYSLOG_IDENTIFIER' value of 'CRON', but a '_COMM value of 'sh'.
- TagPid: [default: 's'] This boolean value (with allowed values of 's' and 'j') selects the field Syslogd2. will use to represent the application PID in the 'tag field' (if it is built). This option only applies to msgs with transport of 'syslog'. The 'SYSLOG_*' fields are not present for other transports. By default, (value of 's') the field 'SYSLOG_PID' is used (this is the field the journal stripped out of the original syslog message it received). Setting this value to 'j' causes Syslogd2 to use the '_PID' field (the Process ID of the program that produced the message). In theh above example, the 'SYSLOG_PID' field would refer to the pid of the cron process. The '_PID' field would refer to the pid of the (child) sh process.

TimeStamp: [default: 's'] This boolean value (with allowed values of 's' and 'j') selects the field Syslogd2. will use to represent the timestamp of the message in the raw data-string sent to the processing phase of Syslogd2. The 'SYSLOG_TIMESTAMP' field ('s') is preferred if available. Otherwise the '_SOURCE_REALTIME_TIMESTAMP' field will be used (and adjusted from micro-seconds to seconds).

The systemd journal tracks approximately 35 different values for most events. Of these around 6-10 are of little to no practical use to netowrk managers and another 4-5 are already provided as syslog msg components (though perhaps not always the preferred fields). This leaves around 25 fields that are worth 'exploring' for insertion into syslog events.

The --JournalField Command

The --journalfield command is secondary to (and supports) the --journal command. Multiple instances of the --journalfield command may be used.

Each instance of the --journalfield command configures one journal field. The order in which the --journalfield are encountered in the configuration file dictates the order in whichh the fields are defined in the sd-string portion of journal-originated messages. If the --journal's AllFields value is not set, only the fields specified by the --journalfield command will be included in the message. if AllFields is set, then (after all --journalfield entries are resolved), all remaining fields will be added to the sd-string in the order they were read from the journal interface.

The format of the --journalfield command is:
~ --journalfield = <journal-field-name> [ = <user-preferred-name>], <option-list>

The options to the --journalfield command are:

Disable: This boolean value will suppress the inclusion of the specified field in the syslog msg being created.

MaxLength (Length) : [default: inherited] This non-negative integer allows setting custom max-length for this specific field. It inherits its default value from the --journal's MaxLength value.

The ability to rename the default journal-defined fields is intended to facilitate database insertion and manipulation operations for downstream applications (such as DBD2.

Other Recommended Settings

Development of the Journal input threadpool subsystem highlighted some additional topics that deserve mention here so that precautionary action can be taken.

File Handles: Setting a fairly high numer of concurrent threads for the journal threadpool (in addition to some 35+ output files and other test input/output connections) on my local Ubuntu system caused me to slam past the 1024 files-per-process limit set in Ubuntu. This forced me to revise some elements of Syslogd2 to get around this limit. Still, if you are going to use more than about 4 concurrent journal threads, or if you will be opening lots of threads or files, you should consider raising the max file-handles-per-process limit to 2048 or even higher:
root# ulimit -n
root# ulimit -n 2048
root# ulimit -n

The above settings will be temporary and limited to the shell in which the command is issued, so to make it permanent, the applicable command must be added to whatever script will be used to start Syslogd2.
CAP_VARBUFFERS: Due to the string-lengths approaching 800 bytes of my test configuration with journal threadpool, I immediately started blowing up Syslogd2 due to over-length buffers. This cause me to move the introduction of 'CAP_VARBUFFERS' forward and to add some mitigating code'.

When CAP_JOURNALTHREADS is defined, Syslogd2 now calculates an 'estimate' of the required sd-string size and quietly sets that value as a 'floor' for the MaxSDStringLength global variable that user estimates can still over-ride if provided. This approach seems to work for long data-strings produced internally by the systemd journal API interface, but will NOT work if users produce their own long SD-strings and fail to appropriately estimate any required buffer-expansions needed.

Queue Settings: It makes little sense to be able to produce input quickly if a high percentage of that input will be discarded because a hand-off buffer fills up due to processing threads being unable to handle the presented traffic level. Syslogd2 addressed this 'queue quandry' by adding the 'LossLess' parameter to all threadpool types that use input-FIFO queues. This setting, combined with the overall structure of Syslogd2 allows input connections that can control their own data-production rates to safely use 'lossless' queues because the only impact will be that their threads will get 'backed-up' whenever they exceed the processing rate of the threads that are trying to empty the queue(s).

Input threadpool types that are the best candidates for 'lossless' queueing are journalthreads and tailfiles. To a lesser extent, kernel-threadpools may also be able to use 'lossless' queues because kernel activity levels are usually quite low.

To the current performance level of your syslog processing environment, I've prepared a demonstration using Syslogd2 that will allow you to experiment with different queue and thread settings to start 'taming' your 'syslog beast'.

Configuring Kernel Input

The keyword to configure kernel input in Syslogd2 is '--kernel'.

If the CAP_KERNELTHREADS option is defined at compile-time, a kernel threadpool will automatically be defined and used. A minimal kernel input definition is automatically provided (unless either kernelthreads or kernellogging is disabled) using a line-number higher than the last line in the configuration file (as in the default syslog and IP socket port definitions).

Only one instance of the --kernel keyword is allowed.

There are no identity options for the '--kernel' keyword.

All incoming data processed by this threadpool is automatically designated as coming from the kernel. This means it will use default facility/priority values from kernelfaciltiy instead of from userfacility

There are two threadpool-specific options with this keyword:

Poll (Polling): [default: off] This boolean value specifies whether the special kernel code should use system-calls (off) or poll the /proc/kmsg file (on).

PollInterval (PollTime): [default: inherited] This positive integer value specifies how long Syslogd2 should wait for more kernel events to arrive after it has reached the 'end' of the /proc/kmsg pseudo-file.

Global parameters affecting Kernel logging:

KernelLogging (kernellog, klog): [default: enabled] This global boolean controls all kernel-logging done by Syslogd2 from any source.

KernelThreads: [default: enabled] This global boolean value allows the kernel input threadpool to be disabled while leaving kernel logging enabled. If you want to use --tailfiles to read kernel input instead of using the kernel threadpool, disable this option, leave kernellogging enabled, and configure a '--tailfile' input with primary value of /proc/kmsg and the additional option 'kernel' to tell Syslogd2 that the data from that 'file' is kernel data.

Example: (redundantly) enables kernel log input and uses the kernel threadpool, specifies to use system calls and defines an input-filter and at least 4 read-threads.

~ --enable kernellogging = yes, kernelthreads = yes ~ --kernel filter = filters/kernel.input.filter, poll = no, r=4

Configuring Named-Pipe Input

The keyword used to configure named-pipe input in Syslogd2 is --pipe.

This input method requires that the CAP_PIPESIN symbol was declared at compile-time for this binary image. Named-pipe input does not have to be enabled. There is no default value or automatic configuration element.

Multiple instances of the --pipe option are allowed.

Named pipes may mix with sockets in the same threadpool.

There are no identity options for named-pipes. (Their unique pathnames identify them).

There are no threadpool-specific options for named-pipes

The only global parameter that affects named-pipe input is the global 'PollInterval' setting that is shared with the kernel threadpool and the tailfile threadpools.

There is one connection-specific option for named-pipes:

PollInterval (PollTime): [default: inherited] This positive integer specifies the time (in seconds) Syslogd2 should delay after a named-pipe is closed before that named-pipe is re-checked to see if it can be re-opened. For named-pipes that have not yet been opened for traffic, this parameter does not (yet) apply.

An example of a named-pipe input command. This command forces all incoming traffic to 'extra4' (while keeping pre-existing priority values) in order to 'dodge' other traffic.

~ --pipe /dev/pipe2syslog, polltime=60, id=3, filter=filters/pipe2syslog.input.filter, facility=extra4, ignore=facility

Configuring Text-File Input