Conceptual Overview of the Configuration File
[Top of page]
A Syslogd2 cofiguration file starts with a standard, traditional syslog configuration file, then adds additional options and 'configuration commands'.
We will do the same with this overview.
Conceptually, all syslog configuration files consist of an (empty) input-definition section, an (empty) set of include-files,
an (empty) general-processing-configuration section, and a (non-empty) output-section.
In practice (with Syslogd2), elements of these conceptual sections may appear in any order (including intermixed) in the configuration file.
The above description of a legacy configuration file also describes the Syslogd2 minimum configuration file. In fact, a traditional (legacy) syslog-configuration file is also
an acceptable Syslogd2 configuration file if you are just doing local-host-logging:
*.* - /var/log/syslog.log
kern.* - /var/log/kernel.log
...
What makes Syslogd2 so different is that so little syntax structure had to be added to support Syslogd2's many features.
A second major (conceptual) change is that Syslogd2 shifts the configuration focus from global paraneter settings to individualized connection settings, providing much more
control over individual input sources (as well as more granular control over what data gets sent to output destinations).
Conceptually, the additional syntax elements and concepts of Syslogd2 are:
- Command-line entries. 'Moving' command-line elements into the configuration file requires that:
- Each logical line of the configuraton file that will contain command-line parameters must start with the 'command-line recognition character': a tilde ('~').
- All quotes and escape characters must be removed from the command-line parameter string. They are not needed since the shell will not be parsing these strings.
Note: All input-specifications (commands that define Syslogd2 inputs) and other non-output configurations are structured as command-line entires.
- An optional (possibly empty) option-list has been added to the end of each line of the traditional configuration file.
- 'Extra' facilities. The range of accessible facilities is now configurable, extending from 'local7' to 'extra0' then extending through 'extra31'
(by default) up to a maximum of 'extra999'.
- In-line comments. These are comments that begin on the same line as (but following) a line of configuration.
- A 'concatenation character'. Two or more physical lines may be concatenated into a single logical line by adding a backslash ('\') as the last non-printing character of a line.
Any whitespace at the start of the next line will be ignored, so feel free to indent concatenated lines for clarity and redability.
The rules concerning the syntax of the configuration file have been somewhat relaxed:
- All keywords (command-names, option-names, keyword values, etc) in Syslogd2 are non-case-sensitive.
The sole exception to this is single-letter command-abbreviations which (due to limited supply) may use upper- and lower-case to indicate thir assignment
to different commands.
- Whitespace is ignored (espcially around equal-signs and list-delimiters). All values are both left- and right-trimmed of white-space before use.
- The hashtag remains the comment-character, but now starts a 'soft comment'.
Any logical line that does not start with an appropriate 'line-recogntion token' is assumed to be a 'soft comment' and ignored. Line recognition tokens
are (1) a tilde or (2) any valid syslog-facility name or a facility-name-wild-card ('*' and 'none').
- Syslogd2 implements a strict 'first-come-first-served policy when parsing the configuration file. The first instance of a setting that is parsed is kept, all
subsequent settings for that same variable instance are ignored (usually with an appropriate error-file notification). Several commands may be issued multiple times as long as they
work with different settings.
Syntax Additions for Input and General-Processing Sections
Additional syntax nuances apply to the input-definition and general-configuration elements contained within the file:
- All input and general-processing commands in Syslogd2 are command-line parameters.
Though you CAN configure (virtually) all features of Syslogd2 using nothing but the command-line, attempting to do so will likely exceed the maximum
length of the Linux command-line buffer.
It is for this reason that Syslogd2 allows you to move command-line options into the configuration file as long as the following conditions are met:
- Every logical (input- or general-processing- ) line of the configuration file must begin with a tilde ('~').
The tilde signals to the Syslogd2 parser that such lines contain command-line input.
- All quotes and escape-characters must be stripped out of the command. Syslogd2's parser may not properly read lines with escape-characters or quotes.
- Most of the time, the equal-sign after the keyword can be omitted if whitespace is present.
The sole exception to this is the first comma-separated option following a comma-separated list of users in certain output-lines.
(The euqal-sign is used to tell the Syslogd2 parser where that the user-list has ended and the option-list has begun.)
- Soft-comments are always 'on' for all command-line-formatted options. This means that a single hashtag may precede the tilde line-recognition-character.
- NOTE: A few commands MUST remain on the command-line: the '--configfile=' ['-c'] option (specifying an alternate configuration file), and all options that
exit as soon as they display their information ('--help' ['-h']['-?']['-u'], '--version' ['-v'], --TestConfig ['-T'])
# ~ --socket = /dev/logstream, stream, command # This is an in-line comment.
Contents of the Configuration File
[Top of page]
A Syslogd2 configuraiton file (no matter how complex) will consist only of:
- Output Lines consisting of traditionally-formatted entries that are optionally 'enhanced' by an appended option-list and extended facilities.
- Command-line options
- Comments
The specific compile-time configuration of any Syslogd2 binary can be determined with either the --help or the --version commands. Either one will provide
a list of all supported capabilities (in the form of 'CAP_*' declarations and their Enabled/Disabled status) as well as the compiled-in number of 'extra'
facilities provided by that binary image.
Command-liune options can generally be divided into the following three categories.
- Input Specifications: These commands define various inputs to Syslogd2.
Their option-lists specify the any special processing requests the administrator wants the 'processing phase' to accomodate.
Such requests and options are limited in scope to the single input for which they are defined.
- Parsing Control: A handful of command-line options allow some control over parsing of the main configuration files, declaration of 'include-files'
and definitions of an error-file and a log-file.
- Global and Threadpool Parameter Management: The single 'threapool management' command of this group allows creation and definition of any number of threadpool types
supported by a given binary image. Any threadpools that are defined, but not used will be deleted prior to run-time.
Other than Parsing Control parameters (which are all base-level [built-in] command-line parameters), the numbers of command-line options and number of sub-options
in each category will vary based on the compile-time feature-set selected for each compiled binary variant.
The option-list that has been added to both input and output connection specifications consists of:
- Identity Options: Options that identify specifics of the connection itself (port, protocol, filename, etc), separating one instance of this connection-type
from another.
- Threadpool-Specific Options: Options that are unique to a particular type of threadpool and that apply to the overall threadpool rather than to a specific
connection within the threadpool.
These options affect the way the threadpool processes data and are usually provided either in the '--ThreadMaps' option or in individual connection option-lists.
One example would be whether to poll the /proc/kmsg pseudo-file for kernel input or use a system-call to directly access the kernel's ring-buffer (completely
bypassing the /proc filesystem for syslog input).
- Threadpool Selection and Confiuration Options: These are a set of common, general-purpose options available to all input and all output commands, regardless of
threadpool-type.
- Other Generic Options: These options generally provide configuration for compile-time features, but can include choosing to enable/disable the internal cache/DNS
support for individual connections, the specification of a filter to apply, a default facility/priority/hostname to use, whether to 'force' the default, or other settings
such as connection-specific spooling parameters or output formats.
In more advanced configurations, the configuration file will likely contain parameters for various threadpools and (optional) feature-sets allocating system resources (thread-counts,
buffer-space, etc) or specifying operational parameters, default values and/or filters to be applied to messages being processed, cache-file / DNS usage, etc).
The configuration file will also contain information on the number and content of each threadpool used (among other settings).
The processing of each individual syslog message passes through three (3) distinct 'phases' of processing. This is true for all syslog processors, Syslogd2 just makes this process
explicit.
These 'phases' may be coupled loosely (via threadpool hand-off using FIFO buffers) or tightly (via direct subroutine calls).
Regardless of the hand-off method between processing 'phases', the three phases remain pretty much the same: (1) data-input (obtain the raw data
string and 'message envelope' data), (2) generic processing (parse and process the raw data string using the 'message envelope' data as required),
(3) data-output (send appropriate data to specified destiunations).
The processing phase is almost completely dependent on the options selected for the individual connections whose data is it processing.
Therefore almost all options for the processing 'phase' are actually configured in the various 'input-connection option-lists'.
[Top of page]