Syslogd2 References & Tables
(Reference Resource For All Keywords and Options)
This document focuses on the 'core' command-line options and parameters supported by Syslogd2.
Command-line keywords or options that are dependent on compile-time
features will be presented more fully in the context of the page devoted to that feature.
All keywords in Syslogd2
(both commands and options) are case-nonsensitive. The only
case-sensitive options are the single-letter ones displayed with a
single hyphen. (This is because I sometimes use the upper-case letter
for a different option).
Please Note: These are just the command-line options and settings
that are available in ALL compiled instances of Syslogd2.
Each additional feature that is enabled at compile-time may add
entirely new command-line options or may enable additional options in
existing commands.
Syslogd2 embraces the concept of inheritance. If we (arbitrarily) assign the compiled-in values (the least binding) as 'priority 1', then we can say that at every level, each value set will have a higher priority than previous levels, over-riding all previous value settings. Likewise, each value NOT set, will inherit the (highest-priority) value previously set for it. The following chart attempts to depict this concept...
| Priority | Description | Comments |
|---|---|---|
| 1 | Compiled-in Values |
|
| 2 |
--defaults command-line options. |
This is the only level that directly replaces compiled-in values. It can raise or lower values that are restricted in higher priorities. |
| 3 | Other global settings (--enable/--disable, --threadmaps, etcr) |
Generally time-intervals can only shortened. Thread-counts and buffer lines can only be increased from previous settings. |
| 4 | Options expressed in input keyword option-lists. |
|
Facility-names in parentheses ['( )'] are not defined in the Linux header file. These facilities are only available because of Syslogd2 extensions.
|
Numeric Value |
Facility Name (alternate) |
Facility Comments |
Priority Name (deprecated name) |
Priority Comments |
|---|---|---|---|---|
| 0 | kern | emerg (panic) | ||
| 1 | user | alert | ||
| 2 | crit | |||
| 3 | daemon | err (error) | ||
| 4 | auth (security) | warning (warn) | ||
| 5 | syslog | notice | ||
| 6 | lpr | Line-Printer issues (out-of-paper, jammed, etc) => now mostly obsolete | info | |
| 7 | news | Mostly obsolete.... | debug | |
| 8 | uucp | Modem Connection issues (later ADSL): => now mostly obsolete due to broadband. | ||
| 9 | cron | |||
| 10 | authpriv | |||
| 11 | ftp | |||
| 12 | (Reserved0) | Not assigned or supported by Linux system header files - supported via Syslogd2 extension. | ||
| 13 | (Reserved1) | Not assigned or supported by Linux system header files - supported via Syslogd2 extension. | ||
| 14 | (Reserved2) | Not assigned or supported by Linux system header files - supported via Syslogd2 extension. | ||
| 15 | (Reserved3) | Not assigned or supported by Linux system header files - supported via Syslogd2 extension. | ||
| 16 | local0 | |||
| 17 | local1 | |||
| 18 | local2 | |||
| 19 | local3 | |||
| 20 | local4 | |||
| 21 | local5 | |||
| 22 | local6 | |||
| 23 | local7 | |||
| 24 | (extra0) |
All 'extra' or 'reserved' facilities are ones
that Syslogd2 makes available to syslog managers as
'working spaces'. No known Linux applications use these entries.
It is believed that no (other) Linux syslog processor supports
them. Non-Linux operating systems may assign additional
facilities for their own internal use. Known instances of this
are listed after this table. Syslogd2 defines 32 'extra' facilities by default. This number can be increased up to 1000 at compile-time. |
||
| ... | ||||
| 55 | (extra31) | |||
| ... | ||||
| 1023 | (extra999) | |||
Note 1: Exceptions or facility conflicts to the above table:
Note 2: The universal convention when writing syslog configuration files is that the expression facility.priority means “all priorities within the given facility that are numerically less-than or equal to the named priority”. With Syslogd2 extensions, this can be explicitly stated as “facility.<=priority”. Additional nuance is possible in Syslogd2 using any combination of '<', '=', '>', '!' and '~' as explained elsewhere in this document. The '<', '=' and '>' comparisons refer to the numeric priority values.
In the table(s) below, the core command-line options have been grouped by function and the cell backgrounds have been color-coded to indicate functional groupings. Each command in the table is hyperlinked to the section that discusses it.
Command-line keywords grouped by function:
|
Keyword (alias, alias) |
Mandatory Parameter (default value) |
Optional Parameters (default values) |
Comments |
|---|---|---|---|
| Command-Line Options Not Supported Inside the Configuration File | |||
| --ConfigFile (-c) | Absolute FilePath | <none> |
Path to alternate configuration file. Default: /etc/syslog.conf |
|
--help (-h) (--usage, -?) |
Prints a help screen and exits immediately. | <none> | |
| --TestConfig (-T) | <none> | comma-separated list of display conditions and output location. | --TestConfig displays the current parsed, built or staged configuration in a structured, easy-to-understand format to one of a set of locations, then exits immediately. (default output is to the local terminal). It requires CAP_WHATIF and is an optional feature-set. Please refer to separate documentation on the CAP_WHATIF feature-set. |
| --version (-v) | Prints version info and exits immediately. | <none> | |
| Command-Line Options for General Configuration of Syslogd2 | |||
| --ApplicationMode |
Boolean [disabled] |
This command-line macro is short for “--enable ApplicationMode”. Details can be found at the hyperlink. | |
| --Defaults (-d) | comma-separated list of keyword=<value> pairs | <none> |
This is the only command that can completely
re-set compiled-in values. The name=value sub-options
provide complete run-time over-rides for any compiled-in value
without restriction. Values set by this option become the new run-time default values. This keyword may appear multiple times. |
|
--Disable (-n) (--Suppress, --no) |
comma-separated list of “boolean-keyword [=value]” (no default value) |
<none> |
Same comments as for --enable. (I actually had trouble deciding on a name for this option.....) |
| --Enable (-e) |
comma-separated list of “boolean-keyword [=value]” (no default value) |
<none> |
The list of global boolean keywords and their
impact can be found here. This entry may appear multiple times. |
|
--IncludeConfig (-I, --Include) |
<none> --OR-- Absolute directory-path --OR-- Absolute filename |
--InputParms (-i, --input) --OutputParms (-o, --output) |
This option is used to include other
configuration files. They will be parsed as if they had been
in-lined at the point of the '--include' statement for purposes
of “first-come-first-served”. The secondary parameters (--input and --output) provide comma-separated option-lists that act as 'temporary default values' for the included file. This entry may appear multiple times. Include-files may also be nested. |
|
--Input (--Socket) |
Absolute pathname for Linux socket --OR-- IP hostname of an interface --OR-- IPv4 address --OR-- IPv6 address (no default value) |
comma-separated list of identity options and input options (will be a mix of boolean and name=value pairs) |
This is one of the most important commands in Syslogd2.
This entry may appear multiple times. |
| --Journal | <none> | comma-separated list of input options and journal-specific options |
The --journal command configures the default operation of the
specialized input threadpool that reads input from the systemd journal. SInce there is only one journal, there is only (ever) one threadpool, so this entry may appear only once. |
| --JournalField |
A systemd journal-specified field-name with an optional replacement name. [fieldname=newvalue] [no default] |
comma-separated list of journalfield-specific options |
Similar to Syslogd2 filters, journal fields can be default-enabled or default-disabled depending the --journal value of IncludeAllFields.
The default is disabled. In the default mode, the user specifies a list of desired fields using the --journalfield command. Fields will be displayed in the order they are requested, using any alternate name specified by the user. In the alternate mode, the user specifies a list of fields and may change their name or maximun length or may use the 'disable' boolean option to suppress that field. |
| --Kernel | <none> | comma-separated list of identity options and input options (will be a mix of boolean and name=value pairs) |
This (Linux-only) command is used to configure
local kernel input into Syslogd2. There are 2 methods of reading kernel input -- both of which require a compile-time option: CAP_KERNELTHREADS (preferred) or CAP_TAILFILES. This command is used with CAP_KERNELTHREADS. The filename is a global parameter obtained from system header-files so should not be specified. |
| --Network | comma-separated list of keywords (Any) | <none> |
This is a first attempt to make syslog responsive when run on a corporate laptop. The code has been well tested and works properly -- the concept is somewhat experimental, and yet (partially) implemented. More details can be found at the hyperlink. This entry may appear multiple times. |
| --Pipe | Absolute filename | comma-separated list of input options (will be a mix of boolean and name=value pairs) |
--Pipe defines and configures a
named-pipe as a Syslogd2 input source. It requires the CAP_PIPESIN feature-set. Please see the documentation of CAP_PIPESIN for details. |
| --systemd |
Boolean [disabled] |
This command-line macro is short for “--enable systemd”. Details can be found at the hyperlink. | |
| --TailFile (--Tail) | Absolute filename | comma-separated list of input options (will be a mix of boolean and name=value pairs) |
--TailFile defines and configures a
text file as a Syslogd2
input source. It requires the CAP_TAILFILES feature-set. Please see the documentation of CAP_TAILFILES for details. |
| Traditional Command-Line Options (Including new 'Systemd' and 'ApplicationMode' switches) | |||
| --inet (-i) | Boolean [disabled] | Enables IP support for input and output. | |
|
--Forward (-f) --forwarding |
Boolean [disabled] | Enables forwarding of traffic received over IP to IP-socket-connected destinations. | |
| --LocalHosts (-l) |
Semi-colon-separated list of hostnames. (empty-list) |
<none> | Any resolved hostname whose 'hostname' portion matches an entry in this list is logged to local files using the hostname-only. The entire FQDN is still transmitted to all remote processes. |
| --Remote (-r) | Boolean [disabled] | This macro is short for “--enable remote”. Enables both --inet and --forward. | |
| --StripDomains (-s) |
Semi-colon-separated list of domain-names (empty-list) |
<none> | For any resolved hostname, if all or a portion of its domain-name (starting from the right) matches a string in this list, that string is removed from the hostname before logging to local files. FQDN is still transmitted to all remote processes. |
| Command-Line Options for Parsing and Log-file Control | |||
| --Skip |
Positive Integer (default: 1 line) |
Skip (read and ignore) the specified number of logical lines in the configuration file before attempting to resume parsing input. This option can be used to 'skip over' lines that Syslogd2 would misinterpret or that are known to be in an incompatible format that would produce useless error-file entries if read and parsed. | |
| --SkipTo |
String [No Default] |
Skip (read and ignore) lines until the parameter string is found as a case-sensitive substring of the configuration file line (comment or command). This option can be used to 'skip over' lines that Syslogd2 would misinterpret or that are known to be in an incompatible format that might cause problems if read and parsed. | |
| [--enable +] 'SoftComment' |
<none> ('SoftComment' is the parameter.) |
<none> | Technically this is a <command-line-option-plus-value> This is also the only other command-line setting (other than --stderr and stdout) that takes effect as soon as it's parsed, so it also fits here. Multiple instances of enabling/disabling Softcomment are allowed. |
| --StdErr (-E) |
Absolute FilePath (no default) |
uid: int or string [default: 0] gid: int or string [default: 0] mode: octal value [default: 600] level: syslog pri [default: debug] sync: boolean [default: off] append: boolean [default: off] |
Specifies a logfile for error reporting during
parsing, build and execution. Recommended at top of config-file or on actual command-line for maximum effect. sync means to flush to disk after every write (same effect as not putting '-' on file-based output-lines). |
| --StdOut (-O) |
Absolute FilePath (no default) |
uid: (int or string) [0] gid: (int or string) [0] mode: (octal value) [600] level: (syslog priority) [debug] sync: (bool) [off] append: (bool) [off] |
Specifies a known output-file for error
reporting during parsing, build and execution. Recommended at top of config-file or on actual command-line for maximum effect. sync means to flush to disk after every write (same effect as not putting '-' on file-based output-lines). |
| Command-Line Options for Threadpool Configuration Management | |||
|
--ThreadMaps (--threads, -m, --maps) |
semi-colon-separated list of comma-separated list of keywords and values | <none> |
This command can be used to centrally
configure and customize threadpools so they won't need to be
modified by individual input-specifications.
This command cannot define or modify a threadpool to have fewer threads or queue-lines than the global default setting for that threadpool type. |
If started with no parameters whatsoever, all Syslogd2 daemons will look for /etc/syslog.conf and attempt to parse and utilize the contents of that file. If you wish to use a different file, the --configfile command-line option (-c filename or --configfile=filename) must be used.
As the name implies this keyword
allows the administrator to set default run-time settings or
(global variables) for all Syslogd2 variables.
As Syslogd2 matured, it became obvious real fast that using a unique parameter
for every setting I wanted control in various circumstances over was
not going to be feasible. (I literally ran out of letters in the
alphabet and nearly ran out of capital letters as well.) My solution
was to define a single command-line parameter that would use
name=value pairs and
that would act as an 'umbrella' for all the individual option values
I wanted. This was the birth and origin of the defaults
keyword.
~ --defaults markinterval = 1h, readers=4, fileuid=tom, filegid=staff, filemode=444, userfacility=extra1, kernelfacility-extra2
As mentioned above in the Configuring Boolean Values
section, These commands set or clear global boolean values. This list
of global boolean keywords will
expand if additional features are added at compile-time. Additional
boolean keywords are documented with the documentation that describes
the feature.
Every boolean value has a default
value. In all cases that default is “off”. If a boolean value
defaults to 'on', the name will be expressed in the negative
('Suppress......' --> which requires action to 'turn it off')
instead of in the positive ('Enable...' --> which requires action
to 'turn it on').
~ --enable inet, forwarding, dns = no, cache=no, NoForcePrintable ## turns off processing to rewrite binary content as printable chars.
This option (when set) is the same as --enable forwarding. It sets the same boolean switch internally, but is presented here as a direct command-line option. As in traditional syslog processors, this switch allows Syslogd2 to forward input received over an IP socket to transmit (or forward) that information over another IP socket to a remote host. With this option disabled (if the Inet boolean has been enable), Syslogd2 would be able to receive traffic from IP sources and Syslogd2 can transmit locally-generated traffic to remote IP hosts, but Syslogd2 may NOT re-transmit any data received to a remote host. The intent and purpose of this switch is to prevent loops in syslog reporting implementations.
I'm struggling with what this page
should contain. One screen does not seem large enough to to even list
all options -- much less sub-options. Any input would be
appreciated.
One thing that the usage page
DOES display is the enable-status of all CAP_* options and the number
of 'extra' facilities compiled into the given Syslogd2
binary image. This information is VERY useful as admins gain
confidence with Syslogd2
After displaying the information,
Syslogd2 exits.
This option enables or disable IP
support for all of Syslogd2.
It is a master on/off switch
for the IP protocol. For traditional (and still valid security)
reasons, IP support defaults to off
in Syslogd2.
Among the security reasons for
disabling IP support
by default is the consideration that a laptop on an unknown
coffee-shop or hotel network is still subject to hacking attacks
against the syslog port if it is open. Not having the port open to
accept traffic is the best defense.
The idea behind this option is to allow network-administrators to
pre-configure directories (with preset default settings) into which
application files can be 'dropped' to more easily handle
application-management without needing to make independent edits to
monolithic configuration files.
Alternatively, all logging requirements for individual
applications can be pre-defined by administrators and just 'dropped
into place' when those applications are installed on a new
(additional) host.
Input file and directory permissions are not under Syslogd2's
control, so must be set manually, however.
No threadpools are actually allocated until after all files have
been parsed and Syslogd2 has moved into its build phase.
During the build-phase, base structures for all referenced
threadpools are allocated, but unless they are actually used (a
socket is eventually assigned to them), they will be deleted at the
end of the build-phase before any system-resources can be consumed or
opened.
The use of the IncludeConfig command changes the format of
error-reporting from a simple line-number (such as “Warning: LIne
25: ….” to “Warning: Line <base-filename>:25: ...”.
This is intended to assist admins in resolving conflicts between
multiple input files.
Syslogd2 keeps a list of the
input-files it encounters to prevent IncludeConfig loops.
Syslogd2 places a limit on
the length of a single input file of 4095 lines. You may use up to
127 total files in your configuration in any way you desire (bush,
tree, forrest, hybrid, etc). In Syslogd2,
include-files can include other include-files or directories.
Three options are supported for the primary parameter.
NOTE: In the following example, the '--inputparams' and '--outputparams' options are subordinate to the '--IncludeConfig' option which is why they are not preceded with a tilde ('~') as expected from command-line opitons.
~ --includeConfig ## This is a simple 'include' to include all *.conf files in Syslogd2's specified ConfigDir directory.
## The next logical line (spanning 3 physical lines) includes all ('*.conf') files in a directory:
~ --includeConfig /etc/syslog.d/mysql.d \
--inputparams=id=1,readers=3, ignore-facility, facility=extra14 \
--outputparams id=8,gid=mysql,mode=660
# The next line sets up a file for network staff to manage their own output destination-list.
~ --includeConfig /etc/syslog.d/network.d/syslog.conf --inputparams id=3 --outputparams id=4
Note: The choice of delimiters (using '--' instead of commas for the two secondary options are (perhaps) an exception to the general rule that secondary options are combined into a single comma-separated list, but in this case, we are dealing with two lists of options and my choice was either to make this exception or to require that secondary option-lists be separated with semi-colons so the comma could be used to separate secondary options. This in turn would have meant that any secondary sub-option (normally separated by semi-coln) be separated by a new (one-off) delimiter. In the end, I decided in favor of keeping the delimiter hierarchy (mostly) intact and 'promoting' the --inputparms and --outputparms 'suboptions' to the syntax normally reserved for top-level command-line parameters. (This decision ultimately made parsing easier as well).
This is the Syslogd2
keyword to use to create a socket input specification. I refer to
this as a specification because not only does this option create a
user-defined input socket, but it can also assign the socket to a
threadpool (id=), increase (but never decrease) the minimum number of
threads in that threadpool (readers=), assign an input filter
(filter=), specify in which network states the entry is valid
(network=), specify IP versions to listen for with this socket
(version=), and much more.
The set of identity options is that subset of all input options that
directly affect how and what type of socket is built:
Linux sockets may be created anywhere
in the file-system (subject to system constraints), but are limited
to datagram sockets in the core feature-set. If the path-name for
the socket is omitted, an error is generated.
IP input-sockets are limited to UDP in
the core feature-set but use additional identity options:
Syslogd2 does extensive error-
and conflict-checking while building the configurations and is aware
that the Linux IP stack auto-maps IPv4 addresses to the IPv6 address
space when both are enabled.
Should conflicting or overlapping
IP addresses or ports be configured, Syslogd2
will keep the first entry ('first-come-first-served') and internally
mark the other entry as 'in-conflict' so it will not be used. A
warning or error message will also be logged to the stderr file (if
one has been defined).
~ --disable defaultip, syslog ## disable the default Linux and IP input ports as an (unnecessary) precursor to redefining them.
~ --input *, filter=defaultIP.filter, ## redefine the default IP port to use an input filter as of this line.
~ --input /run/systemd/journal/syslog, nohost, filter=defaultLinux.filter ## redefine the default Linux input socket under systemd with a filter.
This command sets parameters for the journalthreads threadpool.
This capability must be declared (with
CAP_JOURNALTHREADS) and enabled before it will activate.
All options to the --journal command are effectively global in scope. (As there is at most one [1] JournalThreads threadpool).
For individual field-level options and control, use the --journal-field command as additional configuration.
WIth no options, this command will produce standard syslog-formatted input records. Options to this command and to the
--journalfield command allow for a full range of control over how many and what order any (or all) of the journal-fields
will be added to each record as well as which journal sources information will be drawn from.
Though the journalthreads input-threadpool in Syslogd2 draws from the same source as rsyslog's journal-input module,
it is my belief that Syslogd2's implemendation is much more versatile.
Use this command to configure individual journal fields. The order in which the (multiple) instances of this command are encountered determines
the order in which they are included in the final syslog string. This command can re-name the journal's default name=value pairs or suppress or truncate
them as desired.
The --journalfield command complements the --journal command.
This command-line option is only available when either CAP_KERNELTHREADS is declared at compile-time and kernelthreads is enabled at run-time or when CAP_TAILFILES is declared at compile-time. The aforementioned feature-sets are the only mechanisms for reading local Linux kernel input into Syslogd2 Of the two, CAP_KERNELTHREADS is preferred. Please see separate documentation on the use and configuration of those feature-sets.
This is the traditional syslog
hostname-only display for local files. This option applies to local
files (and local files only). It will remove all domain-name
information from any source host matched by this list prior to
logging the message to local files.
The parameter to this command is a
semi-colon-separated list of host-names. Any source host-name
matching a name in this list will be logged using only the host-name
(without domain information).
The network command is stable and usable, but very
much an immature feature that shows lots of promise for laptops and
much promise for several other uses. This command is key to Syslogd2's
current and future network-state spooling capability for example.
This command is part of a
development experiment along with the 'network' option on input and
output connections, the spool
option on output commands, and the (embryonic) CheckNet
library and utility tool.
This command is (partially)
implemented in the sense that of the 3 default network-states
that are defined as of now, Syslogd2
already recognizes and accommodates them.
The non-implemented portions of
this command stem from a lack of information and experience to define
a comprehensive set of network tests and criteria for 'recognizing'
what network Syslogd2
is currently connected to (basically a method to configure the
CheckNet library and
lack of both bandwidth and experience to distinguish differences that
may be used to reliably and effectively define a laptop's “location”
when connected to a random external network).
Syslogd2
recognizes that '127.0.0.1' is available when the network-state is in
'Local' mode and will open only the IP ports that apply ('*' and
specific addresses for 127.0.0.1 or for ::1 or for the loopback
host-name).
Only when the network-state has been detected as Other
will Syslogd2 attempt
to open external sockets that may require DNS lookups (with lengthy
DNS timeouts if DNS is not reachable) or links to other hosts.
When connectivity is lost
Syslogd2 recognizes the return to the Local state.
Please see external documentation on input processing for more details on how
the 'network' options on connection specifications work and the
advantages this network-state recognition currently provides.
In addition to the practical gains made above, the factors behind the network experiment include:
When
Syslogd2 is deployed
on a laptop and when that laptop is operated off-line (or on a
different network), the Syslogd2 code
should be able to accept log entries from any input that it would
normally service, and (if the laptop is offline or attached to a
'wrong' network) spool that data until Syslogd2
determines that the network has been returned to the 'correct'
network at which point Syslogd2
will flush the entire (spooled) file to the designated receiver
including all status-updates and logged failures (including security
attacks ?) that have occurred during it's absence.
A second goal of the experiment is to provide a means to specify when
it is safe to transmit log data and when (because the laptop is
offline or on the wrong network), such data should be spooled for
later transmission. This goal covers the situation where employees
work from home on laptops and may only occasionally dial into work
via VPN. When connected to the VPN, Syslogd2
should (may) be updating the parent network's laptop-status or
reporting laptop hardware failures.
A third advantage
accrues in that when the laptop is re-located to a coffee shop
environment (or hotel, airport, etc), the external syslog input port
(if used at work) can be automatically disabled because it is on the
'wrong' network. Likewise all syslog-reporting output can be disabled
(in case there are any network-sniffers around looking for IT
bread-crumbs....).
This experiment needs at least 3 elements to work:
The --network
option (and the network
options on input and output connection specifications) address part 3
of the above needs. They specify which network-state(s) a given
configuration statement or connection is valid. For configuration
purposes, a pseudo-state is created ('Any').
This is the default for all entries -- both inet and non-inet alike.
The --network
command is designed to work by dividing the configuration file into
virtual 'sections' -- each containing modifications to the 'default
parsed configuration' to 'customize' the configuration for a
different set of network-states. This means that IPv6 and internal
cache might be enabled on the dhcp-enabled network at work, but
perhaps disabled on a network identified as 'home' or 'hotel'.
Since inputs and output definitions
cross all networks, they have their own 'network' option in their
configuration-lists.
This sensitivity to network-state
(combined with the output spool
option, is still useful in that data received by the Linux socket
(from services or the kernel) prior to the network being started may
be spooled and forwarded to a recording hosts AFTER the network is
fully up and running by use of the spool
option on a UDP output definition (and the marking of that
destination as Other
-- meaning 'not valid in Down
or Local').
The --Pipe command-line
option is used to specify and configure a named-pipe as an input
source for Syslogd2.
The --Pipe command-line option is only available when CAP_PIPESIN is declared at
compile-time. Please see separate documentation on the use and
configuration of named-pipes.
Remote [macro to enable IP and enable forwarding]
This option is more of a macro than a true 'command-line option'. It is the equivalent of --enable inet, forwarding. It is here primarily for historical consistency.
This command-line option (not applicable on the actual command-line) can be used to 'ignore' active configuration file lines such as rsyslog command-lines or certain output-lines to prevent Syslogd2 from attempting to parse them producing either unwanted results or extraneous error-file entries.
This command-line option (not applicable on the actual command-line) can be used to 'ignore' active configuration file lines such as rsyslog command-lines or certain output-lines to prevent Syslogd2 from attempting to parse them producing either unwanted results or extraneous error-file entries.
This command-line option defines
an error file for Syslogd2
The command takes effect (the error file is actually opened) while
the file is being parsed (normal processing is to parse the entire
file so all control options are read into simple array structures,
then those control options influence how the rest of the run-time
structures get built).
It is recommended that this line
be the 1st
line in the configuration file or on the actual command-line.
The parameter to this option is a
comma-separated list of values. The primary (first) parameter must
be an absolute pathname to the desired error file. All other options
are name=value pairs as described in the following table. As with
all keywords, all option-names and keyword-values below are
case-non-sensitive.
| Name (alias) | Type | Default Value | Comments |
|---|---|---|---|
| Uid (u) | String or Integer | 0 (root) | The desired owner of the file. |
| Gid (g) | String or Integer | 0 (root) | The desired group-owner of the file. |
| Mode (m) | 3-octal digits | 600 | The 'permissions mask' for the file. |
| Level (l) | String or integer | debug (7) | A syslog priority name or numeric value. |
| Append | Boolean | disabled | When opening this file keep any pre-existing content. (default is to truncate & restart the file from 0 bytes. |
| Sync | Boolean | disabled | Do not return from each write until the data has been flushed to disk. |
# ~ --stderr=/var/log/syslog.err, uid=root, gid=0, level=debug, mode=640
This command-line option
configures an auxilary output file that can be used as the target of
various advanced Syslogd2
options. It is not currently used by core Syslogd2
features.
The syntax, options, and rules
are identical to --stdErr
except for the ultimate purpose of the file (error output vs
informational output).
This is the traditional syslog
domain-stripping to shorten long host.subdomain.sub.domain names for
local files. This option applies to local files (and local files
only). It will remove any matching (full or partial) domain-name(s)
from the source-host FQDN (Fully-Qualified-Domain-Name) prior to
logging the message to local files. Because forwarding source-host
information with anything other than FQDN is poor network-management
practice, this function is limited to local-file-outputs only.
The parameter to this command is a
semi-colon-separated list of fully-qualified domain-names or
sub-domains. Processing attempts to match the FQDN to each list entry
from right to left. A match must be end precisely on a subdomain
delimiter.
The default is an empty list.
Syslgod2 is designed and
written (and has always been conceived) as a traditional (forking)
daemon. Therefore, modifications are necessary to work with the
relatively new 'systemd' init system in Linux.
This option does nothing more
than set the global boolean value --enable systemd
but is presented as a direct command-line option. This option is
automatically included by the pre-configured syslogd2.service file in
the install/systemd
directory. This file is installed if you choose to run 'make install
X' to make Syslogd2 your new syslog daemon (You must be the root user
to do so). (The 'X' stands for one of the suffixes that distinguish
different variants of Syslogd2).
The systemd
boolean switch makes the following modifications:
If you choose to manually configure these changes (or wish to
modify the default definition), please remember to add the nohost
option to the default Linux socket under systemd or your output will
appear to have an extra 'time'-field.
The nohost option is not required when Syslogd2 operates
under (more traditional) SysV init systems because incoming data is
not 'pre-processed' before arriving at Syslogd2.
The --TailFile command-line
option is used to specify and configure a text-file (usually an
application log file) as an input source for Syslogd2.
The --TailFile command-line option is only available when
CAP_TAILFILES is declared at compile-time.
Please see separate documentation on the use and configuration of
tailfiles.
The --TestConfig
command-line option provides a structured display of the entire
configuration. It is an extremely powerful tool for debugging and
for understanding exactly how Syslogd2
has been configured to run. It can also be used to show partial
configurations by selecting only certain sections for display. Output
can be sent to the logFile defined by --stdOut,
to the error file (--stdErr)
or to the user terminal (default).
It displays the configuration at any of
3 points during startup: AsParsed: after the parsing phase, AsBuilt:
after the build phase and AsStaged: just prior to starting actual
threaads. This command is safe to run on a system already running a
syslog process because (even though it uses the 'live' startup code),
no system resources (sockets, files, mutexes, semaphores, etc) will
have yet been initialized.
It can also show the resulting
configuration under the assumption that the network-state is other
than what it is actually detected to be. (This requires the
additional assumption that all declared input addresses (if IP is
enabled) are assigned to local interfaces and are valid).
The --TestConfig command-line option is only available when
CAP_WHATIF is declared at compile-time.
Please see separate documentation on the use of the --TestConfig
command-line option.
This command can be used to centralize
all thread-pool configuration so that input and output statements
need only specify to which of the resulting thread-pools they should
be assigned.
The parameter to this command is
a semi-colon-separated list of thread-pool definitions -- each of
which is a comma-separated list of options from the following table.
Syslogd2 supports
several different types of thread-pools overall, but the core
feature-set only supports the 'input' type (so-named because of the
keyword (input).
Because of Syslogd2's
inheritance rules, any omissions will
be filled from lower priority inheritance levels. Once Syslogd2
completes the build-process, any thread-pools that have not been
assigned sockets to manage will be deleted.
Note: Syslogd2
thread-pools are defined by two components: a type
and a numeric-identifier.
The only thread-pool type supported in the core feature-set is the
socket (input)
thread-pool type. All other thread-pool types require the inclusion
(at compile-time) of an optional feature-set.
| Name (alias) | Value Type | Comments |
|---|---|---|
| thread-pool type | Keyword token Mandatory / Identity) |
Identifies the type of thread pool to create or configure.
For housekeeping thread-pools, this token is housekeeping (house). |
| id |
Non-Negative Integer Mandatory / Identity) |
The numeric identifier of this threadpool. The
default thread-pool is always 0 (zero). Each unique positive
entry will define / create an additional (new) thread-pool.
Thread-pool identifiers need not be sequential. The range is all
(system-supported) positive integers + 0 (zero). Does not apply to kernel, user or housekeeping thread-pool types. |
| readers (r) | Positive Integer |
Number of input (reader) threads to create and assign to this thread-pool. This number may increase (but can never decrease) the (inherited) global number of reader-threads for a specific threadpool. Applies only to input, kernel, tailfile threadpool types. |
| queue (q) |
Non-Negative Integer [default: 0] |
Worker-queue identifier that will process all traffic from this threadpool. Applies only to input, kernel, tailfile threadpool types -- and then only when CAP_WORKERTHREADS is declared. |
| workers (w) | Positive Integer |
Number of worker threads to create and assign to this thread-pool. This number may increase (but can never decrease) the (inherited) global number of worker-threads for a specific threadpool. Applies only to User, Worker, Output, Housekeeping threadpool types. |
| lines (l) [ell] | Positive Integer |
Number of FIFO queue 'slots' to allocate in the defined threadpool (number of
'lines' in the threadpool's input buffer). This number may increase (but can never decrease) the (inherited) global number of buffer lines for the specific threadpool. Applies only to User, Worker, Output threadpool types. |
|
lossLess |
Boolean [Default: False] |
LossLess changes the queueing algorithm of worker, output
or user threadpools only.
It has no effect on other threadpool types. The default mode of operation is 'lossy': When the queue is full, new (incoming) data is discarded. Under lossless queueing, when the queue is full, incoming data is delayed. (the new data is not accepted until at least one message has been removed from the queue. This operational mode can result in data backups - particularly for streaming inputs (or data loss for datagram inputs, but has usage in some situations. Examples of such situations include reading from pre--exisiing or dynamic log files where speed of input is not important or reading from the systemd journal where avoiding message loss is more important than speed. |
|
pollInterval (pollTime) (p) |
Positive Integer |
PollInterval may
decrease (never increase) the polling-cycle interval for the
thread-pool. A polling-cycle ends when every active file has
been checked and found to have no new data. Applies only to TailFile threadpool types and the Kernel thread-pool when in 'polling' mode. |
# In the
line below, 3 threadpools are defined. Input threadpool 12 inherits
the global setting for reader threadcount.
# The global default value may be the result of compiled-in-values or the
over-ridden value set by [~ --defaults = readers=<N>]
# ~ --threadmaps input, id=0, readers=4; input, id=4, r = 3; input,
id=12; housekeeping, w=4
# ~ --threadmaps input, id=8024, readers = 2
This command displays the compilation
version, the enable status of all CAP_* options and the number of
'extra' faciltiies compiled into the given Syslogd2
binary image.
After displaying the information,
Syslogd2 exits.
Syslogd2 uses many boolean values. Boolean values are used to enable or disable optional processing steps (both globally and at the individual connection-level).
They are used to enable or disable entire features, as configuratioin values, or to enable / disable optional processing.
Boolean values are set / reset with the --enable/--disable command-line options.
Because of the impact of inheritance on boolean settings, Syslogd2 recognizes the need to be able to clearly and unconditionally set or reset boolean values, and allows an optional value to be used for this purpose. If the results seems strange to you, remember that “two negatives cancel each other out”.
| Boolean Value | Example | Meaning |
|---|---|---|
| <missing> (not set) |
|
This boolean value is set to its inherited value. |
| Present with no value. |
--enable variable --disable variable |
This boolean value is forced “on” (enabled). This boolean value is forced “off” (disabled). |
| 'yes', 'y', 1 |
--enable variable = y --disable variable = yes |
This boolean vale is forced “on” (enabled). This boolean value is forced “off” (disabled). |
| 'no', 'n', 0 |
--enable variable = n --disable variable = no |
This boolean value is forced “off” (disabled). This boolean vale is forced “on” (enabled). |
The following is a list of the global boolean
keywords supported by Syslgod2's
core feature-set. Additional boolean values will be added by
optional feature-sets that are selected when compiling an individual
binary instance.
It is worth noting that setting optional boolean values whose
feature set is not enabled will not produce an error message. Any
such settings are quietly ignored.
The --defaults command-line option is used to configure global variables.
It accepts a comma-separated list of keyword=value
pairs where each keyword is the name of a global variable to which a
value is to be assigned.
This section documents all Syslogd2
global-variable keywords -- even those that are not in the
'core feature-set' of Syslogd2.
For those keywords that require or configure optional features, the
applicable feature is identified as a dependency. Additional information (with context) on feature-dependent keywords can be found on the capabilities page.
Because the implementation of some configuration settings depends on which
optional features were compiled into the
code, it is possible that otherwise valid keywords may (in context) not
be accepted by a particular binary if their underlying feature was not
included at compile-time. Syslogd2 error messages attempt to distinguish
between valid keywords that are not supported due to non-included features and unknown
(invalid) keywords.
Unknown (or unsupported) options are flagged to the error log with what I
believe may be their intended value, often highlighting misspellings
or invalid syntax.
Reminder: All keywords in Syslogd2
are case-non-sensitive.
Intervals in Syslogd2
are measured from the end of processing of one pass to the start of
processing for the next pass. Due to race-conditions and resource
contention, intervals should be considered 'minimum values' rather
than any attempt at exact timings. Should resource conflicts occur,
a scheduled process may not run at the precise time expected, but
(because it is due or past-due) will run as soon as resources become
available. (Glossary: Time Interval)
|
Keyword (Alias) DEPENDENCY |
Value Type (default] |
Comments |
|---|---|---|
| Simple Intervals | ||
|
MarkInterval (m) |
time-string [1h] |
The interval at which inactive files receive a '-- Mark --' message to demonstrate they are still active. |
|
OutputFileCheckInterval (FileCheckInterval, ofci, f) CAP_FILEROTATE |
time-string [20m] |
The interval between output-file 'size-check' cycles. Set to zero to disable this feature. |
|
MaxFileSize (mfs) CAP_FILEROTATE |
size-string [2g] |
The minimum size at which an output file qualifies for rotation during a 'size-check' processing cycle. |
|
MaxSpoolFileSize (maxsfs) CAP_SPOOLFILES |
size-string [20M] |
Maximum size a spool-file is allowed to grow before some type of 'limiting action' is taken. |
|
StatInterval (si) CAP_STATS |
time-string [0] (disabled) |
Time between
updates to performance statistics counters. Set to zero to disable this feature. This feature is not yet complete - it may not be stable. |
|
PollInterval (PollTime) CAP_TAILFILES or CAP_KERNELTHREADS |
Integer [60] (seconds) |
Interval at which files are polled to check for new input. Applies to entire threadpool. May be decreased on per-threadpool or per-input basis, but not increased. |
| Thread-Pool Configuration | ||
|
Readers (r) |
Positive Integer [2] |
Default number of threads to create for each socket threadpool. (Also applies to TailFile threadpool types). |
|
Workers (w) CAP_WORKERTHREADS |
Positive Integer [3] |
Default number of
worker threads to create in each worker-type threadpool. May be increased by --threadmaps command or individual input specifications, but not decreased. |
|
WorkerLines (wl) CAP_WORKERTHREADS |
Positive Integer [30] |
Default Number of
'message slots' to create in each worker-type threadpool's FIFO
queue. May be increased by --threadmaps command or individual input specifications, but not decreased. |
|
OutputWorkers (o) [lowercase 'oh'] CAP_OUTPUTTHREADS |
Positive Integer [3] |
Default number of output threads to create in each output-type threadpool. May be increased by --threadmaps command or individual input specifications, but not decreased. |
|
OutputLines (ol) CAP_OUTPUTTHREADS |
Positive Integer [30] |
Default Number of
'message slots' to create in each output-type threadpool's FIFO
queue. May be increased by --threadmaps command or individual output specifications, but not decreased. |
|
UserWorkers (uw) CAP_USERTHREADS |
Positive Integer [1] |
Default number of
user threads to create in the optional user-thread threadpool
when enabled. May be increased by --threadmaps command but not decreased. |
|
UserLines (ul) CAP_USERTHREADS |
Positive Integer [5] |
Default Number of
'message slots' to create in the optional user-thread
threadpool's FIFO queue when enabled. May be increased by --threadmaps command but not decreased. |
|
KernelReaders (kr) CAP_KERNELTHREADS |
Positive Integer [3] |
Default number of
kernel threads to create in the optional kernel-thread threadpool
when enabled. May be increased by --threadmaps command or individual input specification(s), but not decreased. |
|
HouseKeeping (hk) CAP_HOUSEKEEPING |
Positive Integer [2] |
Number of
housekeeping threads to create in the optional housekeeping
threadpool when enabled. May be increased by --threadmaps command but not decreased. |
|
Nice (NiceValue) |
Integer [Range: -21 to 19] [-21] |
'Nice' value for parent thread. Works like Unix/Linux 'nice' command. If value is -21, this setting is disabled and all threads run at system-default 'nice' priority level of 0. Otherwise: Parent thread runs at the specified 'nice-priority' level nice-value. All input-threads run at nice-value - 1. Any worker-threads run at nice-value - 2. Any output-threads or user-threads run at nice-value - 3. Any housekeeping-threads run at nice-value + 1. |
| System-derived or System-related | ||
|
ConfigDir (cd) |
absolute-directory path [etc/syslog.d] |
ConfigDir is where
Syslogd2 expects to find all of its input files:
Cache-file. |
|
SpoolDir (sd) CAP_SPOOLFILES |
absolute-directory path [var/spool/syslog] |
This directory must be created by the admin. If missing, all spooling will be disabled. This directory will not be created by Syslogd2. |
|
HostName (HN) |
string [from system] |
This is the name Syslogd2 will use to identify itself in local logging. This allows the log-administrator to over-ride the value obtained from a (possibly broken) system-call. |
|
DomainName (DN) |
string [from system] |
This is the name Syslogd2 will use to identify itself in local logging. This allows the log-administrator to over-ride the value obtained from a (possibly broken) system-call. |
|
UserFacility (UserFac) |
string or numeric format [user.notice] |
Global default facility and priority assigned when incoming non-kernel data does not contain either facility or priority information. |
|
KernelFacility (KernelFac) |
string or numeric format [kern.notice] |
Global default facility and priority assigned when incoming kernel data does not contain either facility or priority information. |
|
LogFacility (LogFac) |
string or numeric format [syslog] |
Syslog facility Syslogd2 will use to report its own errors (parsing, build-conflicts, etc) after it is up-and-running. Set to 'none' or an empty string to route this traffic to the stderr file instead (or discard if stderr is not defined). Syslogd2 generates no messages that do not contain facility and priority, so the '.priority' component is not required. (It will be ignored if present). |
|
KLogFile (KFile) |
The name of the default kernel log file. [from system] |
Even though this is
system-obtained, porting to a different OS may not support this
value. This allows the administrator to designate a different
file as 'kernel input'. (For Linux, this filename is /proc/kmsg.) Perhaps a file produced by a native logging daemon that Syslogd2 is to read and parse into an overall network-management system.) |
| LogSocket |
The default Linux input socket. [from system] |
On Linux the system
reports this as /dev/log. Under systemd init-scripts on Linux, this changes to /run/systemd/journald/syslog because the systemd init scripts want to intercept (and control) all syslog traffic. This is the location that Syslogd2 will open as its 'default Linux input socket'. |
|
FileOwner (FileUid) |
string or integer [root] |
The default owner for all output-files created by Syslogd2. This value can be over-ridden on a per-file basis via the Uid output option. |
|
FileGroup (FileGid) |
string or integer [root] |
The default group-owner for all output-files created by Syslogd2. This value can be over-ridden on a per-file basis via the Gid output option. |
|
FileMode (FMode) |
3 octal digits [600] |
The default file-mode (permissions) for all output-files created by Syslogd2. This value can be over-ridden on a per-file basis via the Mode output option. |
|
SocketOwner (SocketUid) |
string or integer [root] |
The default owner for all input Linux sockets created by Syslogd2. This value can be over-ridden on a per-file basis via the Uid input option. |
|
SocketGroup (SocketGid) |
string or integer [root] |
The default group-owner for all input Linux sockets created by Syslogd2. This value can be over-ridden on a per-file basis via the Gid input option. |
|
SocketMode (SMode) |
3 octal values [666] |
The default file-mode (permissions) for all Linux input sockets created by Syslogd2. This value can be over-ridden on a per-file basis via the Mode input option. |
| Schedule-based Settings | ||
|
FlushIntervals (fi) |
basic schedule [30 60 90 120] |
Intervals at which inactive (or low-volume) files with any received duplicate messages are flushed to disk along with their current repeat-count. This value does not apply if the AllMsgs boolean flag is set. |
|
SourceCheckIntervals (sci) |
enhanced schedule [0] (disabled) |
Intervals at which
input files and sockets (both Linux and IP) are re-opened if
temporarily closed due to communication errors or network failure
during normal operation. Normal operation will make one attempt to open each socket/input-file. If that fails, the socket/file is marked & ignored until this schedule runs CheckSources to attempt to re-open failed inputs in the background. |
|
DestinationCheckIntervals (DestCheckIntervals, dci) |
enhanced schedule [0] (disabled) |
Intervals at which
output files, sockets (both Linux and IP), pipes and devices are
re-opened if temporarily closed due to communication or other
errors during normal operation. Normal operation will make one attempt to open each socket/file/pipe/device. If that fails, the connection is marked 'down' & ignored until this schedule runs CheckDestinations to attempt to re-open failed connections in the background. |
|
FilterCheckIntervals (fci) CAP_FILTERSIN -or- CAP_FILTERSOUT |
schedule [0] (disabled) |
This schedule
controls how often Syslogd2 checks to see if any filters
have been updated and if so, parses and implements the new
filters “on-the-fly”. Finding a modified (or new)
filter-file is considered a 'success' and resets the global index
value to 0 (zero). For additional details on Syslogd2 filters, please consult external documentation. |
| Programmable Interrupts | ||
|
SigHup |
Function-name [RotateFiles] |
These 4 programmable signals may each be preset with one of a number of
different routines that are executed when the appropriate signal
is sent to the daemon using the 'kill' or 'killall' commands.
kill -HUP <pid>' |
| SigInt |
Function-name [CheckFilters] |
|
| SigUsr1 |
Function-name [CheckSources] |
|
| SigUsr2 |
Function-name [CheckDestinations] |
|
| Other (un-grouped) settings | ||
|
SelfAddress (Self) |
Semi-colon-separated list of: IP-address [[=] <whitespace-separated list of hostnames> [No default value] |
Some systems with
broken IP implementations (Such as OSX's IPv6) do not properly
report IP addresses assigned to their interfaces. This command
allows admins to manually declare IP addresses (and associated
hostname(s)) as configured on local interfaces so they can be
used as input sockets. This option is not needed if Syslogd2's default interface scan correctly identifies all locally-assigned iP addresses. Multiple instances of this option are allowed. Example: # ~ --defaults self = fec0:2::90 = linux1 linux2; fec0:3::80=linux4; 1.2.3.4; 2.4.5.6 myhost |
|
SpoolFileAction (sfa) CAP_SPOOLFILES |
integer [0] (zero) |
Action to take when a spool-file has reached maximum size. Currently hard-coded (only one action is currently supported).
0: Stop spooling |
As a reminder on intervals in Syslogd2,
a time-interval expresses the minimum interval between the end of the previous cycle
and the start of the next cycle. Due to resource conflicts, not all
scheduled events will start at the expected intervals over time
(though they should remain 'close' -- within a few seconds of the
expected start time.) This is primarily a result of the decision to
make 'housekeeping chores' like 'mark', flushing duplicate messages,
reopening failed connections and automatic file rotation lesser
priority activities than processing message traffic.
Please see external documentation
on the housekeeping threads
for details on why this decision was made.
This variable allows the
administrator to move (redefine) the ConfigDir
[Configuration Directory] value for Syslogd2
from /etc/syslog.d to
another filesystem location. ConfigDir
is the default location where Syslgod2
looks for all ancillary input files (included-configuration files,
filter files, cache file, etc).
If the --IncludeConfig command-line option is used either
with a relative filename or with no parameter (implying the use of
ConfigDir), any desired changes to this setting must precede
any such usage in the configuration file. No change to this variable
will be allowed after a referencing --IncludeConfig
command-line option has been processed.
# ~ --defaults configdir = /etc/syslogd2.d ## restates the default location for ConfigDir for purposes of example.
DestinationCheckIntervals schedules the execution of the
CheckDestinations internal housekeeping routine. Like
CheckSources does for source
inputs, CheckDestinations
will attempt to rehabilitate any failed output connections (sockets,
pipes, files, or devices). On success the (newly rehabilitated)
destination is put back 'into service' for live traffic. On failure,
the link is marked as unavailable and another attempt at
rehabilitation will be made during the next applicable cycle.
In addition to simply checking
and resetting connections, however, CheckDestinations
also checks to see if any failed connection with spooled data is now
reachable for traffic delivery. If it finds such a situation, it
schedules a separate (possibly long-running) housekeeping process to
'unspool' (flush) the accumulated data to the remote host over the
newly-opened connection, then resets all spooling indicators, letting
the 'live' threads know the connection is available again but the
spool-file is (temporarily) unavailable until the 'flush' process is
complete.
Please refer to separate
documentation on Syslogd2's
output processing practices and on the spooling subsystem for more
information on those topics.
This is the global DNS domain-name of the local host. This is obtained from the system via a system call, but in some circumstances, the administrator may wish to change the host-name and/or domain-name Syslogd2 uses to identify itself for local logging purposes. This variable provides that over-ride mechanism.
This value is the default group ownership for all output files created by Syslogd2. Option format can be a group name or a numeric group-id number (gid). The default is group 0 (zero). This value can be over-ridden on a per-file basis using configuration-file output-line option-list options.
This value is the default file mode (permissions) for all output files created by Syslogd2. Option format is a 3-digit octal value. The default value is 600. This value can be over-ridden on a per-file basis using configuration-file output-line option-list options.
This value is the default owner for all output files created by Syslogd2. Option format can be a user name or a numeric user-id number (uid). The default owner is root (uid 0) (zero). This value can be over-ridden on a per-file basis using configuration-file output-line option-list options.
FilterCheckIntervals schedules execution of the CheckFilters housekeeping subroutine. This routine (when run) walks the list of all known filters and compares the stored modify-date timestamp to the one on disk. If the file has been updated, deleted or added (as a new file), the CheckFilters process will parse the new file and insert the new (updated) filter into the active list immediately. The CheckFilters application may also be run via interrupt-call or via the command-tool interface. For more details see the external documentation on Syslogd2 filters and on Syslogd2's interactive command-tool.
FlushIntervals is a schedule for how long duplicate messages to local files may be stored (and counted) before being flushed to disk. Each time a 'different' message is written to the file, the old message is 'flushed' and the index resets to zero. As long as the same message continues to be received (again and again), the index will be incremented as each time-period expires. When the index reaches the end-of-the schedule, it will continue to use the last entry until reset. This variable SHOULD be quickly approaching obsolescence, especially since the dreaded “Last message repeated....” message is totally meaningless to network-management systems, but Syslogd2 continues to support it for historical consistency.
This is the host-name of the local host. This is obtained from the system via a system call, but in some circumstances, the administrator may wish to change the host-name and/or domain-name Syslogd2 uses to identify itself for local logging purposes. This variable provides that over-ride mechanism.
HouseKeeping defines the
number of threads to be allocated for the optional housekeeping thread-pool. The housekeeping
thread-pool thread-count may be
increased (never decreased) by the --ThreadMaps
command for the houseKeeping
thread-type. The --ThreadMaps
command may never decrease the number of threads in the housekeeping
thread-pool, but may increase it.
Please refer to external
documentation on the HouseKeeping
feature for more details.
This is actually a traditional
syslog parameter that Syslogd2
exposes to administrative control.
KernelFacility (default:
kern.notice) is the
default facility.priority
value assigned to all kernel-originated events that do not have
assigned facility and/or priority values when received by Syslgod2.
KernelReaders defines the number of threads to be allocated for the optional kernel thread-pool. This thread-pool is supported on Linux only and is the preferred method of reading local kernel log messages. KernelReaders may be increased (never decreased) by the --ThreadMaps command for the kernel thread-type or by options in the option-list of kernel command-line definitions. Neither --ThreadMaps nor input configuration options may decrease the number of threads in the kernel thread-pool, but either may increase it.
The KLogFile
(KernelLogFile) default value
(/proc/kmsg on Linux) is
normally obtained from a system-call. This does not mean that when
running as an application (--enable ApplicationMode),
an administrator may wish to read a different file as kernel input
(such as /var/log/kern.log) -- especially since 2 applications
interfere with each other if both try to directly read the kernel
buffer. This variable allows Linux systems to read kernel input from
an alternate file if desired. It also allows future ports of
Syslogd2 (if I'm so lucky) to
specify kernel log files on their hosts.
This parameter will be used as
the filename for all --kernel
commands that specify the 'file-polling' method of input. It is also
used for comparison with all --tailfile
entries to detect any attempts to define the system kernel file as
regular input without also specifying the kernel
sub-option. If such an attempt is made, (based on the status of
CAP_KERNELTHREADS and CAP_TAILFILES) either the --tailfile
entry will be converted to a --kernel
entry or vice versa.
Please refer to the external
documentation for details about reading kernel input in Syslogd2.
LogFacility (default:
syslog) is the
facility value that
Syslogd2 will use to
report issues, notifications and errors to down-stream processes once
initial startup is complete.
The LogFacility setting will
re-direct all configuration or operational issues that are detected
during any activity that occurs after initial startup of Syslogd2
to the syslog facility specified. This syslog data stream may then be
written to disk, modified with output-filters, or sent to a central
host to monitor run-time issues from the Syslogd2
daemon.
The general idea is that (as a
network management tool), Syslogd2 should
be able to use the syslog protocol to report it's own 'distress',
issues, and status through the syslog network-management reporting
hierarchy.
When run as the primary syslog
process, there is (clearly) no way to report issues through a syslog
daemon that has not yet been initialized. This is why an error file
(--stdErr) is
necessary at startup. Post-startup, there is no reason not to switch
to 'injecting' Syslogd2's
errors and notifications into the (now-existing) syslog processing
stream to allow remote (consolidated) monitoring of run-time
(operational) errors and notifications instead of requiring admins to
log in and review individual log files.
Re-routing traffic via the syslog
protocol can be disabled by setting LogFacility
to either an empty string or to the value 'none' as follows. When
re-routing is disabled, notifications will be sent to the stderr
file. If no stderr file is defined, these notifications will be
discarded.
(Note: If an invalid syslog
facility is provided (typos are known to happen), LogFacility
will be disabled.
# ~
--defaults LogFacility= [, …]
# ~ --defaults LogFacility = none [, ...]
LogSocket is the full filesystem
path to the Linux socket that Syslogd2
should open as its default (primary) input. The default is obtained
from a system-call (/dev/log for SysV init systems). The --systemd
command-line option changes this value to
/run/systemd/journald/syslog
to be compliant with the systemd init scripts. This value is parsed
before the default input socket is opened, so should an administrator
wish to change the location to elsewhere (because of running in
applicationMode and not wishing to cause resource-conflict with an
existing syslog daemon, this is where that would be done.
Note that modification of the
LogSocket value is
part of the configuration change that occurs when the --systemd
command-line option is present. Moving the default Linux log socket
and the use of --systemd
are mutually exclusive options: moving the default socket from
/dev/log to anywhere but the systemd-defined
location defeats the need for the --Systemd
configuration change.
# ~ --defaults logsocket = /newdir/log
This configures the traditional syslog 'mark' message for
low-volume local files. Console devices will not receive 'mark'
messages from Syslogd2,
meaning the last 'useful' message will remain on the console screen
(with it's time / datestamp) for as long as possible.
In a departure from tradition,
Syslogd2 expresses the
MarkInterval as a
time-string, allowing
the administrator to choose the units in which to declare it and
simplifying the conversion of long time-delays into minutes (with
possible human error if converted to seconds) at the same time.
The 'mark' messages is disabled if this
interval is set to 0 (zero).
MaxFileSize is the maximum size
an output file can grow when the CAP_FILEROTATE feature is present
and enabled before the file becomes eligible for rotation. Once
eligible for rotation, the next file-rotation cycle will internally
lock the destination, close the file, rename the file, open a new
file and unlock the destination for output by active threads. To
disable file rotation, set the global OutputFileCheckInterval
value to 0 (zero). This is a 64-bit value on both 32- and 64-bit
systems.
Because we are dealing with 'big
numbers', the size-string
input format was chosen to guard against human error in counting
digits.
The value of '2g' (lower-case 'g') as a
default value was chosen to provide (for high-volume input)
sufficient 'head-room' to grow until the next file-rotation cycle.
All sorts of 'bad things' can happen with 32-bit code and
applications when they try to process files larger than '2G'
(upper-case 'G'):
Syslogd2
was initially developed on and for 32-bit Linux. On 32-bit systems,
files (with rare exceptions) have a 'hard' size limit of '4G'.
Because [(2 ^ 32) = '4G'], '4G - 1' is the largest (unsigned)
positive value that can be expressed with a 32-bit value. 32-bit
systems use unsigned 32-bit values as file-pointers making 4G the
largest file-size usable on 32-bit systems.
Unfortunately,
many (most?) Microsoft-based analysis tools of the time (improperly)
used signed (vs unsigned) 32-bit internal values when processing
files. This means that the about half-way through reading a 4GB input
file, the 'high-order bit' of the file-pointer would get set and
(because it is signed vs unsigned) the file-pointer would then turn
into a (large) negative value. This (not surprisingly) caused the
application to crash.
Keeping the
file-size under 2G resolved this issue 20 yrs ago and allowed all
Microsoft tools (too often the tools of choice for security analysis)
to read and process raw log-files whether they used internal signed
or unsigned file-pointers. When Microsoft started shipping 64-bit
code, it surprised nobody that they left a LOT of 32-bit code in
their operating system and utilities that perpetuated this problem.
I have no confirmation that they have removed all 32-bit references
from their 64-bit products today, but it's probably a safe bet that
somewhere 3rd-party applications running on 64-bit systems still use
signed 32-bit values as internal file-pointers.
Secondarily,
because 32-bit systems (both Linux and MSWindows) are still around, I
think prudence requires that I assume a conservative value for
maximum file sizes based on the foregoing discussion. Hence the '2g'
default in order to avoid files exceeding '2G'.
MaxSpoolFileSize specifies
the maximum size a spool-file may grow to before automatic action is
taken as a limit on 'run-away' growth. The default is '20M' due to
the anticipation that a relatively short SpoolFileAge of 2h combined
with robust filtering will decrease the demand for lots of
transmitted syslog output (and hence less need for spooling). Like
the MaxFileSize value,
this is a 64-bit value on both 32- and 64-bit systems.
This value is only valid when the
CAP_SPOOLFILES feature set is selected at compile-time.
When a spool-file size exceeds
this value, the action specified by SpoolFileAction
is taken.
Refer to external documentation
for details on how the spooling feature works in Syslogd2.
This value allows child threads to be created with various
system-level scheduling priorities in an attempt to better manage queues and
overall data flow.
With a value of -21, Syslog will retain the default thread-scheduler priority
(nice value of 0) and scheduler selection (SCHED_OTHER) for all threads.
Other values will cause Syslogd2 to use the SCHED_RR (Round-Robin) (basically
SCHED_FIFO with time-slicing) scheduler and set the thread priority value (if not clipped
by system limits) so that housekeeping threads (responsible for support tasks only) run at
one priority level higher (one lower run-time priority level) than the parent thread that
will be configured to run at priority level nice-level.
Output threads (if any) are considered the most 'crucial' child-thread type and will be assigned the
lowest priority value (highest run-time priority) of nice-level - 3 in an
attempt to keep the internal output-queues as empty as possible.
Worker threads (if any) are 2nd most important and will be assigned nice-level - 2 in an
attempt to keep the internal worker-queues as empty as possible.
Finally, all input threads wil be assigned the priority of nice-level - 1 in an
attempt to try to keep up with as much offered traffic as possible in competition with
other jobs on the host.
The projected use for this setting is primarily on systems that run Syslogd2 as a primary
application (or those that run on busy server systems) where administrators want to assure that
sufficient CPU cycles are provided to Syslogd2 to allow it to compete with other applications or
(on dedicated hosts) to out-compete user-activity that may also be present on the same host (such as
log-file manipulations).
This value sets the interval
between file-rotation cycles when the CAP_FILEROTATE feature has been
compiled into the current instance of Syslogd2.
The default is '20m'.
If the value is set to 0 (zero),
file-rotation is disabled. Negative values are not allowed.
OutputLines specifies the
number of lines
(message-slots) to be
allocated for each output
thread-pool type FIFO queue. This value may be over-ridden by the
--ThreadMaps command
or by options in the configuration file's output-line option-list.
Neither --ThreadMaps
nor the output configuration options may decrease the number of
buffer-lines in a given threadpool, but either may increase it.
Refer to external documentation
for details on the output thread-pool
and how thread-pools work in Syslogd2.
OutputWorkers defines the
number of threads to be allocated for each output thread-pool type. This value may
be increased (never decreased) by the --ThreadMaps
command for individual thread-pools or by options in the
configuration file's output-line option-list. Neither --ThreadMaps
nor the output configuration options may decrease the number of
threads in a given threadpool, but either may increase it.
Refer to external documentation
for details on the output thread-pool
and how thread-pools work in Syslogd2.
PollInterval is the default
interval between when one input-file polling-cycle completes and the
next cycle begins. it is a threadpool-level parameter. This value can
be decreased (never increased) by the --ThreadMaps
command for individual threadpools or by individual file-inputs as
they are assigned to thread-pools.
An 'input-file polling-cycle' lasts as
long as any of the files being polled present new input. The cycle
ends only when 1 complete check of all files results in no new input
being available to read.
This sets the default polling interval
for kernel-threads (if CAP_KERNELTHREADS was selected and enabled and
if the polling input-method is selected for kernel input).
It also sets the default polling
interval for --tailfile input
thread-pools (if CAP_TAILFILES was selected and at least one
text-file input-source was configured).
Refer to external documentation
for details on how the kernel-input and TailFile feature-sets work in
Syslogd2.
Readers defines the number of threads to be allocated for each input-type thread-pool. Input-type thread-pools are types kernel. socket, and tailfile. This value may be increased (never decreased) by the --ThreadMaps command for individual thread-pools or by options in the option-list of individual input command definitions (--input, --tailfile, --kernel. Neither --ThreadMaps nor input configuration options may decrease the number of threads in a given thread-pool, but either may increase it.
This global variable is used to
define IP addresses and associated hostnames assigned to local
interfaces that cannot be properly detected by Syslogd2's
system-call method due to broken IP stacks or broken system calls.
It's origin is in my early
attempts to continue development of Syslogd2
on my MacBook Pro. I quickly discovered that the (perfectly valid)
IPv6 address of “fec0:2::90” was reported by the OSX utilities to
Syslogd2 as “fec0:0:2::90” (the wrong address). Furthermore, the 'ifconfig'
utility crashed when I tried to verify the address via the
command-line.
In order to continue development,
I created the “SelfAddress” variable to declare (as in
/etc/hosts) that a
particular address with a particular set of hostnames should simply
be assumed to be present and valid on the local IP network interface
for purposes of Syslogd2's
configuration and validation of input IP addresses.
The input format is
semi-colon-separated list of entries. Each entry consists of a
comma-separated list starting with an IP address followed by zero or
more hostnames.
# ~ --defaults selfsddress = fec0:2::90, ubuntudesktop, linuxlaptop; 1.2.1.2, dummyhost
Each of these four variables may be configured with the name of one of a list of housekeeping routines that will be run when that interrupt is invoked from the command-line. Invocations may either use the kill or killall commmand syntax. Proper syntax to invoke these interrupts is:
The values '-HUP', '-INT', '-USR1' and '-USR2' must be expressed
in upper-case or the command will fail.
The list of available subroutine names that these interrupts can
be configured for depends on the optional feature-sets that have been
selected at compile-time. Most of these routines have their own
timers or schedules. Executing the routine via interrupt does not
wait for the next scheduled execution and will execute the subroutine
(on a one-shot basis) even if the schedule has been disabled. This
provides an alternative way to run up to 4 of these routines via
either crontab daemon or interactively at need.
|
Function Name DEPENDENCIES |
Comments |
|---|---|
| CheckDestinations | Resolve any output connections that can be newly resolved, deconflict all entries and rehabilitate destination connections. Flush spool-files (if any) on success. |
| CheckSources | Resolve any source connections that can be newly resolved, deconflict all entries and rehabilitate input connections. |
| Mark | Send '-- Mark --' messages to all local files. |
|
ResetCache CAP_CACHE |
Create and activate a new (freshly-created) internal name-cache from the contents of /etc/hosts and the defined cache-file (if any). |
| NoOp | This is a no-operation function. It will do nothing.(It essentially sets the selected interrupt to a 'disabled' condition). |
|
RotateFiles CAP_FILEROTATE |
Check the current size of all output-files, rotating any file that is larger than MaxFileSize bytes and creating new (emtpy) output files for live traffic. |
|
CheckFilters CAP_FILTERSIN -or- CAP_FILTERSOUT |
Compare all each filter-timestamp in memory to
the appilicable file's on-disk timestamp. If any disk file has
been deleted, added or modified, re-parse that filter and
implement the modified filter immediately. |
|
FlushSpoolFiles CAP_SPOOLFILES CAP_STREAMOUT |
Search the list of outputs, inspecting all streaming Linux sockets, pipes and TCP sockets to see if they have both a valid connection and a stored spool-file on disk. If so, flush the spool-file over the open connection, then delete the spool-file and reset all spool settings to 'normal'. |
~ --defaults sighup = RotateFiles, sigint = CheckFilters, sigusr1 = CheckSources, sigusr2 = CheckDestinations
This value is the default group ownership for all input Linux sockets created by Syslogd2. Input format can be a group name or a numeric group-id number (gid). The default is group 0 (zero). This value can be over-ridden on a per-socket basis using the --input command's option-list arguments.
This value is the default filesystem 'mode' (permissions) for all input Linux sockets created by Syslogd2. Input format is a 3-digit octal value. The default value is 666. This value can be over-ridden on a per-socket basis using the --input command's option-list arguments.
This value is the default owner for all input Linux sockets created by Syslogd2. Input format can be a user name or a numeric user-id number (uid). The default is root (user 0) (zero). This value can be over-ridden on a per-socket basis using the --input command's option-list arguments.
SourceCheckIntervals schedules the execution of the CheckSources internal housekeeping subroutine. CheckSources' job is to rehabilitate any source that has encountered errors and to initialize any newly-resolvable entries:
This variables allows the
administrator to move (redefine) Syslogd2's spooling
directory from its default location of /var/spool/syslog.
All created (temporary) spool-files will be stored in this directory
so it needs to be large enough to support the numbers and
MaxSpoolSize of any
spool-files you configure. This directory is not automatically
created by Syslogd2,
and must be created by the administrator. It must reside on a
writable disk partition. Only the root user needs permission to
write to this directory as there are no user-maintainance or access
requirements for any file in this directory.
Refer to external documentation
for details on how the spooling feature works in Syslogd2.
This is currently a placeholder
and a potential growth direction for Syslogd2.
The only currently-permissible value is 0 (zero) which 'selects'
'stop polling'. When
the spool-file gets 'full', spooling for that destination is disabled
until the connection is restored and the existing spool-file data is
flushed.
This value is only valid when the
CAP_SPOOLFILES feature set is selected at compile-time.
Refer to external documentation
for details on how the spooling feature works in Syslogd2.
StatInterval specifies the update
interval for Syslogd2 to
collect performance stats in an attempt to provide feedback for
capacity analysis and tuning of parameters for peak performance. The
gathering (and reporting) of statistics will be disabled if this
value is set to 0 (zero).
This feature is still incomplete and
very experimental as of this writing.
This value is only valid when the
CAP_STATS feature set is selected at compile-time.
Refer to external documentation
for details on how the current status of the statistics feature in
Syslogd2.
This is actually a traditional
syslog parameter that Syslogd2
exposes to administrative control.
UserFacility (default:
user.notice) is the
default facility.priority
value assigned to all non-kernel-originated events that do not have
assigned facility and/or priority values when received by Syslgod2.
UserLines configures the
number of messages that may be stored in the optional user
thread-pool's FIFO queue for
processing. (The number of 'message-slots' to allocate in the FIFO.)
This value may be over-ridden by the --ThreadMaps
command but there are no individual output options to do so.
Refer to external documentation
for details on the user thread-pool
and how thread-pools work in Syslogd2.
UserWorkers configures the
number of threads to be allocated in the optional User
thread-pool. This value may be
over-ridden by the --ThreadMaps
command but there are no individual output options to do so.
Refer to external documentation
for details on the user thread-pool
and how thread-pools work in Syslogd2.
WorkerLines specifies the
number of lines
(message-slots) to be
allocated for each worker
thread-pool type FIFO queue. This value may be over-ridden by the
--ThreadMaps command.
There is no command-line option
to directly configure worker-threads. Instead, Worker
thread-pools may now be configured via the input-connections that
participate in the input-thread-pools that reference them. (In early
Syslogd2 iterations,
the input threadpool and worker threadpool were actually combined
into a single structure, so putting both input and worker-thread
parameters in the same option-list made sense. [It still does].)
Any input specification that provides values for lines
or workers is actually
specifying minimum values for the worker thread-pool that will handle
the traffic from that input-source. The responsible worker
thread-pool will be specified by the input-connection's queueid
parameter. Input connection options may increase the worker-thread
values, but not decrease them.
Refer to external documentation
for details on the worker thread-pool
and how thread-pools work in Syslogd2.
Workers defines the number
of threads to be allocated for each worker thread-pool
type. This value may be increased (never decreased) by the
--ThreadMaps command
for individual threadpools.
There is no command-line option
to directly configure worker-threads. Instead, Worker
thread-pools may now be configured via the input-connections that
participate in the input-thread-pools that reference them. (In early
Syslogd2 iterations,
the input threadpool and worker threadpool were actually combined
into a single structure, so putting both input and worker-thread
parameters in the same option-list made sense. [It still does].)
Any input specification that provides values for lines
or workers is actually
specifying minimum values for the worker thread-pool that will handle
the traffic from that input-source. The responsible worker
thread-pool will be specified by the input-connection's queueid
parameter. Input connection options may increase the worker-thread
values, but not decrease them.
Refer to external documentation
for details on the worker thread-pool
and how thread-pools work in Syslogd2.
All input commands use the same general syntax:
--[keyword] = location, comma-separated-opition-list
Syslogd2 supports the use
of two variables ($h and $d) that specify the name of the system-host
(and it's associated external IP address as determined by DNS or
name-cache lookujp). The expression “$h.$d” expands to
“HostName.DomainName”
as defined by the global variables HostName
and DomainName. Using
these variables, it is now possible to specify generically (for all
hosts), the specific external address assigned to its primary
external interface. Depending on host-name conventions (or
cache-file contents), the expression “$h.$d2” may define a
secondary host-name (associated with a secondary network interface)
that can be resolved via DNS or name-cache (/etc/hosts) lookup.
All keywords in Syslogd2 are case-non-sensitive.
Single-letter options and system-values (filenames, function-names,
etc) are the exceptions and are case-sensitive.
All input-confirmation boolean options
support the optional boolean parameters
of their global peers.
All (current) input sources to Syslogd2 are defined via the command-line options --input, --kernel, --pipe, --journal
and --tailfile.
All input methods are covered in depth on the Syslogd2 Input Processing page.
|
Option (Aliases) DEPENDENCIES |
Value-type [default value] |
Comments |
|---|---|---|
| Identity Options | ||
|
datagram UDP, d, U) |
Boolean [default: enabled] |
This is the inverse of the stream
option above. It is the default value. This keyword is provided
for completeness and readability purposes. The datagram option is only valid for socket input - either IP or Linux sockets. |
|
port (p) CAP_STREAMIN for TCP |
String or Integer [514 for UDP] [none for TCP] |
Applies only to IP socket declarations (--input command). TCP is not supported unless CAP_STREAMIN was declared at compile-time. |
|
stream (s, TCP, T) CAP_STREAMIN |
Boolean [default: disabled] |
This option is only valid for socket input - either IP or Linux sockets. |
|
version (ver) |
Keyword: 4 | 6 | 46 [default: 46] |
Specifies the address families that should be
supported by this socket. Applies only to IP socket declarations
(--input command). This option may be automatically limited by Syslogd2 if one or both address families are not configured on an interface or if one or both address-families have been administratively disabled (--disable IPv4, IPv6). This option will be automatically limited if an IP-address is provided (limited to the address-family of the provided address). |
| Options Concerned with Input-Data Format | ||
|
NoForcePrintable (nfp) |
Boolean [default: inherited] | This is the same as the global NoForcePrintable boolean value except it is applied to an individual connection-definition to allow this definition to over-ride the global value for this one connection. |
|
Kernel (k) CAP_KERNELTHREADS |
Boolean [default: disabled] |
This option is set automatically by --kernel commands. This flag designates input from this source as coming from 'a' kernel (ie: any messages without a specified facility/priority should default to KernelFacility instead of UserFacility). |
|
NoHeader (nh) |
Boolean [default: disabled] |
Syslogd2 defines a message-header as a combination of the time-field and the host-field. If you know these fields to be missing from your input, you can save the processing cycles that Syslogd2 would otherwise spend attempting to parse the missing fields by using this option. NoHeader should be considered when receiving traffic from hardware network devices, (most) firewalls, and non-relay syslog host-systems (where all traffic is locally-generated). If you see multiple-time-stamps+host-fields in the output, removal of this option will cause Syslogd2 to parse & use the pre-exising header information. |
| NoHost |
Boolean [default: disabled] |
This option informs Syslogd2 that incoming traffic contains a time-field but no host-field. Normally when a time-field is recognized the next 'word' is blindly accepted as the host-field. Tihs option tells Syslogd2 to use the global HostName.DomainName values instead of attempting to parse what is not there. This option has already been identified as required for proper parsing of systemd-relayed traffic and MariaDB log files. |
|
Poll (Polling) CAP_KERNELTHREADS |
Boolean [default: disabled] |
This option applies to --kernel definitions only (local-kernel input on Linux hosts). This switches the Kernel thread-pool from using system-calls to directly read the kernel syslog buffer to using a polling method to read KLogFile (by default: /proc/kmsg unless altered by administrative setting). |
|
TraceFilter CAP_TRACEFILTER |
Filename [default: disabled] |
This option may be configured for any input type.
It is configured similar to filters using the syntax 'tracefilter=filename'
in the input-option list. The filename follows the same guidelines as for filters.
The contents of this file differ substantially, however. (See the included sample file or the
discussion of CAP_TRACEFILTER). Unlike input-filters and output-filters that are executed in processing and output phases of execution, tracefilters are processed in the input phase as soon as each new data-string is read. This is to retain the sequencing of incoming strings which is the basis for the tracefilter functionality. |
| Processing Options that Apply to Individual Input-Definitions | ||
|
Append CAP_TAILFILES - or - CAP_KERNELTHREADS |
Boolean [default: disabled] |
Applies to --tailfile and --kernel (in polling mode) only: The default action when Syslogd2 starts up is to read the applicable input file from the start of the file. This option causes Syslogd2 to open the file for reading at the current end-of-file (ignoring any preexisting file-content). |
|
Command (c) CAP_STREAMIN - and - CAP_COMMAND - and - CAP_HOUSEKEEPING |
Boolean [default: disabled] |
This option is only valid on streaming Linux sockets. It designates the current input-definition as a valid connection by the interactive command-line tool (tsucX). Refer to external documentation on the tools subdirectory for more details. |
|
Cache (NameCache) CAP_CACHE |
Boolean [default: inherited] |
This is the same as the global NameCache
boolean value except it is applied to an individual
connection-definition to allow this definition to over-ride the
global value for this one connection. When Cache is enabled on an input connection, all incoming messages through that socket will attempt name-resolution using the internal name-cache before attempting to contact the DNS server (if DNS is also enabled). For non-IP-sockets, this setting will act as a 'host-name-normalization step to 'resolve' host-name aliases to canonical names, thereby providing more accurate and consistent host- and network-management results. If a host-name is not found in the Cache, but IS found by a DNS call, it is added to the internal NameCache. When Cache is disabled, it is not used for through-traffic name-resolution. If a (complete) cache-file is supplied and NameCache is enabled, Syslogd2 should never need to request information from the DNS server for name-resolution. |
| DNS |
Boolean [default: inherited] |
This is the same as the global DNS boolean value except it is applied to an individual connection-definition to allow this definition to over-ride the global value for this one connection. |
|
Facility (Fac) |
String or Numeric [default: inherited] |
This option changes the default facility
(or facility.priority) for this one conneciton-definition,
over-riding the global KernelFacility/UserFacility
values (whichever is applicable). If the '.priority' component is not provided, only the facility is over-ridden and the global-setting's prioirty is used. This option only comes into play when a message is received over this input without an embedded facility/priority field or when the ignore input option is set to include 'facility'. |
|
Filter (f) CAP_FILTERSIN |
A filename. [default: none] |
This option specifies the file containing the
input-filter to apply to all traffic received by this
input-source. An absolute filename will be used as provided. A relative filename is relative to ConfigDir. |
|
HostName (Host) |
String |
This is the host-name that will be used as the
'source-host' for traffic received by this interface if there is no incoming
host-field component or if the incoming host field is to be ignored (NoHeader or NoHost may be in
use). Another way to understand this option is as a 'default host' for use when there is no (usable) hostname in the data. If this value is unset and the incoming data has no identifiable host-field, the default will be the local host-name as given by the global variables HostName.DomainName. |
| Ignore |
Semi-colon-separated list of keywords. [default: empty-list] |
Supported keywords are facility, priority,
and hostname. This option ignores these fields if set in incoming traffic, If these field are ignored, the input options facility, priority, and hostname (or their global defaults) will then be applied. |
|
Priority (Pri) |
String or Numeric [default: inherited] |
This option changes the default priority
(or facility.priority) for this one conneciton-definition,
over-riding the global KernelFacility/UserFacility
values (whichever is applicable). If the 'facility' component is not provided, only the priority is over-ridden and the global-setting's facility is used. This option only comes into play when a message is received over this input without an embedded facility/priority field or when the ignore input option is set to include 'priority'. If the ignore option-list includes both facility and priority or if the priority field is entirely missing in the input string, either the facility or the priority option may specify both components as in the 2nd example below. --input = … , priority=warning --input = … , priority=extra3.warning |
| Thread-Pool Selection and Configuration | ||
|
Id [eye-dee] (i) [eye] |
Non-Negative Integer [default 0] (zero) |
Selects which thread-pool this input-source
should be a member of. If the thread-pool does not yet exist, it
is created using global default values. Ignored when used with the --kenrel command-line option because is never more than one of the 'special' Kernel thread-pools. For --kernel commands, this is silently set to 0 (zero).) |
|
Lines (l) [ell] CAP_WORKERTHREADS |
Positive Integer | This specifies the minimum number of 'lines' in the FIFO queue of worker-thread-pool identified by the queueid parameter. This value can raise the currently-assigned number of FIFO-queue-lines, but cannot decrease it. |
|
LossLess CAP_WORKERTHREADS CAP_OUTPUTTHREADS --or-- CAP_USERTHREADS |
Boolean |
The input-queue for this threadpool
is to operate in 'lossless' mode (as opposed to the 'default' or 'lossy' mode).
When an attempt is made to queue a message to a queue that is currently full,
'lossy-mode' queuing will quietly discard the message as 'overflow'. 'Lossless-mode' queuing will block the new addition (locking the inserting thread in the process) until space is made available by the 'dequeuing' threads. The 'LossLess' keyword is only valid in the --ThreadMaps command. |
|
PollTime (PollInterval) CAP_TAILFILES - or - CAP_KERNELTHREADS |
Positive Integer time in seconds) [default: inherited] |
This option may shorten (but never increase) the current calculated polling interval for the input-thread-pool to which this definition is assigned. This is the per-instance over-ride of the --ThreadMaps setting which is the global per-thread-pool over-ride of the global --defaults setting that applies to all input-type (or Kernel type) thread-pools built by Syslogd2. |
|
Queueid (Queue, q) CAP_WORKERTHREADS |
Non-Negative Integer [default 0] (zero) |
This option specifies the worker-thread-pool
identifier that will process the data from this
input-type-thread-pool. If the worker-thread-pool does not yet exist, it will be created with default values. |
|
Readers (r) |
Positive Integer | This specifies the minimum number of threads to be created in the thread-pool identified by the id parameter (the thread-pool this input definition is assigned to). This value can raise the currently-assigned number of threads, but cannot decrease it. |
|
Workers (w) CAP_WORKERTHREADS |
Positive Integer | This specifies the minimum number of threads to be created in the worker-thread-pool identified by the queueid parameter. This value can raise the currently-assigned number of threads, but cannot decrease it. |
| Other Options | ||
|
Uid (u) |
String or Integer [default 0] (root) |
For Linux sockets only, this is the id or name of the user that the filesystem will consider to 'own' the Linux-socket entry. |
|
Gid (g) |
String or Integer [default 0] (root) |
For Linux sockets only, this is the id or name of the group that the filesystem will use as 'group-owner' of the Linux-socket entry. |
|
Mode (m) |
3-digit Octal Int [default 666] |
For Linux sockets only, this is the file-permissions Syslogd2 will assign to the Linux-socket filesystem entry. |
|
Network (net) |
Semi-colon-separated list of network-state
keywords. [default: any] |
Sets the network-states in which this
connection is valid. If the network is detected in a state for
which this connection is not valid, this connection will be
automatically disabled. Syslogd2 already knows that IP connections are not valid when the IP network is down. It also knows the difference between a loop-back address and all other addresses (the difference between network-states Local and Other. Non-IP inputs are assumed to be valid in all network-states (Down, Local, Other). The special network-state Any is a shortcut for 'all network-states whether known or unknown'. |
When Syslogd2 is polling a file for input, it normally opens the file on start-up at the beginning of the file and reads the entire file-contents before pausing at the end of the file to wait for more input. This option tells Syslogd2 to open the file and start reading from the END of the file instead. This option is most likely to be used for large files or where Syslogd2 is restarted frequently and would be in danger of transmitting the same (outdated) data multiple times (once for each time it is restarted).
This is a local (per-input-connection setting) to enable or disable the use of the internal name-cache for host-name-resolution. The internal name-cache serves multiple purposes:
Refer to external documentation on the CAP_CACHE feature-set for more information on creating and using a cach-file.
This option is the result of
another experiment. The goal of this experiment was to find a way to
monitor and interactively reconfigure (in small ways at first) an
operating instance of Syslogd2
in order to avoid the loss of buffered data that always results from
restarting the daemon.
In the 'tools' directory of the
main build directory at compilation time, there will be a series of
binaries of the name tsucX where 'X' is the same suffix appended to
the string syslogd2 to
get the full name of the executable (syslogd2't', syslogd2's', etc).
The 'tsuc' stands for 'Transmit Stream Unix Command'
one instance for (and corresponding to) each instance of the main
binary.
This 'tsuc' tool is used to connect to
a designated 'command-socket' to interact with the running system.
Actions are currently limited (the experiment is still in an
embryonic phase), but currently includes a complete configuration
display (containing file-descriptors for open connections), including
a rudimentary performance statistics display, and the ability to
review and reset the programmable interrupts.
One goal of this experiment is to see
what kind of (if any) feedback I get from people that try the feature
to see whether the experiment is worth continuing.
Refer to external documentation on the
command-utility for more details and explicit instructions on use.
Datagram (UDP) is the default
socket-type protocol for syslog. It is the only socket-type
supported unless either CAP_STREAMIN (for input sockets) or
CAP_STREAMOUT (for output sockets) is declared at compile-time.
Because datagram (UDP) is the
default, there is no need to specify this option unless you
specifically want the configuration-file to state the default for
readability purposes.
If CAP_STREAMIN was declared at
compile-time, the stream
option is available as the non-default choice to create streaming
Linux input sockets or input TCP/IP sockets.
This option (especially when combined with boolean options) is a per-connection over-ride to the global DNS variable setting. The default value is whatever the global DNS setting has been configured to.
Though it may be uncommon,
Syslogd2 is expected
to accept an IP connection (either UDP or TCP) that transmits nothing
but a data-string as a 'syslog' message. Such a connection would
contain no facility/priority field, no time-date field and no
host-string.
For such an eventuality as well
as when reading kernel input and text log files that may not contain
the '<' + facpri-value + '>' sequence indicating a
facility-priority value exists at the start of the message, syslog
processors implement a global default value to use when the
facility/priority information is missing (Syslogd2's
KernelFaciltiy and UserFacility
global variables).
This facility
option is the per-input-source over-ride to the aforementioned global
values. The value to facility
may be expressed as facility=extra1
or as facility=extra1.warning
or as the numeric variants facility=25
or facility=25.4 (but
who would really want to use the numeric versions?)
In the case of a value being a partial
expression (the former example), only the facility portion of the
global value is over-ridden. The global priority component
('.notice') would still be the default for the message's priority
field. In the second instance, the entire global variable (in this
case UserFacility [user.notice]) is over-ridden, resulting in a
facility/priority of 'extra1.warning' for incoming traffic with no
specified value of its own.
For all 'properly-formatted'
syslog messages that contain a valid syslog-priority-value, the
facility option's value does not get used. (See also the ignore
input-option.)
This option is used to specify an
input-filter to apply to all data that enters this connection. An
absolute filename is acceptable. A relative file-name (preferred) is
converted to an absolute file-name relative to the ConfigDir
input-directory setting. Sub-directories within ConfigDir
may be specified for file-management purposes by using a
relative-filename syntax such as filter=filterdir/filterFilename.
Instructions on how to create filter-files in Syslog2.
This option applies to Linux input-sockets only. It is part of an attempt to improve security by using file-system permission-bits to try to control who gets to write to which Syslogd2 Linux-input sockets. This is the group-ownership component of 3 settings (Uid, Gid, Mode) with default values of (0, 0, 666). I am open to feedback from users as to whether or not to keep this option.
The idea for this option was
'born' while conceptualizing about tailfile
input. I belatedly realized that it also may apply to Linux sockets
as well (but not so much IP sockets however).
The idea is that it is common for
network- and host-management systems today to write scripts to try to
monitor log files or to receive data from dedicated devices to create
performance or error logs. If the results of those log files are to
be useful (and especially if more than one log may exist on the same
syslog-reporting platform), down-stream processors need a way to tell
the output of one log-file from another.
What the hostname
option does is to set the value of the host-name-field for any record
that does not specify a hostname in the data. The default value is
to combine the global 'HostName.DomainName' variables into a
Fully-Qualified-Domain-Name (FQDN) for down-stream reporting, but if
multiple log files from the same box are being reported, it seems the
results wold be jumbled unintelligible mess unless each file could
'become' a unique host.
With that last thought in mind,
the hostname field
also supports 2 replacement-strings ($H) and ($D) representing the
global HostName and DomainName variables respectively. This makes
the following syntax valid and easy-to-read (Note the 'H' and 'D'
following the dollar-sign are case-non-senstive):
# ~ --tailfile = <mariadb log file>, nohost, hostname = mariadb.$H.$D
The down-stream processor now receives syslog traffic from a virtual host named 'mariadb.<HostName>.<DomainName> based on the name of the host running the database. The 'base hostname' is derived from the system's hostname on each different host that uses this configuration file. (See also the ignore command for ignoring facility/priority/hostname fields that may be present in the data thereby forcing the use of default values to 'fill in' for the now 'missing' [ignored] values.)
This thread-pool-identifier specifies which input thread-pool this particular input specification should belong to. The default if not provided is 0 (zero). If the input thread-pool has not yet been created, merely specifying an unused identifier will cause it to be created with using global-default values. The range of values for this integer is the range of non-negative 32-bit (or 64-bit) integers. There is no need for thread-pool-identification numbers to be sequential.
The parameter to this option is a semi-colon-separated list of
keywords: facility, priority, hostname.
This input option tells the Syslogd2
message-parser to ignore the values of whichever fields are listed in
the parameter-list. Because these fields are ignored, even if
present, they will be set to their system (or per-connection)
'default' values as described under each of the field-descriptions in
this section.
Using a combination of
ignore=facility and
facility=extra1 has
the result of all over-riding only the facility component of incoming
traffic on this connection. Regardless of what facility the traffic
was originally destined for, it will now be sent to 'extra1' and will
retain its original (non-ignored) priority value. Setting up a
specific address on a Syslogd2
host in this manner is a good way to accept network-device syslog as
input. It does not matter what facility the external device is
configured to send to as long as the address is correct.
Simultaneously, all network-input is kept separate from local host
logging and reporting, allowing for clean Syslogd2
sorting and forwarding decisions using filters and Syslogd2
output options.
Likewise, using a combination of
ignore=hostname and hostname=... allows either Linux-connected input
pre-processors or log-file data to over-ride any host-names in
incoming data and to replace them with either real or virtual
host-names depending on the capabilities and desires of your overall
network-management system. Conceivably, you could even create a
standard where log-file issues are routinely reported as
<application-name>.$h.$d, then on a centralized
Syslogd2-equipped host, use that virtual host-name in an
input-filter to sort the application log-data into different log
files for application teams to review and monitor.
This boolean option flag simply identifies data received by this
connection as 'kernel-data' as opposed to 'user-data'. This means it
will use the KernelFacility
global variable for missing facility/priority values instead of the
UserFacility value.
I wish to allow the option to
monitor multiple kernel files (possibly NFS-mounted) from UNIX hosts
if so desired. There is nothing I know of in the code of Syslogd2
that would prevent monitoring files on NFS-mounted drives -- that
should include remote-mounted kernel log-files.
When the CAP_WORKERTHREADS feature-set is declared at
compile-time, three additional input options become available
(workers, lines and queueId) to select (or create) and
configure the worker-thread-pool that will process the data from this
input-thread-pool.
Lines is the minimum number of
FIFO-queue-lines to allocate in the worker-thread-pool identified by
the queueId option. The
lines parameter can increase
(but never decrease) the cumulatively-calculated number of lines
determined as the maximum of the global variable workerLines,
the --threadmaps definition
for the specified worker-thread-pool (if any) and all input
specifications that belong to any input thread-pool that 'feeds' that
worker-thread-pool.
The lossless option applies to the --ThreadMaps command only (and then only to threadpool types of
worker, output or user).
It changes the operating mode of the FIFO queue associated with the designated --ThreadMaps section from the default of
'lossy' to 'lossless'.
When a FIFO queue is in default ('lossy') mode, any attempt to add a new entry to a full queue will result in that entry being
immdediately discarded (silently due to the usual very high quantity of messages wich could also be being discarded), which allows the calling thread to return to its work.
When a FIFO queue is in 'lossless' mode, the calling thread is blocked until space is made available in the queue for the message to be stored.
Only after the message is stored is the calling thread allowed to return to its normal processing. This is discussed in more detail in the page on queueing.
This option applies to Linux input-sockets only. It is part of an attempt to improve security by using file-system permission-bits to try to control who gets to write to which Syslogd2 Linux-input sockets. This is the file-mode (permission-bits) component of 3 settings (Uid, Gid, Mode) with default values of (0, 0, 666). I am open to feedback from users as to whether or not to keep this option.
This is a component of the network-awareness
experiment mentioned earlier in this document.
There is a separate
input-connection option for network
because while external network-states such as “work”, “home”,
“hotel”, “gym”, etc are not yet implemented, there are 3
network-states that very much affect the creation of a failure-free
configuration (Down, Local
and Other). There is
a separate output-option
for network as well --
for many of the same reasons as the input-option exists.
In addition to its participation in the experiment however, the
network option has real-world value in Syslogd2
based solely on the level of 'network-awareness' already in place in
Syslogd2. Finding/knowing the network-state is used for the
delayed-resolution and connection-recovery features that are built
into the core Syslogd2
code.
This option allows you to specify
one or more specific network-states (Down, Local, Other
or Any) for each input-connection. The default value if not specified is Any (meaning
'all of the above').
Syslogd2 support for
individualized input sockets on different host interfaces means that
a MUCH greater chance of port or address conflicts exists between
multiple interfaces. To help compensate for this, Syslogd2
does significant error-checking during the build phase
(and rebuild phases) of startup.
To properly detect IP issues,
Syslogd2 must know the
current state of the network. (For example, it is not possible to
access a remote DNS server to resolve input host-names or output
destinations if the network-state is down
or local.)
An address of [*, 514, udp] (the default IP address) conflicts with a
specification of [$h.$d, 514, udp] when both are present and the
network state is other.
If the network state is local,
the “$h.$d” address is in conflict with the network-state and is
therefore invalid while [*, 514, udp] remains valid because of the
loopback interface. (Again, local
mode is when only the loopback interface is up).
If the network-state is down,
the entire network-stack is down so “who cares” ?
As you can see, the network-state makes
a difference in whether address conflicts between different input
addresses may exist.
Syslogd2's 'delayed-resolution'
and 'connection-resolution' features are based on knowing the
network-state and only attempting to resolve external IP addresses
(not the loopback addresses [127.0.0.1 or ::1] or the '*' wildcard
address) when the network-state is other.
To do this, it relies on the CheckSources routine
to run RebuildSources when
the network-state changes in an upwards direction. the RebuildSources
routine attempts to re-resolve any failed resolutions and then
deconflicts all inputs based on the current network-state and on
knowledge of which addresses are valid in which network-state.
As it turns out, the following
structure is particularly problematic for wireless-connected laptops:
Line N : # ~ --input = $h.$d, filter=filterFile1
Line N + 1: # ~ --input = *, filter=filterFile3
Line N + 2: # ~ --input = localhost, filter=filterFile2
When the network-state is local,
line N is in conflict, line N + 1 is valid (and first seen), line N +
2 is in conflict with line N + 1 (and is ignored).
When the network-state is other,
line N is valid (and first seen), line N + 1 is in conflict with line
N (and is ignored), line N + 2 now has no conflict and is valid.
Syslogd2's
implementation considers this and other situations during both the
initial Build and the
Rebuild processing in
order to produce 'clean' operational configurations under all
anticipated circumstances.
Deconflicting IP addresses turns
out to be one of the most complex processes in Syslogd2
precisely BECAUSE of network-state and IP-address-validity concerns
across multiple interfaces and address-families.
The experimental
part of the 'network experiment' actually has more to do with
identifying and acting on the network-state when it may be names like
“home”, “work”, “jim” or “sally” and automatically
implementing or suspending potentially different connections
accordingly.
This is the per-connection over-ride setting for the global boolean value of the same name.
This option informs Syslogd2
that incoming data to this connection has no time-field or host-field
information and that it should not attempt to find either. Incoming
data may or may not have facility/priority-field information as this
option does not interact with that msg-field.
Syslogd2 will immediately
use the system-clock (time-of-receipt) for the time-field and the
incoming IP address (or global default values of HostName.DomainName
for non-IP connections).
This is a common format for hardware
network-devices and for LInux/Unix hosts that generate (but do not
receive) IP traffic to an external syslog collector.
This option is mutually-exclusive with
NoHost.
This option informs Syslogd2
that after it parses the time-field, it should not look for or assume
that the next token is a host-name-field. Instead, Syslogd2
should use it's (default) host-name-field value for this connection's
input-type (for IP that is the IP address, for non-IP, it is the
global HostName.DomainName
values).
This option is mutually-exclusive with NoHeader.
This boolean option switches the kernel input thread-pool (local-kernel-input for Linux systems only) from using system-calls to directly read the kernel's syslog buffer (meaning /proc/ does not need to be mounted to access the /proc/kmsg file) to using a file-polling algorithm reading from the global variable KLogFile (/proc/kmsg) that represents the file to be read. The default is disabled (use system-calls)
This is the per-input-connection over-ride of the global variable of the same name. This option may cause the tailfile-thread-pool this connection becomes a part of to decrease its polling interval, but never to increase it.
This is an identity option for IP sockets. For UDP, it can be omitted and will then take the value of 514. This option is mandatory for all TCP/IP input sockets because there is no official global syslog TCP port and therefore no default value for TCP input connections.
This option is the same as for the Facility option above except that it focuses on the priority component rather than on the facility component. All other aspects of operation are identical.
When the CAP_WORKERTHREADS feature-set is declared at
compile-time, three additional input options become available
(workers, lines and queueId) to select (or create) and
configure the worker-thread-pool that will process the data from this
input-thread-pool.
QueueId is the thread-pool
identifier of the worker-thread-pool that this input connection will
help 'feed'. QueueId
is actually an input-threadpool-level value that is latched so it can
only set once. An input-thread-pool's queueId
value specifies the thread-pool identifier of the worker-thread-pool that will process the data read
by that input-thread-pool.
The default value of queueId
is always 0 (zero).
The range for queueId
is all non-negative integers supported by the system. This is a
different range than for input-thread-pools (that is:
input-thread-pool 3 is a different thread-pool than
worker-thread-pool 3 and the two need not necessarily have any
relationship to each other).
Multiple input-thread-pools may 'share'
a single worker-thread-pool.
This is the minimum number of threads to be assigned to the input thread-pool that this connection will become a part of. This option may increase the thread-count, but may never decrease it.
This boolean value causes this connection to be a streaming socket (either a streaming Linux socket or a TCP/IP socket). Valid for socket inputs only (--input keyword).
This option applies to all input types. It identifies an external file
containing tracefilter specifications. There are two types of filter-lines in
a tracefilter file (in addition to blank lines and comments [beginning with '#']
that are ignored).
The first line-type consits of 's' + a delimited string separated by whitespace.
The second line type consits of 'e' + a delimited string separated by whitespace.
As in filters, the delimtie is the selected individually for each line and identified as
the first character after the first whitespace after the 'field-type' identifier. There
may be multiple instances of either 's' or 'e' lines or both. ('s' lines identify
'start'-strings to be discarded while 'e' strings identify 'end'-strings. For example,
kernel trace information can be ignored (discarded) using a tracefilter file containing:
s /------------[ cut here ]------------/
e /---[ end trace 0000000000000000 ]---/
This option applies to Linux input-sockets only. It is part of an attempt to improve security by using file-system permission-bits to try to control who gets to write to which Syslogd2 Linux-input sockets. This is the ownership component of 3 settings (Uid, Gid, Mode) with default values of (0, 0, 666). I am open to feedback from users as to whether or not to keep this option.
This option sets the user's address-family preferences for this
IP-socket connection. Valid entries are '4', '6' and '46'. If
omitted, '46' is assumed.
These are preferences, (requests -- not demand)s. Several reasons
may exist for disallowing or ignoring user preferences (the
address-family requested may have been administratively disabled or
not configured onto any interface, the entire inet subsystem may not
have been enabled, the IP-address entered is not compatible with the
user's requested address-family, etc)
This option is primarily used when using the wild-card address
('*') and multiple address-families are configured on the interface
and enabled in Syslogd2 or
when a host-name is provided that resolves to addresses in both
address-families and both address-families are configured on one or
more external interfaces.
When the CAP_WORKERTHREADS feature-set is declared at
compile-time, three additional input options become available
(workers, lines and queueId) to select (or create) and
configure the worker-thread-pool that will process the data from this
input-thread-pool.
Workers is the minimum
number of threads to allocate in the worker-thread-pool identified by
the queueId option.
The workers parameter
can increase (but never decrease) the cumulatively-calculated number
of threads determined as the maximum of the global variable workers,
the --threadmaps
definition for the specified worker-thread-pool (if any) and all
input specifications that belong to any input thread-pool that
'feeds' that worker-thread-pool.
The general form of a Syslogd2 output line is an extension of the traditional output-line:
<selector-string> [-] <destination> [, comma-separated-option-list] # Inline-Comment (with SoftComment disabled)
-- OR --
[#] <selector-string> [-] <destination> [, comma-separated-option-list] ## Inline-Comment (with SoftComment enabled)
Note: Syslogd2 defines a 'soft' comment as configuration text that is found between the first hashtag in a line and the 2nd hashtag.
Conceptually, Syslogd2
divides each output-line into 4 distinct components:
Syslogd2 supports the standard
syslog data-types using the standard syntax and type-indicators. The socket ('@') and 'file' types are expanded.
Output options are specified as a comma-separated list
following the destination component of the output line. Though they
are configured as a single set of options, some options 'belong' to
the selector-string component and others 'belong' to the destination
component of the output line.
Note: The 'Affiliation' reference in the following table indicates whether the option is 'affiliated' with the selector-string or the location.
|
Option-Name, (Alias-Names) |
Parameter Type [A: <affiliation>] [default value] |
Comments |
|---|---|---|
| Threadpool Options (Requires CAP_OUTPUTTHREADS) | ||
|
Id (ThreadPoolId, i [eye]) |
non-negative integer [A: Destination] [default: 0] |
This option selects the output threadpool this destination will belong to (and be serviced by). |
|
Workers, (WorkerThreads, w) |
positive integer [A: Destination] [default: inherited] |
This option allows optional specification of a minimum number of worker-threads to be any output threadpool that supports this destination. It can raise (but never lower) the values inherited from the --defaults option that sets global values or the --threadmaps option that pre-defines individual threadpool parameters. |
|
Lines, (QueueLines) l [ell]) |
positive integer [A: Destination] [default: inherited] |
This option allows optional specification of a minimum number of FIFO-queue lines in any output threadpool that supports this destination. It can raise (but never lower) the values inherited from the --defaults option that sets global values or the --threadmaps option that pre-defines individual threadpool parameters. |
| Spooling Options (Requires CAP_SPOOLFILES) | ||
|
SpoolFile, (sf) |
Boolean (Optional string parameter for a suggested spool-file name.) [A: Selector-String] [A: Destination] [default: False] |
This option declares
connection-spooling is to be active for data selected by this
selector-string. Applies only to streaming sockets (Linux stream
and TCP) and to named pipes. (Sockets require CAP_STREAMOUT) This option also creates a destination-level structure and associated spool-file) to manage the spooling for all selectors that 'belong' to this destination. May also (optionally) suggest a spool-file name to be associated with this definition. If no name is given or comes into conflict, an automatic name will be generated. Spool files should not be user-visible or user-accessible. They live in SpoolDir -- a directory users should not have access to. Spooling requires the configuration of the global SpoolDir variable to define the working spool-file directory. |
| Spool |
Boolean [A: Selector-String] [default: False] |
This option declares network-state
spooling is to be enabled for data selected by this
selector-string. Applies to any socket or named-pipe. Whenever
the network is not in 'valid' state, data will be spooled. When
the network is in a valid state, any spooled data is flushed. If 'SpoolFile' is not also selected, all spoolfile names will be auto-generated. |
|
SpoolFileMaxSize, SpoolFileMax (sfmax) |
Size-string [A: Destination] [default: inherited] |
This is the minimum size of a spool-file at which some type of action will be taken. (The only current action is to stop spooling). |
|
SpoolFileAction (sfa) |
Numeric value [A: Destination] [default: 0] |
Currently, only one action is supported: action 0 (zero) which is to stop spooling. If any other action is provided (or if omitted), this value will be forced to zero. |
| SpoolFileAge |
Time-String [A: Selector-String] [Default: inherited] |
This value is inherited from the
global value set by the --defaults
command-line parameter if not provided. This value specifies the
'age' after which a spooled event is to be discarded rather than
forwarded to the ultimate destination once a connection (or
network-state) is restored. Because this value is based on the data-content of a selector-string, its value may differ between selector-strings assigned to the same destination. |
| Identity Options (Those that distinguish one destination from another) | ||
|
Port (p) |
Positive Integer [A: Destination] [UDP default: 514] [TCP default: <none>] |
Applies to IP connections only.
Specifies the remote IP port to connect to. Must be provided for outbound TCP connections. |
|
Stream (s, TCP, t [tee]) |
Boolean [A: Destination] [default: False] |
Applies to socket connections only
(either IP or Unix). Identifies this connection as using a
streaming protocol (TCP/IP or Unix streaming socket protocol). Requires CAP_STREAMOUT |
|
Datagram (dgram, d, UDP, u [you]) |
Boolean [A: Destination] [default: True] |
This is the default protocol for
syslog traffic. If omitted, all socket connections default to
'datagram'. 'Datagram' is mutually-exclusive with 'stream'. |
| Other Options | ||
|
AllMessages (AllMsgs, a, EveryMsg) |
Boolean [A: Destination] [Default: inherited] |
This value is inherited from the
global (boolean) AllMessages
setting. The connection-level setting allows individual over-ride
of the global value. When enabled, this setting disables the practices of collecting and comparing successive events and the printing of “last message repeated <n> times” when a different messages is received. Instead, each individual messages will be logged in its own right. This option is automatically enabled for all 'remote' connections (all sockets, and pipes) and all character devices. It applies only to local disk files. |
|
FilterFile (Filter) |
String: (filename) [A: Selector-String] [Default: none] |
This specifies the name of an output filter to be applied to data selected by this selector-string. If a relative filename (or relative directory path), it will be relative to ConfigDir (/etc/syslog.d or as changed by the --defaults command-line option). |
| Format |
Semi-colon-separated list of
keywords [A: Destination] [Default: none] |
This allows some limited optional output format variations. If not provided, the 'traditional' syslog output-line formats will be used. This option applies to all destination-types (including user-lists & 'write-all' destinations). See yperlink for list of valid keywords. |
| '-' |
Boolean [A: Destination] [Default: False] |
Applies to files only. This option must PRECEDE the filename on the output line. When present, it means that Syslogd2 should NOT sync the file to disk (waiting until the disk write has been confirmed) after every write. Not setting this option can help safeguard data in unstable situations, but is also detrimental to overall performance. |
|
Gid (g) |
Integer Group-ID or String
Group-Name [A: Destination] [Default: 0 (zero)] |
Applies to files only. Specifies the group-ownership of this output file. |
|
Uid (u) |
Integer User-Id or String User-Name [A: Destination] [Default: 0 (zero)] |
Applies to files only. Specifies the owner of this output file. |
|
Mode (m) |
Octal value [A: Destination] [Default: 600] |
Applies to files only. Specifies the permission-bits of this output file. |
|
Network (Net) |
Semi-colon-separated list of
keywords [A: Selector-String] [Default: 'Any'] |
Keywords are 'down', 'local', 'other' or 'any'. Specifies the network-states in which this destination is considered 'valid'. |
|
Version (Ver) |
Keyword: (4 | 6 | 46) [A: Destination] [Default: 46] |
If a remote IP host is specified as a host-name, this option specifies which address-families it is allowed to use when attempting to establish a data-connection. Does not apply to IP addresses or non-IP destinations. |
As noted in the Forward,
the traditional selector-string in Syslogd2 may be 'chopped up' into
different lines where different selector-string options may be
applied to different facilities and priority levels.
As hinted at in the
Concept Summary,
different types of data have different 'life-times' before they
become obsolete.
Additionally, there is
the problem of 'traffic-control'. In order for Syslogd2
to create the 'file of record' for forensic analysis, all received
data from a given host will normally be logged to disk. Obviously
this much data cannot be allowed onto the network links or congestion
will occur. To send data from multiple sources to a single (central)
host, it will be necessary to apply facility/priority-unique filters
to the output stream (output filters) to discard all output unwanted
at that destination while continuing to log all received data to
local disk. To do this, filters will need to be applied on (perhaps)
a per-facility basis, resulting in yet another reason to 'break up'
the traditional selector-string into mulitple lines.
This is the current list of all Selector-string output options:
This option requires that
CAP_FILTERSOUT has been declared at compile-time.
This option specifies
an output-filter filepath (relative to ConfigDir
if not an absolute filename). The difference between an output filter
and an input filter is when they are applied. An input-filter is
applied after all message prep has been done and just before scanning
the output lists or output threadpools for message delivery. This
allows input filters to 're-route' traffic by changing their
facility/priority values.
An output filter is
applied after a matching selector-string has been found but before
application of --LocalHosts or --StripDomains lists or application of
the 'format' option (below). Output filters are most useful in
discarding traffic that should not
be sent over network links or in final (destination-specific)
modification of messages before transmission (such as to DBD2).
The parameter to this option is a semi-colon-separated list of keywords. This option specifies in which network states you wish a given connection to be valid. Keyword values are:
Syslogd2 will (by default) disable the network state 'Down' for all IP connections. It will also automatically disable the 'Local' state for all IP output-destinations that do not resolve to an address on a local interface.
This option requests
connection-spooling
for messages matching this selector-string segment.
It requires that
CAP_SPOOLING was enabled at compile-time. It applies to pipes and
streaming sockets only. (Streaming socket additionally require
CAP_STREAMOUT to have been enabled at compile-time). Detection of
connection failure relies on the loss of handshake protocol that is
missing in datagram transmissions.
When a connection has
failed and cannot be immediately re-established, Syslogd2
will begin (continue) to write data to a spool-file until either the
connection is restored or the spool-file fills up. When the
connection is restored, the contents of the spool-file are flushed to
the destination (assuming they have not yet exceeded their assigned
age limit).
The age limit (SpoolFileAge)
allows an administrator to specify that once an undelivered message
has reached a certain age, it should be discarded instead of
forwarded.
Connection spooling and
network-state spooling do not conflict with each other. On the
contrary, they complement each other.
This option requests
network-state-spooling
for messages matching this selector-string segment.
It requires that
CAP_SPOOLING was enabled at compile-time. It applies to pipes and all
socket connections. (Streaming socket additionally require
CAP_STREAMOUT to have been enabled at compile-time).
Instead of monitoring the
status of a connection, network-state-spooling checks the state of
the network to see if an attempt at data-transmission should occur or
if data should immediately be spooled to disk for later transmission.
For example:
extra3.* @sallyhost.localdomain, stream, port=1025, network=other, spool, spoolfile…
… tells Syslogd2
that the connection to sallyhost is not valid while the network is
down or while the network is in local (loopback) mode (with no
external connections). However, any data collected from facility
'extra3' that would have otherwise been transmitted during the time
the network is not in a valid state should be spooled to disk. Once
the external interfaces become available (the network reaches the
state of 'other'), transmit the spooled data to sallyhost port 1025
using TCP. If the connection fails, resume spooling to disk until
the connection is restored. (The network specification above is
actually redundant because of default processing, but provided for
clarity).
Connection spooling and
network-state spooling do not conflict with each other. On the
contrary, they complement each other.
Network-state-spooling is
able to support UDP and Linux Datagram connections because it does
not care about the state of the connection -- only about the state of
the network.
This option takes a time-string parameter (30 [seconds], 3m, 2h, etc). It specifies the maximum 'age' after which the message should be discarded if not yet forwarded to the downstream processor.
Contrasting with Selector-string options (that focus on
the differences of data going to a common location),
Destination/Location options control the commonalities
among the data. These include which threadpool the output 'umbrella'
structure (and all Selector-string substructures) will be served by,
the address and port of the (common) connection, whether a spool-file
is active for this destination (and the management of that spool-file
for all selector-strings), and the output-format to be written to the
device (or file).
This is the complete
list of location-options currently supported by Syslogd2:
Note: All Thread-pool options require that CAP_OUTPUTTHREADS has been declared at compile-time. Without CAP_OUTPUTTHREADS, there is no use for these settings as no output thread-pools exist.
This optional integer (if not given) will default to inherited settings (usually thread-pool 0 [zero]). The default threadpool may change however if this output line is in an include-file and the line that included this file contained an --OutputParms parameter-string specifying a different threadpool-id.
Most administrators will likely opt to leave this optional value unset and to configure thread-pools using the more consolidated and centralized --ThreadMaps command-line option
Most administrators will likely opt to leave this optional value unset and to configure thread-pools using the more consolidated and centralized --ThreadMaps command-line option.
Note: All Spooling options require that CAP_SPOOLFILES has been declared at compile-time. Without CAP_SPOOLFILES, there is no use for these settings as no spooling support is implemented.
As an optional parameter to the boolean option 'spoolfile', the user may suggest a name for the spool-file to be created. In case of omission or conflict with the user's name, a spool-file name will be auto-generated.
This is the maximum
size a spool-file is allowed to grow before it becomes eligible for
some SpoolFileAction.
Spool-file sizes are checked each time CheckDestinations is executed.
If the CheckDestinations schedule has been disabled, Syslogd2
provides no other recurring process to detect / validate output
spool-file sizes or to initiate spool-file 'flushing' upon link
restoration.
Each time
CheckDestinations executes, it checks the spool-file size and if the
size exceeds this value, takes the specified SpoolFileAction.
Administrators should leave room for 'growth' between
MaxSpoolFileSize and the maximun tolerable size due to delays between
cycles of CheckDestinations.
This specifies the action to take when a spool-file is detected as 'full'. The default (and only currently supported) action is StopSpooling.
Identity options are those that define a unique location / destination:
This option applies to IP connections only. It specifies the IP port to connect to.
These options apply to
either Linux or IP connections. They are interchangeable with
'TCP/UDP' and specify the type of protocol to use when communicating
with the 'remote end'.
Stream (TCP) protocol
requires the compiler symbol CAP_STREAMOUT to have been defined at
compile-time.
Datagram is the
default protocol for Syslog (and Syslogd2)
and is always enabled by default.
This option is automatically
enabled for all connections that are not files, TTY, Console or PTY
devices.
For Files/TTY/Console/PTY
devices, when this option is disabled, duplicate messages will be
stored and counted until either a non-duplicate message is received
or until a timer expires (FlushIntervals).
Upon expiration of the
timer, the stored message and repeat-count is output to the device
(“Last messae repreated <N> times...”), the counter is
reset to 0 and the process continues.
The reason that pipes,
sockets and user lists are exempted is that users gain no value from
the repeat-counts and pipes and that sockets are expected to be read
by processes that have no way to re-integrate (potentially)
out-of-sequence “Last message...” events with the applicable
repeated message.
This option provides
some user-control over the format of the output line produced by
Syslogd2.
The parameter is a
semi-colon-separated list of keywords that modify the format of an
output line in various ways,
The default output line
format (traditionally) depends on the destination-type:
If the list is non-empty, the output format is further modified by the selected keywords:
The version
option applies to IP connections only. It determines which
address-families Syslogd2
is allowed to use (if not otherwise disabled) to connect to the
remote IP host. (IPv4, IPv6 or both). If omitted, the default is
both.
The parameter value for the version option
is selected from [ 4 | 6 | 46 ].
This 'option' is placed before the filepath in the output file and retains the traditional meaning of “Do not sync this file after every write”.
Sets the owner of the output file. May be numeric or a name-string.
Sets the group owner of the output file. May be numeric or a group-name-string.
Sets the permissions of the output file. The parameter to this option is 3 octal digits [Default: 660].