logwisp/doc/formatters.md

Formatters

LogWisp formatters transform log entries before output to sinks.

Formatter Types

Raw Formatter

Outputs the log message as-is with optional newline.

[pipelines.flow.format]
type = "raw"
sanitizer_policy = "raw"
flags = 1

Configuration Options:

Option	Type	Default	Description
add_new_line	bool	true	Append newline to messages
type	string	"raw"	raw, json, or txt
flags	int64	0	log/formatter flags override
sanitizer_policy	string		Sanitizer policy (e.g. "json", "raw", "txt", "shell")

JSON Formatter

Produces structured JSON output.

[pipelines.format]
type = "json"

+[pipelines.flow.format]
 type = "json"
sanitizer_policy = "json"

Output Structure:

{
  "timestamp": "2024-01-01T12:00:00Z",
  "level": "ERROR",
  "source": "app",
  "message": "Connection failed"
}

Text Formatter

Template-based text formatting.

[pipelines.flow.format]
type = "txt"
sanitizer_policy = "txt"
timestamp_format = "2006-01-02T15:04:05.000Z07:00"

Configuration Options:

Option	Type	Default	Description
timestamp_format	string	""	Time format override

Default Template:

[{{.Timestamp | FmtTime}}] [{{.Level | ToUpper}}] {{.Source}} - {{.Message}}{{ if .Fields }} {{.Fields}}{{ end }}

Template Functions

Available functions in text templates:

Function	Description	Example
FmtTime	Format timestamp	{{.Timestamp | FmtTime}}
ToUpper	Convert to uppercase	{{.Level | ToUpper}}
ToLower	Convert to lowercase	{{.Source | ToLower}}
TrimSpace	Remove whitespace	{{.Message | TrimSpace}}

Template Variables

Available variables in templates:

Variable	Type	Description
.Timestamp	time.Time	Entry timestamp
.Level	string	Log level
.Source	string	Source identifier
.Message	string	Log message
.Fields	string	Additional fields (JSON)

Time Format Strings

Common Go time format patterns:

Pattern	Example Output
2006-01-02T15:04:05Z07:00	2024-01-02T15:04:05Z
2006-01-02 15:04:05	2024-01-02 15:04:05
Jan 2 15:04:05	Jan 2 15:04:05
15:04:05.000	15:04:05.123
2006/01/02	2024/01/02

Format Selection

Default Behavior

If no formatter specified:

• HTTP/TCP sinks: JSON format
• Console/File sinks: Raw format
• Client sinks: JSON format

Per-Pipeline Configuration

Each pipeline can have its own formatter:

[[pipelines]]
name = "json-pipeline"
[pipelines.flow.format]
type = "json"

[[pipelines]]
name = "text-pipeline"
[pipelines.flow.format]
type = "txt"

Message Processing

JSON Message Handling

When using JSON formatter with JSON log messages:

1. Attempts to parse message as JSON
2. Merges fields with LogWisp metadata
3. LogWisp fields take precedence
4. Falls back to string if parsing fails

Field Preservation

LogWisp metadata always includes:

• Timestamp (from source or current time)
• Level (detected or default)
• Source (origin identifier)
• Message (original content)

Performance Characteristics

Formatter Performance

Relative performance (fastest to slowest):

1. Raw: Direct passthrough
2. Text: Template execution
3. JSON: Serialization
4. JSON (pretty): Formatted serialization

Optimization Tips

• Use raw format for high throughput
• Cache template compilation (automatic)
• Minimize template complexity
• Avoid pretty JSON in production

Common Configurations

Structured Logging

[pipelines.flow.format]
type = "json"

Human-Readable Logs

[pipelines.flow.format]
type = "txt"
timestamp_format = "15:04:05"