Skip to content

[improve] PIP-467: Convert pulsar-functions module logging from SLF4J to slog#25508

Merged
merlimat merged 9 commits intoapache:masterfrom
merlimat:slog-functions
Apr 15, 2026
Merged

[improve] PIP-467: Convert pulsar-functions module logging from SLF4J to slog#25508
merlimat merged 9 commits intoapache:masterfrom
merlimat:slog-functions

Conversation

@merlimat
Copy link
Copy Markdown
Contributor

@merlimat merlimat commented Apr 10, 2026

Summary

  • Convert 106 source and test files across all pulsar-functions submodules from SLF4J to slog structured logging
  • Submodules: worker, instance, runtime, utils, java-examples, localrun, api-java
  • Files using context.getLogger() (public API returning org.slf4j.Logger) left unchanged

Test plan

  • compileJava and compileTestJava pass for all submodules
  • checkstyleMain and checkstyleTest pass for all submodules
  • CI passes

Motivation

PIP-467

The slog library classes are now part of the runtime-all shaded jar
since pulsar-functions-api exports slog as an API dependency.
The integration test checks that "TestPayloadProcessor constructor
with configs key=value" appears as a substring in the function logs.
Using infof() ensures the configs string is inlined in the message.
The checkLogs test was checking for the full formatted log message
as a substring. With slog, the config values are structured attrs
rather than inlined in the message. Fix the test to check for the
message and each config key=value pair independently.
The test was checking for the exact Lombok toString() output of
BatchingConfig in the function logs. With slog, the config is
logged as a structured attr, so the exact format differs. Check
for the key fields instead.
The THREAD runtime doesn't produce the function config log line in
the function logs file, so this assertion only makes sense for
PROCESS runtime (which is already where the more detailed checks
were happening).
The assertion was relying on the Lombok @DaTa toString format
produced by SLF4J parameter substitution. With slog, the attr value
is rendered differently and THREAD runtime doesn't produce this log
line at all. The PROCESS-runtime JSON check ("batchingEnabled":false)
already covers this.
In THREAD runtime, function logs are routed to per-function log files
via log4j2 RoutingAppender using `$${ctx:function}` etc, which reads
from the LogEvent's contextData. JavaInstanceRunnable populates these
keys via `ThreadContext.put(...)`, and SLF4J/log4j2 normally injects
the ThreadContext into each event's contextData automatically.

slog's Log4j2Logger builds its own MutableLogEvent and calls
`event.setContextData(buildContextData(...))`, which only includes
slog attrs (or null when there are none). This bypasses log4j2's
ContextDataInjector, so `ctx:function` is not found at routing time
and the log message never reaches the function-specific log file.

The integration test `testJavaExclamationMessagePayloadProcessor`
asserts that TestPayloadProcessor's constructor log appears in the
function logs even in THREAD runtime, which worked before because
`@Slf4j` went through the normal log4j2 path. Keeping the example as
@slf4j restores this behavior without touching slog or the test.
@merlimat merlimat merged commit 840fc34 into apache:master Apr 15, 2026
80 of 82 checks passed
@merlimat merlimat deleted the slog-functions branch April 15, 2026 05:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants