Bug: sys.stderr is used as default logging stream

[bram-tv]

Expected Behaviour

The API documentation of Logger() contains:

stream : sys.stdout, optional
   valid output for a logging stream, by default sys.stdout

This is no longer the case.

Current Behaviour

The sys.stderr stream is used as default logging stream.

Code snippet

from aws_lambda_powertools import Logger

logger = Logger()
logger.info("Example log message")

Possible Solution

No response

Steps to Reproduce

Run the above code but redirect stdout to /dev/null:

$ python3 example.py >/dev/null
{"level":"INFO","location":"<module>:4","message":"Example log message","timestamp":"2023-07-10 16:03:20,348+0000","service":"service_undefined"}
$

Message still shows up, i.e. it's not being logged to stdout.

Run the above code but redirect stderr to /dev/null

$ python3 example.py 2>/dev/null
$ 

Another way to see it is by using strace:

$ strace -s 65536 -e write python3 example.py
write(2, "{\"level\":\"INFO\",\"location\":\"<module>:4\",\"message\":\"Example log message\",\"timestamp\":\"2023-07-10 16:05:46,196+0000\",\"service\":\"service_undefined\"}\n", 146{"level":"INFO","location":"<module>:4","message":"Example log message","timestamp":"2023-07-10 16:05:46,196+0000","service":"service_undefined"}
) = 146
+++ exited with 0 +++
$

It's writing to stderr..

Powertools for AWS Lambda (Python) version

latest

AWS Lambda function runtime

3.10

Packaging format used

PyPi

Debugging logs

This issue was introduced in commit 0dc5e1b.
More specific:
old code:

        self._handler = logging.StreamHandler(stream) if stream is not None else logging.StreamHandler(sys.stdout)

new code:

        self.logger_handler = logger_handler or logging.StreamHandler(stream)

Extra context: by default stream is None and using logging.StreamHandler(None) by default uses stderr.

https://docs.python.org/3.10/library/logging.handlers.html

Returns a new instance of the StreamHandler class. If stream is specified, the instance will use it for logging output; otherwise, sys.stderr will be used.

[heitorlessa]

Thank you @bram-tv for reporting it - looking into it. Out of curiosity, how does that affect you when running on AWS Lambda runtime? Or are you running outside of Lambda?

Thank you

[heitorlessa]

Confirmed, working on a fix soon

[heitorlessa]

Fix is up - kept stream as a private property lemme know if you have an use case that warrant otherwise: #2736

took slightly longer than expected due to a bug in pytest when asserting stderr