Abandoning logging for structlog
The standard library’s
logging package is the default in the Python ecosystem. It works but it isn’t great. Setup is annoying:
import logging logger = logging.getLogger(__name__) # ... logger.info("request method=%s", method)
The default formatting isn’t easy to parse and it doesn’t support structured logs.
structlog is a better choice!
Some of the benefits
nice default formatting
And you can also use
logfmt in prod which most logging providers will parse automatically. Much better than having to look at json logs.
You don’t have to munge a bunch of strings together yourself. No dealing with, “should I use
So instead of:
log.info("processing item itemid=%s from queue request_id=%d", item_id, request_id)
log.info("processing item from queue", itemid=item_id, request_id=request_id)
log = log.bind(item_id=item_id) log.info("processing item from queue")
You can build up context that gets shared with future log calls by calling
.bind with your log params.
log.info("starting up...") user = get_user(request) log = log.bind(user_id=user.id) log.info("fetched user") item = pop_from_queue(request) log = log.bind(item_id=item.id) log.info("processing item") res = process_queue_item(item) if not res.ok: log.warning("failed to process item")
which results in the following when using the
event="starting up..." event="fetched user" user_id=usr_123 event="processing item" item_id=item_123 user_id=usr_123 event="failed to process item" item_id=item_123 user_id=usr_123
Structlog also integrates with
ContextVars and supports setting up
processors to munge your logs before they get written.
structlog for logging in Python.