fix(logging): let debug: true reach stderr, not only the log file

[bakiburakogun]

Fixes #347

Problem

logger_config.yaml pins the stderr handler at WARNING, so with debug: true the detail goes to persistent_storage/logs/ccb.log while docker logs stays nearly silent. logger_config.k8s.yaml already has stderr at DEBUG, so only the manual-install and Docker paths are affected.

Everything describing what indexing is doing is INFO or DEBUG — Dispatching N file chunk(s), Waiting for file chunk 1/8 future to complete, embed_sources finished for 4 source(s): 3 succeeded, 1 errored with its per-source error dict. None of it surfaces. What does surface is the occasional ERROR traceback with no context around it, so from outside the container a healthy-but-slow run and a stuck one look identical.

Change

When debug is on, lower the stderr handler along with the logger levels. Reading the app config a few lines earlier is what makes it possible to decide before dictConfig runs. The default stays WARNING, so nothing changes for anyone who has not asked for debug; the file handler is untouched.

Guarded with .get() so a config without a stderr handler — a custom one, say — still starts.

Testing

On a four-node deployment with debug: true: before the change docker logs showed only startup lines and error tracebacks; after it, the batch dispatch and per-source results appear, which is what made diagnosing #345 possible without going into the container to read ccb.log.

[kyteinsky]

thanks, yeah I suppose we can do this.
although the changes should be done in, they should be changed to DEBUG:

context_chat_backend/logger_config.yaml

Line 32 in 040adab

level: WARNING

context_chat_backend/logger_config.k8s.yaml

Line 40 in 040adab

level: WARNING

[bakiburakogun]

Thanks. Before I move it I wanted to check what the change would mean, because I think it changes the behaviour rather than only relocating it.

_setup_log_levels() returns early when debug is false and does not lower anything, and in logger_config.yaml the ccb logger is already at DEBUG. So raising the stderr handler to DEBUG there would make the detail unconditional rather than tied to debug: true:

	stderr handler	ccb logger	on stderr with debug: false
now	WARNING	DEBUG	warnings only
with the change	DEBUG	DEBUG	debug and warnings

logger_config.k8s.yaml arrives at the same place from the other side: the handler is already at DEBUG there, so moving the ccb logger to DEBUG has the same effect.

Which of the two would you prefer?

• If debug output on stderr should simply be on, I will make the two config changes and drop the main.py part of this PR.
• If it should still follow debug: true, the config changes on their own are not enough. _setup_log_levels() would also need to set the loggers to WARNING when it is false rather than returning early, and I am happy to do that here.

The main.py change goes either way. I would just rather know which of the two you want before pushing.