Quick wins for a faster PC:
Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Repair Windows errors before they cause bigger problemsFix Now →Scan for outdated or missing drivers - takes under a minuteDriver Scan →iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
Whether a Python worker inherits logging handlers depends on how it starts. A process created with fork begins with a copy of the parent’s state, so configured loggers and handlers can be present in the child. With spawn, the worker starts a fresh interpreter and should initialize logging itself. For several workers writing to one destination, give the handlers to one listener and send worker records through a queue.
What “inherit logging handlers” means
Python logging routes records through logger objects and their handlers. A logger can have handlers of its own; with propagation enabled, it can also pass records to handlers on ancestor loggers, including the root logger. This can produce duplicate output if a worker adds a handler while inherited handlers remain active.
| # | Preview | Product | Price | |
|---|---|---|---|---|
| 1 |
|
Well Logging Handbook | $44.61 | Buy on Amazon |
| 2 |
|
Competition Car Data Logging: A Practical Handbook | $26.76 | Buy on Amazon |
| 3 |
|
J. J. Keller 2024 OSHA Construction Safety Handbook, English | $15.44 | Buy on Amazon |
| 4 |
|
Handbook on Well Logging | $59.17 | Buy on Amazon |
| 5 |
|
PRODUCTION & LOGGING HANDBOOKS #4 (FOREST INDUSTRIES) | $10.00 | Buy on Amazon |
Under fork, the child starts with a copy of the parent process state, so logging configuration made before the fork can exist in the child. That is not a guarantee for every process or platform. spawn starts a fresh interpreter, so worker code needs to configure logging. forkserver uses a different process-creation arrangement; diagnose behavior by checking the actual start method rather than assuming inheritance.
Check the start method before changing configuration
Defaults vary by platform and Python version. The CPython multiprocessing documentation says macOS has used spawn by default since Python 3.8. In Python 3.14, the POSIX default changed from fork to forkserver. Applications can select a context explicitly, so inspect the context your program uses.
#1 Best Overall
Reusable libraries should let callers provide their multiprocessing context. The Python multiprocessing documentation advises: “Libraries using multiprocessing or ProcessPoolExecutor should be designed to allow their users to provide their own multiprocessing context.” Objects such as locks created by one context may not be compatible with processes created by another.
Choose who owns the destination
| Design | Who owns the file or other destination? | How records reach it | Main trade-off |
|---|---|---|---|
| Handler in each worker | Each worker has its own handler, potentially targeting the same destination. | Workers handle and format records locally. | Ordinary standard-library file handlers do not provide a standard cross-process mechanism to serialize writes to one shared file. Avoid treating this as process-safe. |
| Queue and listener | One listener owns the destination handlers. | Workers send records through a multiprocessing queue; the listener dispatches them to its handlers. | Centralizes destination formatting and filtering, but requires deliberate queue, error, and shutdown handling. |
| Socket receiver | A receiver owns the destination handlers. | Workers send records to a socket-based receiver. | Another centralization option documented in the Logging Cookbook; it introduces a receiver and transport to manage. |
For many workers writing to one file, the queue/listener design is the standard-library pattern to consider first. The Python Logging Cookbook also describes a socket-based receiver. Do not assume that separate ordinary file handlers pointed at the same file make concurrent writes safe.
Configure worker logging intentionally
For a fresh-interpreter worker, configure the intended worker-side handler during worker initialization. If records should go to a listener, that handler is a QueueHandler connected to the queue shared with the listener. Avoid attaching both a queue handler and a destination handler unless you deliberately want each record sent to both places.
import logging.handlers
def configure_worker(log_queue):
root = logging.getLogger()
root.handlers.clear()
root.setLevel(logging.INFO)
root.addHandler(logging.handlers.QueueHandler(log_queue))
This minimal setup replaces handlers on the root logger; it does not clear handlers attached directly to every named logger. If named loggers have their own handlers, inspect them too. With propagation enabled, a record can reach both a named logger’s handler and handlers on its ancestors. Choose one intended route for each record.
Rank #3
- 2024 OSHA Construction Safety Book is the seventh edition with the new OSHA HazCom final rule on 5/20/24. While the rule takes effect 7/19/24, the compliance dates don’t begin until 1/19/26 per 29 CFR 1910.1200(j).
- Construction Site Book offers quick access to essential OSHA regulations, jobsite hazards, and practical safety tips. It also helps employees identify hazards and prevent injuries and illnesses.
- Features easy-to-read format, full-color images, chapter quizzes with answer key, and comes in a compact size making it a convenient reference for employees.
- Critical topics include Confined Space Entry; Cranes & Derricks; Electrical Safety; Emergency Response; Ergonomics & Back Safety; Excavations; Fall Protection; First Aid & Bloodborne Pathogens; HazCom; Health & Wellness; Jobsite Exposures; Lockout/Tagout; Ladders & Stairways; Materials Handling/Storage; Motor Vehicles; PPE; Scaffolds; Site Safety & Security; Slips, Trips & Falls; Tool Safety; Welding, Cutting & Brazing; and Work Zone Safety.
- Specifications: 5 1/4” x 7 1/4", English, Soft bound. 7th Edition. Copyright 2024.
When a worker is forked
If a worker is created with fork, it may retain the parent’s configured logging state. Review inherited handlers and loggers before adding worker configuration, or records can be emitted to both an inherited destination and the new one.
The Logging Cookbook’s multiprocessing example disables existing loggers in its worker and listener configurations to prevent a parent-side setup logger from remaining active after a fork. It is an illustrative configuration, not a rule that every application must disable every logger. Make the change only where inherited setup state is part of the problem, and preserve loggers the application intentionally needs.
Rank #4
- Used Book in Good Condition
Send shared output through a queue and listener
- Create compatible multiprocessing objects. Create the queue and workers from the same multiprocessing context. In a reusable library, accept the caller’s context rather than imposing one.
- Initialize each worker. Attach a
QueueHandlerto the worker’s intended logger or root logger, and ensure unwanted inherited or pre-existing handlers are not also emitting the records. - Give destination handlers to one listener. Configure the listener’s file, rotating-file, console, or other handlers centrally, including their formatters, filters, and levels. Workers then enqueue records instead of writing directly to the shared destination.
- Apply handler levels deliberately. If using
QueueListener, passrespect_handler_level=Truewhen the destination handlers’ levels should filter queued records. Its documented default isFalse. - Shut down in order. Stop and join workers, trigger listener shutdown, then stop and join the listener before application exit.
QueueListener.stop()waits for its thread; if it is not called, queued records may remain unprocessed. Python 3.14 added context-manager support forQueueListener.
Queue pitfalls to plan for
Do not send multiprocessing’s own debug logs into the same queue
multiprocessing.Queue can emit DEBUG messages through multiprocessing’s internal logger when items are queued. If those messages are handled by a QueueHandler using that same queue, Python warns that the result can be deadlock or infinite recursion. Keep multiprocessing’s internal logger from routing those messages back through the queue it is reporting on.
Recommended Free Tools
A full queue can mean lost records
QueueHandler uses nonblocking put_nowait() by default. If a bounded queue is full, handling can fail; when logging.raiseExceptions is false, records may be silently dropped. Choose queue capacity and error handling with the application’s logging needs in mind.
Queued exception formatting has limits
QueueHandler.prepare() merges message arguments and exception information and removes unpickleable items so the record can be sent through the queue. That can limit custom formatting downstream, particularly for exceptions. If the listener needs data that the default preparation removes, customize the handler’s preparation behavior.
Quick Recap
Diagnose duplicate or missing records
- Duplicates after adding worker configuration: check whether a forked child retained parent handlers, and whether a named logger and its ancestors both have handlers while propagation is enabled.
- No output in a spawned worker: verify that worker initialization runs and installs the intended handlers; do not rely on parent logger state being copied.
- Records missing at shutdown: confirm the listener is stopped after workers finish and before the application exits.
- Unexpected listener filtering: check
QueueListener’srespect_handler_levelsetting and the listener-side handler levels. - Deadlock or recursion around queue logging: check whether multiprocessing’s internal DEBUG logger is routed through a
QueueHandlerconnected to the same queue.
Official references
- Python 3.14.8 Logging HOWTO explains logger, handler, and propagation behavior.
- Python 3.12.15 Logging Cookbook documents multiprocessing queue/listener and socket-receiver approaches, including its inherited setup-logger example.
- Python 3.11.17 Logging Cookbook discusses the lack of a standard cross-process method for safely writing to one file.
- CPython multiprocessing documentation covers start methods, defaults, and context guidance.
- Python 3.15.0rc3 logging.handlers documentation describes
QueueHandler,QueueListener, and their caveats.
