Logging for Production
If you didn't log it, it didn't happen. Building the Flight Recorder for your application.
1. The Big Idea (ELI5)
👶 Explain Like I'm 10: The Flight Recorder (Black Box)
Imagine an Airplane (Your App).
- Print Statements: This is the pilot shouting "We are flying!" out the window. If the plane crashes, nobody hears the shout. The information is lost forever.
- Logging: This is the Black Box Flight Recorder. It silently writes everything ("Engine started", "Altitude 10k", "Engine failure") to a fireproof box (Log File). If the plane crashes, investigators open the box to find out exactly what happened.
2. The Levels of Severity
Not all logs are equal. Python's `logging` module has 5 standard levels.
- DEBUG (10): "Variable x is 5". Noisy details for developers only.
- INFO (20): "User logged in". Normal system events.
- WARNING (30): "Disk 90% full". Potential problem, but app is fine.
- ERROR (40): "Database connection failed". A specific request failed.
- CRITICAL (50): "System out of memory". The app is crashing.
3. Basic Configuration
By default, Python logs strictly to the console. Stick this at the top of your script.
PYTHON
import logging
# Configure the system ONCE at startup
logging.basicConfig(
level=logging.INFO, # Ignore DEBUG logs
format='%(asctime)s - %(levelname)s - %(message)s',
filename='app.log', # Write to file instead of console
filemode='a' # Append (Don't overwrite)
)
logging.info("System is starting...")
logging.warning("Config file missing, using defaults.")
logging.error("Unable to contact Payment Gateway!")4. Deep Dive: Structured Logging (JSON)
Text logs (`"User logged in"`) are hard for computers to search.Structured Logging saves logs as JSON objects. This allows tools like Datadog or Splunk to graph your data.
PYTHON
# The Old Way (String)
logging.info(f"Order {order_id} failed for user {user_id}")
# The Modern Way (Structured)
# Using a library like 'structlog' or 'python-json-logger'
logger.info("order_failed", extra={
"order_id": 999,
"user_id": 55,
"reason": "Card Declined"
})
# Result in file:
# {"timestamp": "2023-10-01...", "event": "order_failed", "order_id": 999, ...}5. Advanced: Rotating File Handlers
If you log everything, your `app.log` will grow to 500GB and crash the server. Use `RotatingFileHandler` to keep only the last 5 files of 10MB each.
PYTHON
from logging.handlers import RotatingFileHandler
handler = RotatingFileHandler(
'app.log',
maxBytes=10_000_000, # 10 MB usage limit
backupCount=5 # Keep 5 backups
)
logger = logging.getLogger('my_app')
logger.addHandler(handler)