11  Logging

Logging is the concept of that is used to protocol everything or at least every important action/result that happens in a program in order to allow track how the program is doing.

This is particularly useful for the developer of an application to perform

Tip

A logger is particularly useful when an application is not running under the supervision of the developer. For example by another user, on an embedded system, a server, etc..

Python provides an extensive module named logging for these purposes.

1import logging
numberlist = [-2, -1, 0, 1, 2, "a", 1/4]

for number in numberlist:    
    try:
2        logging.info(f"Working on number {number}")
        inverse = 1.0 / number
    except ZeroDivisionError as e:
        logging.error(f"Tried to divide by zero, error is {e}")
    except TypeError:
        logging.warning(f"The list does not only contain numbers")
1
Include the module
2
We can classify what type of content we are logging, the correspond to the log level 
ERROR:root:Tried to divide by zero, error is float division by zero
WARNING:root:The list does not only contain numbers

By default the logger write to the terminal. This might not always be what you want so this can be changed easily, together with the log level:

import logging
1logging.basicConfig(filename="app.log",
2                    level=logging.INFO)
numberlist = [-2, -1, 0, 1, 2, "a", 1/4]

for number in numberlist:    
    try:
        logging.info(f"Working on number {number}")
        inverse = 1.0 / number
    except ZeroDivisionError as e:
        logging.error(f"Tried to divide by zero, error is {e}")
    except TypeError:
        logging.warning(f"The list does not only contain numbers")
1
we can specify a filename
2
we can also specify what the current log level, i.e. what kind of messages are actually put into the logger. 
cat app.log
INFO:root:Working on number -2
INFO:root:Working on number -1
INFO:root:Working on number 0
ERROR:root:Tried to divide by zero, error is float division by zero
INFO:root:Working on number 1
INFO:root:Working on number 2
INFO:root:Working on number a
WARNING:root:The list does not only contain numbers
INFO:root:Working on number 0.25
DEBUG:asyncio:Using selector: EpollSelector
Caution

We currently use the root logger and as soon as one handler is defined, i.e. the logger is called for the first time, calling logging.basicConfig has no effect, unless force=True is included.

As e consequence you need to restart Python if you want to use the code below as is consecutively.

There is a range of levels available that can always be extended. The default levels are in the following table and if you set the level only message of a type equal or higher (numeric value) are written to the output.:

The different log levels explained. The table is copied from docs, accessed on the 21st Nov. 24.
Level Numeric value What it means / When to use it
logging.NOTSET 0 When set on a logger, indicates that ancestor loggers are to be consulted to determine the effective level. If that still resolves to NOTSET, then all events are logged. When set on a handler, all events are handled.
logging.DEBUG 10 Detailed information, typically only of interest to a developer trying to diagnose a problem.
logging.INFO 20 Confirmation that things are working as expected.
logging.WARNING 30 An indication that something unexpected happened, or that a problem might occur in the near future (e.g. ‘disk space low’). The software is still working as expected.
logging.ERROR 40 Due to a more serious problem, the software has not been able to perform some function.
logging.CRITICAL 50 A serious error, indicating that the program itself may be unable to continue running.
Tip

The default logging level setting is logging.WARNING as can be seen in the output of the first listing.

Tip

Note that the logger module does not automatically flush to the file. Meaning it might not be a real time logging process.

An important additional feature is to specify the format of the message. This is particularly useful to get debugging information with certain timing information.

import logging                                                          
logging.basicConfig(format="%(asctime)s:%(levelname)s: %(message)s",
                    level=logging.DEBUG)
numberlist = [-2, -1, 0, 1, 2, "a", 1/4]

for number in numberlist:    
    try:
        logging.debug(f"Working on number {number}")
        inverse = 1.0 / number
    except ZeroDivisionError as e:
        logging.error(f"Tried to divide by zero, error is {e}")
    except TypeError:
        logging.warning(f"The list does not only contain numbers")
2024-11-29 10:10:16,559:DEBUG: Working on number -2
2024-11-29 10:10:16,560:DEBUG: Working on number -1
2024-11-29 10:10:16,560:DEBUG: Working on number 0
2024-11-29 10:10:16,561:ERROR: Tried to divide by zero, error is float division by zero
2024-11-29 10:10:16,561:DEBUG: Working on number 1
2024-11-29 10:10:16,562:DEBUG: Working on number 2
2024-11-29 10:10:16,563:DEBUG: Working on number a
2024-11-29 10:10:16,564:WARNING: The list does not only contain numbers
2024-11-29 10:10:16,564:DEBUG: Working on number 0.25
Tip

We can use multiple loggers in a project. The usual convention is to use __name__ as the name by calling logging.getLogger(__name__).

import logging                                                          
logger = logging.getLogger(__name__)
numberlist = [-2, -1, 0, 1, 2, "a", 1/4]

for number in numberlist:    
    try:
        logger.debug(f"Working on number {number}")
        inverse = 1.0 / number
    except ZeroDivisionError as e:
        logger.error(f"Tried to divide by zero, error is {e}")
    except TypeError:
        logger.warning(f"The list does not only contain numbers")
ERROR:__main__:Tried to divide by zero, error is float division by zero
WARNING:__main__:The list does not only contain numbers

Once the logger is defined, we can use it in the same fashion as we did before. This allows us to use multiple logger in the same application.

See docs for more information.