Logging uncaught exceptions in Python

Logging is an essential part of programming. It is crucial for identifying bugs in the code and helps us remove them. Adding try-except block is great but you have to know the critical section of your code, so that you can put it in the try block and handle any exception through the except block but what if a part of your code which is not inside any try block raises an exception.

When you give your code to a client or run it in some environment where you don’t have the terminal on which the code was executed, you solely rely on your log file to see what happened. It is very likely that an unanticipated exception occurs while the code was running. Some part of your code outside a try block might raise an exception which abruptly terminates the program and there will be no trace of any such exception in the log file because it would be an uncaught exception.

The python interpreter calls a function called sys.excepthook(type, value, tb)before crashing when it encounters an unhandled exception. So, we can create our function with the same parameters as sys.excepthook(type, value, tb)and assign it to the sys.excepthookso that whenever sys.excepthook(type, value, tb)is called, our function will be called. This is how we can do that

import sys
import logging


def my_exception_hook(type, value, tb):
    message = f'{type}\n{value}\n{tb}'
    logging.error(f'an unhandled error raised {message}')

# adding the created function to the sys.excepthook as a hook
sys.excepthook = my_exception_hook

# configuring a log file
logging.basicConfig(filename='wirmachen.log', level=logging.INFO, format='%(asctime)s:%(levelname)s:%(message)s', filemode='w')

# this line raises an exception
print(10 + 'hello')

When you run the above code, it will terminate smoothly because when the exception is raised, my_exception_hookwill write it in the log file. Here is the output from the log file

Output

You can make the log message more comprehensive if you want (such as which line raised the exception) by extracting the information from the traceback object.

While this is enough in a normal scenario, its not enough in case of multi-threading or multi-processing. When an uncaught exception is raised inside a thread, it is handled by the thread and the main program which started the thread, never gets the chance to handle that exception. In that case, sys.excepthookis never called. There is also a bug report about this issue on python issue tracker but its simple solution is to add the following patch

def patch_threading_excepthook():
    """Installs our exception handler into the threading modules Thread object
    Inspired by https://bugs.python.org/issue1230540
    """
    old_init = threading.Thread.__init__
    def new_init(self, *args, **kwargs):
        old_init(self, *args, **kwargs)
        old_run = self.run
        def run_with_our_excepthook(*args, **kwargs):
            try:
                old_run(*args, **kwargs)
            except (KeyboardInterrupt, SystemExit):
                raise
            except:
                sys.excepthook(*sys.exc_info())
        self.run = run_with_our_excepthook
    threading.Thread.__init__ = new_init

patch_threading_excepthook()

Add the above function below assigning my_exception_hook to the sys.excepthook. This function will install our exception handler in the thread object and then whenever a thread raises an uncaught exception, it will be handled in the same way as it is handled in a single threaded program.

Thanks for reading!

References

https://www.lesinskis.com/python-excepthook-logging.html

https://dev.to/joshuaschlichting/catching-every-single-exception-with-python-40o3