Errors and Documentation
Contents
9. Errors and Documentation#
9.1. Lesson overview#
When writing code in any language, the first draft will hardly ever be perfect. Even the second, third, or fifth
draft of our code can have errors, which are known as bugs in
programming. These bugs can range from simple syntax errors to complicated logic problems. Fortunately, Python has
built-in tools in diagnosing, logging, and reporting programming bugs. Therefore, to become proficient at writing
Python code, we need to become comfortable in understanding Python error logs. In this lesson, we will look at how to
read a Python traceback, how to work around errors with the try-except commands, and how to raise errors when
needed contextual information is needed to correct a bug. We will also look at a general troubleshooting approach by
searching through library documentation.
Definition: bug
An error, flaw, or unintended behavior in a program.
9.2. Reading error messages#
When Python runs into an error it will output a logging report called a traceback that shows the line where an exception has occurred, which has caused the Python program to quit. The traceback will show what part of the code started the exception process and then “traces back” step by step through the code blocks to ultimately where the error originated. Following this step by step logging, the traceback will show the exception type and provide an error message that will hint towards what is wrong with the code.
Definition: traceback
A logging report generated when a programming bug is detected.
Let us demonstrate how a traceback works using the following example:
def fail():
asdf
def start():
fail()
start()
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Input In [1], in <cell line: 7>()
4 def start():
5 fail()
----> 7 start()
Input In [1], in start()
4 def start():
----> 5 fail()
Input In [1], in fail()
1 def fail():
----> 2 asdf
NameError: name 'asdf' is not defined
In this example, we can see the traceback begins with an arrow pointed at the start() function command. This is
the call that started the chain of events that led to the exception:
Input In [1], in <cell line: 7>()
4 def start():
5 fail()
----> 7 start()
Differences in how tracebacks are displayed
Based on your version of Python, Anaconda, and your computer’s operating system, your traceback might look slightly different than what is shown in this tutorial. While the format might be different, the content that is relayed will be the same.
There is nothing wrong with this call by itself. The start() function is able to successfully execute its code block
and call the next function fail() that is in our traceback. Note the first line of this section,
Input In [1], in start(), shows the calling of fail() happened in the start() code block:
Input In [1], in start()
4 def start():
----> 5 fail()
The last call of the traceback shows the line of code that was the root cause of our exception:
Input In [1], in fail()
1 def fail():
----> 2 asdf
The traceback then prints out the exception type and the error message:
NameError: name 'asdf' is not defined
Here, the exception type (NameError) is not as important as the error message: name 'asdf' is not defined. The
traceback has reported that asdf is defined before it is referenced in the code. Therefore, the error in our code
is that a value is not assigned to asdf. From this error message we can deduce a
NameError error type refers to when a variable
name is called without having a value assigned to it.
Following a traceback is important to debug your code. When running this code cell as a file, we can see that the traceback still shows the relevant line numbers in the Python file:
Traceback (most recent call last):
File "./fail.py", line 7, in <module>
start()
File "./fail.py", line 5, in start
fail()
File "./fail.py", line 2, in fail
asdf
NameError: name 'asdf' is not defined
9.2.1. Example: The wrong value#
Let’s register another type of exception called a ValueError. This type of error often occurs when we pass
an argument that is of the right type (e.g., program expects a float and receives a float) but has an incorrect
value. A good example of this is when calculating the square root of a negative number. In this example, create a
simple program that takes the square root of the following list of numbers:
2, 4, 16, -5, 253, -2.5, and 100
Enter this set of number as a list object, and pass this list through a for loop that will take the square root
of each number. Have the for loop print out a phrase that mentions both the number and its square root.
The built-in Python library does not have a square root function. However, the
math standard library has this capability with the
sqrt() function. Therefore, you will need to import this
library into your code. You can review how to import the math library here.
Solution:
As seen in the problem statement, there are two numbers, -2.5 and -5, which should give us an error. The code below
demonstrates the ValueError exception using the given code requirements:
from math import sqrt
list = [2, 4, 16, -5, 253, -2.5, 100]
for i in list:
print(f"The square root of {i} is {sqrt(i):.5f}.")
The square root of 2 is 1.41421.
The square root of 4 is 2.00000.
The square root of 16 is 4.00000.
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
Input In [2], in <cell line: 5>()
3 list = [2, 4, 16, -5, 253, -2.5, 100]
5 for i in list:
----> 6 print(f"The square root of {i} is {sqrt(i):.5f}.")
ValueError: math domain error
As seen above, the for loop successfully iterates over three loops before raising the ValueError exception on
-5. The example in the next section will show how we can have the program flag this error,
while also continuing through the remaining values.
9.3. Error catching with try and except#
When writing code, we may be dealing with input data that are not perfectly uniform. For instance, we may be parsing
a CSV file that supposedly only contains integers but a string is found in the file. This may cause our Python
program to crash when it reaches the string. In order to provide some “flexibility” in how our program operates, we
would prefer that our Python code could try an alternative block of code to resolve the issue rather than simply
crashing. Fortunately, the try - except
coding pattern allows us to implement alternative code blocks to handle these exceptions.
To see how this all works, let us start with the following code block, which will initially fail:
def fail(my_list):
a = 0
for i in my_list:
a += i
return a
b = fail([1, 2, "3", 4])
print("b:", b)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
Input In [3], in <cell line: 7>()
4 a += i
5 return a
----> 7 b = fail([1, 2, "3", 4])
9 print("b:", b)
Input In [3], in fail(my_list)
2 a = 0
3 for i in my_list:
----> 4 a += i
5 return a
TypeError: unsupported operand type(s) for +=: 'int' and 'str'
Here, we have defined a function fail(my_list) that will step through an input list, which ideally should be all
int objects, and returns the sum of the list. However, our input list contains a str object, so our function
crashes when it loops over this object. This is described in the traceback that points at the line
a += 1 along with the error message TypeError: unsupported operand type(s) for +=: 'int' and 'str'. Since the
code is trying to add an int object to a str object (i.e., when we get to "3" in our list), a
TypeError exception type is flagged.
Hey! Listen!
Remember that a += i is shorthand for a = a + i!. If you need a quick review of shorthand versions of
simple mathematical operators,
check out our previous lesson on conditional and loops statements.
The try - except pattern will help us bypass this failure mode in our code. The try - except pattern is
similar in structure to the if - else coding pattern that was described in an earlier lesson.
To use the try - except pattern, we first start with the try: command, followed by the code block that we want to
try to execute. After that, we use the except: command, followed with the code block that will be executed if there
is a problem in the try code block. Below is a simplified version of the previously failed code that implements the
try - except pattern:
a = 0
try:
a += '3'
except:
print('oops, had an error')
try:
a += 4
except:
print('oops, had another error')
print("a:", a)
oops, had an error
a: 4
You can see we got to the end of the code, and we were able to print out the value of a. Without the try -
except pattern our code would have crashed out after a += '3'. Instead, we put a += '3' in a try: code block,
so when it failed it skipped to the except: code block and printed the message oops, had an error instead of
crashing. It then proceeded to the next try: code block with a += 4 and executed this command successfully.
Therefore, the next except: block is skipped. The end result is that a == 4, which reflects that the a += 3
code block is skipped and only a += 4 is implemented.
We can make the except statement more informative by including the exception message that was triggered from the
try code block:
a = 0
try:
a += '3'
except Exception as e:
print(e)
try:
a += 4
except Exception as e:
print(e)
print("a:", a)
unsupported operand type(s) for +=: 'int' and 'str'
a: 4
By extending the except: statement to except Exception as e: we can print out the exception as the variable e.
This will give us more information about how the preceding try: code block failed, instead of simply printing out that
an error happened.
Let us now add the try - except pattern to our original example:
def fail(my_list):
a = 0
try:
for i in my_list:
a += i
except Exception as e:
print(e)
return a
b = fail([1, 2, "3", 4])
print("b:", b)
unsupported operand type(s) for +=: 'int' and 'str'
b: 3
Well, our code did not crash, but we also didn’t finish processing the list. We were able to sum up 1 and 2, but after
the a += '3' was encountered when we broke out of the for loop. This is because we have our entire for loop in
the try code block. When working with try - except, it is important to recognize what would happen if the code
block raises an exception. If we want to make sure all elements of the list is processed, we will need to put the
try - except pattern inside the for loop. The code below implements this change an addition to modifying the
fail() function to return both the sum of integers in the list and a list object of failed elements:
def fail(my_list):
a = 0
failed = []
for i in my_list:
try:
a += i
except Exception as e:
print(e, i)
failed.append(i)
return a, failed
b, failures = fail([1, 2, "3", 4])
print("b:", b)
print("failed:", failures)
unsupported operand type(s) for +=: 'int' and 'str' 3
b: 7
failed: ['3']
Now we are able to get b == 7 by summing up 1, 2, and 4. Additionally, we added a failed list and when the
except block is executed we append the failed element to that list. We can now return a and failed as output of
the function.
9.3.1. Example: Trying to get out of the problem#
Modify the solution from the previous example using the try - except command structure
so the code iterates over all values in the starting list and also mentions if a number in the list is less than zero.
Solution:
Very little needs to be done in order to add the try - except command structure. The try command will have our
main print() code, while the except command issues a new print() call that states that the current value in
the list is less than zero.
from math import sqrt
list = [2, 4, 16, -5, 253, -2.5, 100]
for i in list:
try:
print(f"The square root of {i} is {sqrt(i):.5f}.")
except:
print(f"Cannot take the square root of {i} since it is less than zero!")
The square root of 2 is 1.41421.
The square root of 4 is 2.00000.
The square root of 16 is 4.00000.
Cannot take the square root of -5 since it is less than zero!
The square root of 253 is 15.90597.
Cannot take the square root of -2.5 since it is less than zero!
The square root of 100 is 10.00000.
All in all, the try - except command structure is an effective technique to ensure that erroneous values are
appropriately flagged while ensuring the program completes.
9.4. Returning errors with raise#
While we can use try - except to bypass errors, there are times when we want a program to exit when an error is
encountered and return more information to the user besides a generic exception message. The
raise keyword is designed to exit the code with a
raised exception that can include additional information for the user as to why the program exited. The code
below demonstrates how the raise keyword is used:
def fail(my_list):
a = 0
for i in my_list:
if isinstance(i, int):
a += i
else:
raise Exception("Value in list is not an integer: " + str(i))
return a
b = fail([1, 2, "3", 4])
---------------------------------------------------------------------------
Exception Traceback (most recent call last)
Input In [9], in <cell line: 10>()
7 raise Exception("Value in list is not an integer: " + str(i))
8 return a
---> 10 b = fail([1, 2, "3", 4])
Input In [9], in fail(my_list)
5 a += i
6 else:
----> 7 raise Exception("Value in list is not an integer: " + str(i))
8 return a
Exception: Value in list is not an integer: 3
In this example, a new built-in function is being used in the if statement, isinstance(), which compares the
class of an object to a desired class. In this case, we are testing to see if i belongs to the int class. If
it is an int object, the program moves the following code block with the a += i command. If it
is not an int object, then the program moves to the else: code block and raises an exception.
As seen above, the raise command allows us to insert our own unique exception (in this case
"Value in list is not an integer: " + str(i)) ). If we did not include this but rather used our
except Exception as e structure from before we would have received the generic traceback exception message:
TypeError: unsupported operand type(s) for +=: 'int' and 'str'. Therefore, the raise keyword has allowed us to
produce a more informative error message that clearly says that an element in the list is not an integer and the
erroneous value in question.
There are many built-in Exceptions,
and we can make custom ones too. It is a good practice to match the
exception type to what sort of error is happening. For example, we can use the built-in
ValueError exception which is
better suited for an exception type than the generic
Exception:
def fail(my_list):
a = 0
for i in my_list:
if isinstance(i, int):
a += i
else:
raise ValueError("Value in list is not an integer: " + str(i))
return a
b = fail([1, 2 , "3", 4])
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
Input In [10], in <cell line: 10>()
7 raise ValueError("Value in list is not an integer: " + str(i))
8 return a
---> 10 b = fail([1, 2 , "3", 4])
Input In [10], in fail(my_list)
5 a += i
6 else:
----> 7 raise ValueError("Value in list is not an integer: " + str(i))
8 return a
ValueError: Value in list is not an integer: 3
By using the ValueError exception type we can give a little more context to the error when someone encounters it.
9.4.1. Example: Raising an issue#
An example in a previous lesson created a function that converted an input force to one of three
units. Part of the exercise required the function to notify if incorrect force units were used.
This was done using simple print() calls. In this example, replace the print() calls with raise commands so
ValueError exceptions are issued when incorrect units are entered.
Solution:
Since the print() calls that handle the incorrect units are placed in else statements, it is straightforward to
modify the code. All we need to do is replace the print() functions with a raise ValueError() calls.
def convert_force(initial_force, initial_units="N", converted_units="lbf"):
"""
Convert force between Newtons, pound force, and dynes. Returns the
converted force and units.
:param initial_force: Force to be converted
:type: float
:param initial_units: Initial units of force. Allowable values are
"N", "lbf", and "dyn". Default value is "N"
:type: str
:param converted_units: Units of force to be converted to. Allowable
values are "N", "lbf", and "dyn". Default value is "N"
:type: str
:return:
- converted_force - converted force value (type: float)
- converted_units - final units (type: str)
"""
# Convert force to N to standardize
if initial_units == "N":
force_N = initial_force
elif initial_units == "lbf":
force_N = initial_force / 0.225
elif initial_units == "dyn":
force_N = initial_force / 100000
else:
raise ValueError("Incorrect initial force units")
# Convert to new units
if force_N == "\"incorrect initial force units\"":
converted_force = force_N
elif converted_units == "N":
converted_force = force_N
elif converted_units == "lbf":
converted_force = force_N * 0.225
elif converted_units == "dyn":
converted_force = force_N * 100000
else:
raise ValueError("Incorrect converted force units")
return converted_force, converted_units
The code below demonstrates the error handling in action:
converted_force, converted_units = convert_force(2536.2, initial_units="dyn", converted_units="bad")
print(f"The converted force is {converted_force} {converted_units}")
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
Input In [12], in <cell line: 1>()
----> 1 converted_force, converted_units = convert_force(2536.2, initial_units="dyn", converted_units="bad")
2 print(f"The converted force is {converted_force} {converted_units}")
Input In [11], in convert_force(initial_force, initial_units, converted_units)
38 converted_force = force_N * 100000
39 else:
---> 40 raise ValueError("Incorrect converted force units")
42 return converted_force, converted_units
ValueError: Incorrect converted force units
This route is a more effective way for error handling since it produces both the exception output block and the traceback.
9.5. Documentation and troubleshooting#
It can be difficult to troubleshoot errors in code. Besides the inherent complexity of the code you are working with, Python libraries are constantly in development and changing. However, there are a few general strategies we can use to troubleshoot coding errors:
1. Use help()
As we have discussed in an earlier lesson, the built-in help() function is a useful function to read
the docstrings of functions. Information about default values, normal behavior, and commonly seen errors are often
included in these docstrings.
2. Search online by last traceback line
A simple search online of the last traceback line can help, especially if the last line is from a Python library file. When searching the traceback, you may find other users that have encountered the problem. A popular site for asking questions and getting answers is StackOverflow. In evaluating an answer, it is useful to keep your operating environment in mind. Are the answers for Python 2.7 or for Python 3.10? Are they for your operating system or for a different one? How old is the answer, does it apply to our library version?
3. Check the library API documentation online
Most Python libraries have their documentation published online. We can search online for “LIBRARY NAME documentation” to find the documentation site and check to see if we are using a library or function correctly. You may also be able to select the specific version of the library you are using. For instance, on the popular documentation hosting site Read the Docs, the version of the library will be in the bottom left.
As an example of checking documentation for correct usage,
the numpy.loadtxt() documentation shows the
many functional changes across the different versions. For instance, the quotechar optional argument
was new in version 1.23.0. If we are running code that uses that argument, and our version of NumPy is older than
1.23.0, we will run into an error (the installed version of NumPy can be checked with numpy.version.full_version).
4. Check library “Getting Started” or “Tutorials”
If you are getting an error with a library you are using for the first time, it can be useful to run known working code as a sanity check with your setup. For example, scikit-image is a bundled library with Anaconda and is useful for image processing. We can try out the robust gallery of examples first to make sure the functions we are using in scikit-image are working.
9.6. Conclusion#
Bugs and errors are part of life when coding, and it can be disheartening when we run into them. But with the useful
tracebacks that Python generates, we can see how an exception was raised through step by step analysis. We can also
use the try-except coding pattern to work around some errors, such as bypassing bad inputs from a file.
Sometimes it is important to raise an exception in our code, which will stop execution, but can provide important
contextual information through raise statements. Finally, looking for solutions through online resources is a great
way to get a head start on correcting an error. Keep in mind your operating environment when evaluating different
answers online, as some answers may not apply to your particular situation. Searching through the documentation online
and trying out different tutorials is a great way to sanity check your current code.
9.7. Want to learn more?#
Python Software Foundation - Errors and Exceptions
Python Software Foundation - Built-in Exceptions
Python Software Foundation - The Try Statement