Ways to Eleminate bugs

• Testing
• Defensive Programming
• Eleminating the source of Bugs - Debugging

Densive Programming

• Write Specifications for Functions
• Modularize Program
• Check conditions for input/output (assertion)
  • #### Testing
    • Check input/output pairs to specifications (How to break a program?)
  • #### Debugging
    • Study events leading up to an error (How can I fix my program?)

• from the start, design code to ease this part
• break program into modules that can be tested and debugged individually
• document constraints on modules
• what do you expect the input to be? the output to be?
• document assumptions behind code design

WHEN ARE YOU READY TO TEST?

Ensure code runs

• remove syntax errors
• remove static semantic errors
• Python interpreter can usually find these for you

Have a set of expected results

• an input set
• for each input, the expected output

CLASSES OF TESTS

Unit testing

• validate each piece of program
• testing each function separately

Regression testing

• add test for bugs as you find them in a function
• catch reintroduced errors that were previously fixed

Integration testing

• does overall program work?
• tend to rush to do this (don't start with Integration testing)

TESTING APPROACHES

• intuition about natural boundaries to the problem

defis_bigger(x, y):
""" Assumes x and y are ints Returns True if y is less than x, else False ""

• can you come up with some natural partitions?
• if no natural partitions, might do random testing
• probability that code is correct increases with more tests
• better options below

black box testing

• explore paths through specification

glass box testing

• explore paths through code

BLACK BOX TESTING

defsqrt(x, eps):
""" Assumes x, eps floats, x >= 0, eps > 0
Returns res such that x-eps <= res*res <= x+eps"""

• designed without looking at the code
• can be done by someone other than the implementer to avoid some implementer biases
• testing can be reused if implementation changes
• paths through specification
• build test cases in different natural space partitions
• also consider boundary conditions (empty lists, singleton list, large numbers, small numbers)

GLASS BOX TESTING

• use code directly to guide design of test cases
• called path-complete if every potential path through code is tested at least once

what are some drawbacks of this type of testing?

• can go through loops arbitrarily many times
• missing paths

guidelines

• branches
• for loops
• while loops

BUGS

once you have discovered that your code does not run properly, you want to:

• isolate the bug(s)
• eradicate the bug(s)
• retest until code runs correctly

RUNTIME BUGS

Overt vs. covert:

• Overt has an obvious manifestation
  – code crashes or runs forever
• Covert has no obvious manifestation
  – code returns a value, which may be incorrect but hard to determine

Persistent vs. intermittent:

• Persistentoccurs every time code is run
• Intermittentonly occurs some times, even if run on same input

CATEGORIES OF BUGS

Overt and persistent

• Obvious to detect
• Good programmers use defensive programming to try to ensure that if error is made,bug will fall into this category

Overt and intermittent

• More frustrating, can be harder to debug, but if conditions that prompt bug can be reproduced, can be handled

Covert

• Highly dangerous, as users may not realize answers are incorrect until code has been run for long period

DEBUGGING

• steep learning curve
• goal is to have a bug-free program

tools

• built in to IDLE and Anaconda
• Python Tutor
• print statement
• use your brain, be systematicin your hunt

PRINT STATEMENTS

• good way to test hypothesis

when to print

• enter function
• parameters
• function results

use bisection method

• put print halfway in code
• decide where bug may be depending on values

ERROR MESSAGES -EASY

• trying to access beyond the limits of a list
  • test = [1,2,3]then test[4] -> IndexError

• trying to convert an inappropriate type
  • int(test) -> TypeError

• referencing a non-existent variable
  • a -> NameError

• mixing data types without appropriate coercion
  • '3'/4 -> TypeError

• forgetting to close parenthesis, quotation, etc.
  • a= len([1,2,3]
  • print a -> SyntaxError

LOGIC ERRORS -HARD

• think before writing new code
• draw pictures, take a break

explain the code to

• someone else
• a rubber ducky

DEBUGGING STEPS

study program code

• ask how did I get the unexpected result
• don’t ask what is wrong
• is it part of a family?

scientific method

• study available data
• form hypothesis
• repeatable experiments
• pick simplest input to test with

DON’T

• Write entire program
• Test entire program
• Debug entire program

• Change code without backup
• Forget where bug was or what change you made
• Panic

DO

• Write a function
• Test the function, debug the function
• Write a function
• Test the function, debug the function
• Do integration testing

• Backup code
• Change code
• Write down potential bug in a comment
• Test code
• Compare new version with old version

DEBUGGING SKILLS

Treat as a search problem: looking for explanation for incorrect behavior

• study available data –both correct test cases and incorrect ones
• form an hypothesis consistent with the data
• design and run a repeatable experiment with potential to refute the hypothesis
• keep record of experiments performed: use narrow range of hypotheses

DEBUGGING AS SEARCH

• want to narrow down space of possible sources of error
• design experiments that expose intermediate stages of computation (use print statements!), and use results to further narrow search
• binary search can be a powerful tool for this

SOME PRAGMATIC HINTS

• look for the usual suspects
• ask why the code is doing what it is, not why it is not doing what you want
• the bug is probably not where you think it is –eliminate locations
• explain the problem to someone else
• don’t always believe the documentation
• take a break and come back to the bug later

---

Reference

• edX course offered by MIT
• 6.00.1x Introduction to Computer Science and Programming Using Python