The Hand of FuManChu

I recently had to test output that consisted of a long list of dicts against an expected set. After too many long debugging sessions with copious print statements and lots of hand-comparison, I finally got smart and switched to using Python's builtin difflib to give me just the parts I was interested in (the wrong parts).

With difflib and a little pprint magic, a failing test now looks like this:

Traceback (most recent call last):
  File "C:\Python25\lib\site-packages\app\test\util.py", line 237, in tearDown
    self.assertNoDiff(a, b, "Expected", "Received")
  File "C:\Python25\lib\site-packages\app\test\util.py", line 382, in failIfDiff
    raise self.failureException, msg
AssertionError:
--- Expected

+++ Received

@@ -13,4 +13,3 @@

 {'call': 'getuser101',
 'output': {'first_name': 'Georg',
            'gender': u'Male',
            'last_name': 'Handel',
            ...}}
 {'call': 'getuser1',
 'output': None}
 {'call': 'getuser101',
 'output': {'first_name': 'Georg',
            'gender': u'Male',
            'last_name': 'Handel',
            ...}}
-{'call': 'getuser101',
 'output': {'first_name': 'Georg',
            'gender': u'Male',
            'last_name': 'Handel',
            ...}}

...and I can now easily see that the "Received" data is missing the last dict in the "Expected" list. Here's the code (not exactly what I committed at work, but I think this is even better):

import difflib
from pprint import pformat


class DiffTestCaseMixin(object):

    def get_diff_msg(self, first, second,
                     fromfile='First', tofile='Second'):
        """Return a unified diff between first and second."""
        # Force inputs to iterables for diffing.
        # use pformat instead of str or repr to output dicts and such
        # in a stable order for comparison.
        if isinstance(first, (tuple, list, dict)):
            first = [pformat(d) for d in first]
        else:
            first = [pformat(first)]

        if isinstance(second, (tuple, list, dict)):
            second = [pformat(d) for d in second]
        else:
            second = [pformat(second)]

        diff = difflib.unified_diff(
            first, second, fromfile=fromfile, tofile=tofile)
        # Add line endings.
        return ''.join([d + '\n' for d in diff])

    def failIfDiff(self, first, second, fromfile='First', tofile='Second'):
        """If not first == second, fail with a unified diff."""
        if not first == second:
            msg = self.get_diff_msg(first, second, fromfile, tofile)
            raise self.failureException, msg

    assertNoDiff = failIfDiff

The get_diff_msg function is broken out to allow a test method to call self.fail(msg), where 'msg' might be the join'ed output of several diffs.

Happy testing!