« 2009 Linnaeus Awards99 Bottles... »



Permalink 12:22:51 pm, by fumanchu Email , 369 words   English (US)
Categories: Python


I recently had to test output that consisted of a long list of dicts against an expected set. After too many long debugging sessions with copious print statements and lots of hand-comparison, I finally got smart and switched to using Python's builtin difflib to give me just the parts I was interested in (the wrong parts).

With difflib and a little pprint magic, a failing test now looks like this:

Traceback (most recent call last):
  File "C:\Python25\lib\site-packages\app\test\util.py", line 237, in tearDown
    self.assertNoDiff(a, b, "Expected", "Received")
  File "C:\Python25\lib\site-packages\app\test\util.py", line 382, in failIfDiff
    raise self.failureException, msg
--- Expected

+++ Received

@@ -13,4 +13,3 @@

 {'call': 'getuser101',
 'output': {'first_name': 'Georg',
            'gender': u'Male',
            'last_name': 'Handel',
 {'call': 'getuser1',
 'output': None}
 {'call': 'getuser101',
 'output': {'first_name': 'Georg',
            'gender': u'Male',
            'last_name': 'Handel',
-{'call': 'getuser101',
 'output': {'first_name': 'Georg',
            'gender': u'Male',
            'last_name': 'Handel',

...and I can now easily see that the "Received" data is missing the last dict in the "Expected" list. Here's the code (not exactly what I committed at work, but I think this is even better):

import difflib
from pprint import pformat

class DiffTestCaseMixin(object):

    def get_diff_msg(self, first, second,
                     fromfile='First', tofile='Second'):
        """Return a unified diff between first and second."""
        # Force inputs to iterables for diffing.
        # use pformat instead of str or repr to output dicts and such
        # in a stable order for comparison.
        if isinstance(first, (tuple, list, dict)):
            first = [pformat(d) for d in first]
            first = [pformat(first)]

        if isinstance(second, (tuple, list, dict)):
            second = [pformat(d) for d in second]
            second = [pformat(second)]

        diff = difflib.unified_diff(
            first, second, fromfile=fromfile, tofile=tofile)
        # Add line endings.
        return ''.join([d + '\n' for d in diff])

    def failIfDiff(self, first, second, fromfile='First', tofile='Second'):
        """If not first == second, fail with a unified diff."""
        if not first == second:
            msg = self.get_diff_msg(first, second, fromfile, tofile)
            raise self.failureException, msg

    assertNoDiff = failIfDiff

The get_diff_msg function is broken out to allow a test method to call self.fail(msg), where 'msg' might be the join'ed output of several diffs.

Happy testing!


Comment from: Cory [Visitor] · http://goonmill.org/

Terrific suggestion! I'll be passing this along to my team.

01/09/09 @ 15:45
Comment from: Jonathan Ellis [Visitor] · http://spyced.blogspot.com


01/09/09 @ 17:27
Comment from: Patrick [Visitor]

Thank you so much for this idea! You posted it just when I was about to write a similar test.

01/09/09 @ 18:25
Comment from: Lakin Wecker [Visitor] · http://lakin.weckers.net

Was this in an Etsy test? I could have sworn that I wrote something similar before I left or maybe I should remember thinking that I should have.

In any case, your new version is much nicer than the version I remember writing, or thinking about writing.

Also, don't count on me remembering whether I wrote this comment or not in the future - apparently I'm terrible at that.

01/09/09 @ 19:05
Comment from: Empty [Visitor] · http://blog.michaeltrier.com

Wow, great idea. Very creative. This one will definitely come in handy. Thank you.

01/09/09 @ 21:45
Comment from: Marius Gedminas [Visitor] · http://gedmin.as

Nice name; I used to call this sort of function 'assertEqualWithDiff'. Also, nice idea on passing formatted multi-line representations of dicts as individual items to be compared; I used to concatenate everything and do a line-by-line diff.

Is the strange 'not X == Y' phrasing in the last method intentional? It uses __eq__ where 'X != Y' would use __ne__, but shouldn't any code that overrides __eq__ also override __ne__?

01/10/09 @ 07:55
Comment from: Marijn [Visitor] Email · http://protocultura.cl

Great idea! I maintain the dict_compare package on the python cheeseshop that does something similar. I have updated it to include this solution.

01/10/09 @ 15:50
Comment from: fumanchu [Member] Email

@Marius: yes, it's an intentional copy of unittest's assertEqual code, which uses "if not first == second". I assumed there were some corner cases (perhaps with mock objects) where __ne__ is not actually the inverse of __eq__...

01/11/09 @ 00:32
Comment from: Antti Kaihola [Visitor] · http://antti.kaihola.fi/

What about saving first and second outputs to temporary files and launching meld on them? I've done that with doctest output and it's very useful when there's lots of output.

02/02/09 @ 16:33

Leave a comment

Your email address will not be revealed on this site.

Your URL will be displayed.

Please enter the phrase "I am a real human." in the textbox above.
(Line breaks become <br />)
(Name, email & website)
(Allow users to contact you through a message form (your email will not be revealed.)
September 2017
Sun Mon Tue Wed Thu Fri Sat
 << <   > >>
          1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30


The requested Blog doesn't exist any more!

XML Feeds

powered by b2evolution free blog software