« Specifically designed to be readableVellum coming along nicely »

Tracking memory leaks with Dowser

06/11/08

Permalink 08:56:58 pm, by fumanchu Email , 335 words   English (US)
Categories: Python, CherryPy

Tracking memory leaks with Dowser

Marius Gedminas just wrote a post on memory leaks. He could have used Dowser to find the leak more easily, I'll bet.

Dowser is a CherryPy application for monitoring and managing object references in your Python program. Because CherryPy runs everything (even the listening HTTP socket) in its own threads, it's a snap to include Dowser in any Python process. Dowser is also very lightweight (because CherryPy is). Here's how I added it to a Twisted project we're using at work:

...
from twisted.application import service
application = service.Application("My Server")
s.setServiceParent(application)

import cherrypy
from misc import dowser
cherrypy.config.update({'server.socket_port': 8088})
cherrypy.tree.mount(dowser.Root())
cherrypy.engine.autoreload.unsubscribe()
# Windows only
cherrypy._console_control_handler.unsubscribe()
cherrypy.engine.start()

from twisted.internet import reactor
reactor.addSystemEventTrigger('after', 'shutdown', cherrypy.engine.exit)

The lines before 'import cherrypy' already existed and are here just for context (this is a Twisted service.tac module). Let's quickly discuss the new code:

  1. import cherrypy and dowser. You don't have to stick dowser into a 'misc' folder; that's just how I checked it out from svn.
  2. Set the port you want CherryPy to listen on; pick a port your app isn't already using if it's a TCP server.
  3. Mount the dowser root.
  4. Turn off the CherryPy autoreloader, and the Ctrl-C handler if you're on Windows. I should really turn that off by default in CP. :/
  5. Start the engine, which starts listening on the port in a new thread among other things.
  6. Tell Twisted to stop CherryPy when it stops.

Then browse to http://localhost:8088/ and you'll see pretty sparklines of all the objects. Change the URL to http://localhost:8088/?floor=20 to see graphs for only those objects which have 20 or more objects.

Then, just click on the 'TRACE' links to get lots more information about each object. See the Dowser wiki page for more details and screenshots.

9 comments

Comment from: Rob Cakebread [Visitor] · http://doapspace.org/

Have you got a recipe for dowser with Turbogears? I have a nasty memory leak I'd love to squash.

06/12/08 @ 02:18
Comment from: Marius Gedminas [Visitor] · http://gedmin.as

Thanks, this is an interesting app that I hadn't known about.

06/12/08 @ 06:54
Comment from: fumanchu [Member] Email

Rob,

The dowser package consists of a single CherryPy Root object. You should be able to mount it on your Turbogears app like any other controller:

cherrypy.tree.mount(dowser.Root(), '/dowser', config)

06/12/08 @ 11:28
Comment from: Rob Cakebread [Visitor] · http://doapspace.org/

I'm using the stable version of Turbogears which is based on CherryPy 2, but dowser seems to only work with CherryPy 3.

from cherrypy import tools
ImportError: cannot import name tools

I got around that, but then there's quickstart. Do you think it'll be port-able to CP 2 without too much work?

Thanks,
Rob

06/12/08 @ 11:39
Comment from: fumanchu [Member] Email

Rob, CP 2.3 support added in http://www.aminus.net/changeset/137. You may have to tweak the staticfilter and/or url settings if you mount dowser somewhere other than /.

06/13/08 @ 11:36
Comment from: Rob Cakebread [Visitor] · http://doapspace.org/

Awesome, thanks, I'll give it a shot with TurboGears.

06/13/08 @ 18:24
Comment from: Swinlee [Visitor]

What license is this under? I don't see any license in the code or svn tree.

06/14/08 @ 18:11
Comment from: Rob Cakebread [Visitor] · http://doapspace.org/

For Turbogears it was as simple as:

import dowser

Then in my Root controller:

class Root(controllers.RootController):

memleak = dowser.Root()

Then point your browser at
localhost:7878/memleak

Hmmm. Now to figure out what all that output is telling me...

06/15/08 @ 16:35
Comment from: John Viviano [Visitor] Email · http://www.intercorpinc.com

Here's a minimal but complete example for anyone just trying to get started.

import cherrypy
import dowser

class Root:
    @cherrypy.expose

    def index(self):
        return "hello, world."
    index.exposed = True

if name == 'main':

    cherrypy.engine.autoreload.unsubscribe()
    cherrypy.config.update({'server.socket_port': 8088})
    # http://localhost:8088/dowser
    cherrypy.tree.mount(dowser.Root(), '/dowser')
    # http://localhost:8088/
    # hello, world.
    cherrypy.quickstart(Root(),'/')

03/08/10 @ 21:42

Leave a comment


Your email address will not be revealed on this site.

Your URL will be displayed.

Please enter the phrase "I am a real human." in the textbox above.
(Line breaks become <br />)
(Name, email & website)
(Allow users to contact you through a message form (your email will not be revealed.)
November 2017
Sun Mon Tue Wed Thu Fri Sat
 << <   > >>
      1 2 3 4
5 6 7 8 9 10 11
12 13 14 15 16 17 18
19 20 21 22 23 24 25
26 27 28 29 30    

Search

The requested Blog doesn't exist any more!

XML Feeds

blogging soft