Saturday, October 25, 2008

twill with XHTML (not viewing HTML)

Since I couldn't find this anywhere, I'll add it here for those who have the same problem:

I was trying to test a website with twill and got this at the end of my traceback:

raise BrowserStateError("not viewing HTML")
BrowserStateError: not viewing HTML

After spending a bunch of time making sure that, yes, it was spitting out HTML, I figured out that it specifically means that twill (actually mechanize) doesnt like XHTML.

You can likely fix it by adding this at the top of the script:

b = twill.get_browser()
b._browser._factory.is_html = True
twill.browser = b

Presumably, there's a real reason that check is in place, but works-4-me...

Friday, October 24, 2008

appengine memcache memoize decorator

[NOTE: see the 2nd comment below about using a tuple as a key. better to just use pickle.dumps]
I've been playing with google appengine lately. I'm working on a fun, pointless side project. Here's what I came up with for a cache decorator that pulls from memcache based on the args, kwargs and function name if no explicit key is given. The code for creating a key from those is from the recipe linked in the docstring.
"""
a decorator to use memcache on google appengine.
optional arguments:
`key`: the key to use for the memcache store
`time`: the time to expiry sent to memcache

if no key is given, the function name, args, and kwargs are
used to create a unique key so that the same function can return
different results when called with different arguments (as
expected).

usage:
NOTE: actual usage is simpler as:
@gaecache()
def some_function():
...

but doctest doesnt seem to like that.

>>> import time

>>> def slow_fn():
... time.sleep(1.1)
... return 2 * 2
>>> slow_fn = gaecache()(slow_fn)

this run take over a second.
>>> t = time.time()
>>> slow_fn(), time.time() - t > 1
(4, True)

this grab from cache in under .01 seconds
>>> t = time.time()
>>> slow_fn(), time.time() - t < .01
(4, True)

modified from
http://code.activestate.com/recipes/466320/
and
http://code.activestate.com/recipes/325905/
"""

from google.appengine.api import memcache
import logging
import pickle

class gaecache(object):
"""
memoize decorator to use memcache with a timeout and an optional key.
if no key is given, the func_name, args, kwargs are used to create a key.
"""
def __init__(self, time=3600, key=None):
self.time = time
self.key = key

def __call__(self, f):
def func(*args, **kwargs):
if self.key is None:
t = (f.func_name, args, kwargs.items())
try:
hash(t)
key = t
except TypeError:
try:
key = pickle.dumps(t)
except pickle.PicklingError:
logging.warn("cache FAIL:%s, %s", args, kwargs)
return f(*args, **kwargs)
else:
key = self.key

data = memcache.get(key)
if data is not None:
logging.info("cache HIT: key:%s, args:%s, kwargs:%s", key, args, kwargs)
return data

logging.warn("cache MISS: key:%s, args:%s, kwargs:%s", key, args, kwargs)
data = f(*args, **kwargs)
memcache.set(key, data, self.time)
return data

func.func_name = f.func_name
return func