The Occasional Occurence

So while I was mowing my grass last night*, I got to thinking about Python 2.5's enhanced generators and how I hadn't tried them out yet. Here is a simple example that uses the consumer/pipeline model described in PEP 342.

__author__ = "Christian Wyglendowski"
__license__ = "MIT"

def consumer(func):
    """Automatically advance generator to 1st yield point - (from PEP 342)"""
    def wrapper(*args, **kw):
        gen = func(*args, **kw)
        gen.next()
        return gen
    wrapper.__name__ = func.__name__
    wrapper.__dict__ = func.__dict__
    wrapper.__doc__ = func.__doc__
    return wrapper

@consumer
def reverser(next=None):
    """A string-reversing consumer"""
    t = yield
    if next:
        yield next.send("".join(reversed(t)))
    else:
        yield "".join(reversed(t))

def transformer(methodname, *args, **params):
    """Creates a consumer for the given string methodname.

    Passes *args and **params to the string method."""
    @consumer
    def _transform(next=None):
        text = yield
        m = getattr(text, methodname)
        transformed = m(*args, **params)
        if next:
            yield next.send(transformed)
        else:
            yield transformed
    _transform.__name__ = methodname
    return _transform

def transform(text, transformers):
    """Passes text through a list of transformers and returns the result."""
    next = None
    for t in reversed(transformers):
        next = t(next)
    return next.send(text)

(Is this too much code for a blog post?)

So what can you do with all of that? Easily perform various combinations of string transformations!

In [1]: title = transformer('title')
In [2]: transform('python', [title, reverser])
Out[2]: 'nohtyP'
In [3]: transform('python', [reverser, title])
Out[3]: 'Nohtyp'

It's a fairly contrived example, but the concept of this pipeline of transformations (described in much more detail in PEP 342) is really pretty cool. An application applying a more advanced version of something like this could process it's output using a variety of plugins that a user enabled.

Here is one final example:

In [4]: decode = transformer('decode', 'utf-8')
In [5]: replace = transformer('replace', unichr(347), 's')
In [6]: encode = transformer('encode', 'utf-8')
In [7]: transform('Chri\xc5\x9btian', [decode, replace, encode])
Out[7]: 'Christian'

Has anyone else seen any neat examples of PEP 342 in the wild?

cw

* I usually gravitate to thinking about some obscure programming paradigm or feature while I am mowing grass. I think it has to do with how exceptionally boring mowing is.

EDIT: I guess I could have made the generators, um, more generative, so they aren't simply one-shot function-like simpletons. Oh well.