GAE inheritance filtering

Model inheritance and filtering

I talked about using GAE from the command line. I am going to reuse part of that to show how to make inheritance filtering. First, let’s separate the init code into gaeinit.py:

from appengine_django import LoadSdk
LoadSdk()

from google.appengine.tools import dev_appserver_main as dae
da = dae.DEFAULT_ARGS
da[dae.ARG_DATASTORE_PATH] = '/path/to/datastore/file'

from appengine_django import InstallAppengineHelperForDjango
InstallAppengineHelperForDjango()

Replace the datastore file path as appropriate – any path would do (e.g. ‘/tmp/model-inheritance.datastore’).

Let’s make file called model-inheritance.py in the same folder, initialize using gaeinit and put the models:

import gaeinit
from google.appengine.ext import db

# Main expando
class Word(db.Expando):
  ttype = None

  @classmethod
  def ffil(self, res):
    return res

  @classmethod
  def tfil(self, res):
    if self.ttype == None: 
      return res
    return res.filter('t =', self.ttype)

  @classmethod
  def all(self):
    res = super(Word, Word).all()
    res = self.tfil(res)
    res = self.ffil(res)
    return res

  def __str__(self):
    return "[%s, %s]" % (self.t, self.i)

# Sub-classes
class Adjective(Word):
  ttype = 'adjective'

class Noun(Word):
  ttype = 'noun'

class Verb(Word):
  ttype = 'verb'

class SecondVerb(Word):
  ttype = 'verb'
  
  @classmethod
  def ffil(self, res):
    return res.filter('i =', 2)

The base model class is called Word. It is a normal Expando, with the filtering additions and a standard str just for pretty printing (see Python docs for this one). Filtering additions are simple:

  • all replaces the default Expando all classmethod to allow additional filtering by tfil and ffil,
  • tfil is a type filtering method – it filters by ‘t =’, where t is the attribute on the Expando,
  • ffil is a function filtering method – by default, doesn’t do anything, so it’s basically an IoC placeholder.

The subclasses just define three word types – adjective, noun and verb. SecondVerb is a contrived example that shows how to use ffil to additionally filter by another property, in this case the dummy i variable.

Here’s how we can test and use it:

# Populate with test data
db.delete(Word.all())

for t in ['adjective', 'noun', 'verb']:
  for i in range(0, 4):
    d = Word(t=t, i=i)
    d.put()

# List
for k in [Word, Adjective, Noun, Verb, SecondVerb]:
  print "---- %s" % k.__name__
  for r in k.all():
    print "%s" % r

The above prints the following:

---- Word
[adjective, 0]
[adjective, 1]
[adjective, 2]
[adjective, 3]
[noun, 0]
[noun, 1]
[noun, 2]
[noun, 3]
[verb, 0]
[verb, 1]
[verb, 2]
[verb, 3]
---- Adjective
[adjective, 0]
[adjective, 1]
[adjective, 2]
[adjective, 3]
---- Noun
[noun, 0]
[noun, 1]
[noun, 2]
[noun, 3]
---- Verb
[verb, 0]
[verb, 1]
[verb, 2]
[verb, 3]
---- SecondVerb
[verb, 2]

Obviously, this is query-only – you need to use the base class to input all the values, as using e.g. Noun.put() would put it under Noun, not under Word. I find it useful for model categorization, I find this not to be a huge drawback in these cases.


Google App Engine Python scripting

Command line GAE

I am going to rely on the setup described in one of my previous posts. Not everything is needed – just the basic setup: * Google App Engine SDK for Python, * Application based on google-app-engine-django, * csvup sample from the above post.

Say you have some model in your app. I will use csvup.models.Person for this example. I had a need a few times before to make a simple Python command-line based applications that would rely on things provided by GAE. This allows for doing some of the things you can do in your views, but without the need to manually start the dev server and point your browser or use hacks like curl just to initiate the view action.

This is very simple with google-app-engine-django. All the code needed is actually provided in main.py file in the root of the distribution and boils down to this:

from appengine_django import InstallAppengineHelperForDjango
InstallAppengineHelperForDjango()

from csvup.models import Person

for p in Person.all():
  print p

This will initialize the helper and display all the Person instances currently in your production datastore. Neat.

LoadSdk

If you, for some reason, need to access GAE SDK code, you would not be able to do that before you call InstallAppengineHelperForDjango(). That can be a problem, as we will soon determine. There’s a handy function for that, too:

from appengine_django import LoadSdk
LoadSdk()

This will initialize the SDK itself (e.g. set up the paths to it based on the .google_appengine symlink you created), so you can import google.appengine packages.

Using your own datastore

Why is LoadSdk important? Because you can do this:

from appengine_django import LoadSdk
LoadSdk()

from google.appengine.tools import dev_appserver_main as dae
da = dae.DEFAULT_ARGS
da[dae.ARG_DATASTORE_PATH] = '/tmp/my-datastore-path.datastore'

from appengine_django import InstallAppengineHelperForDjango
InstallAppengineHelperForDjango()

from csvup.models import Person
from datetime import date

Person(name='John', birthdate=date.today(), countriesVisitedCount=3, everMarried=True).save()
for p in Person.all():
  print p
print 'done'

Digging inside the GAE helper, you can see that is was made from the manage.py perspective, quite reasonable and expected. This, however, restricts its use, as manage.py doesn’t have all the options that “original” dev_appserver.py provides (have a peek here). There are some bugs, like this one, that reflect this feature.

For some of these, it uses the default arguments specified in GAE SDK’s google/appengine/tools/dev_appserver_main.py. If you take a look, you will find out there is a dict called DEFAULT_ARGS, containing some settings that you might consider changing to suit your needs. This is exactly what the above code does – it changes the datastore path. This can be useful if your script does some destructive operations on the datastore which warrant a separate datastore file just for it.