API Documentation

Support for application database generations.

zope.generations.interfaces

Interfaces for support for application database generations.

exception zope.generations.interfaces.GenerationError

Bases: Exception

A database generation is invalid.

exception zope.generations.interfaces.GenerationTooHigh

Bases: GenerationError

A database generation is higher than an application generation.

exception zope.generations.interfaces.GenerationTooLow

Bases: GenerationError

A database generation is lower than application minimum generation.

exception zope.generations.interfaces.UnableToEvolve

Bases: GenerationError

A database can’t evolve to an application minimum generation.

interface zope.generations.interfaces.ISchemaManager

Manage schema evolution for an application.

minimum_generation

Minimum supported schema generation

generation

Current schema generation

evolve(context, generation)

Evolve a database to the given schema generation.

The database should be assumed to be at the schema generation one less than the given generation argument. In other words, the evolve method is only required to make one evolutionary step.

The context argument has a connection attribute, providing a database connection to be used to change the database. A context argument is passed rather than a connection to make it possible to provide additional information later, if it becomes necessary.

This method should not commit a transaction. The transaction will be committed by the caller if there is no error. The method may create savepoints.

Changed in version 5.0: Previously this documentation contained a provision for committing the transaction “if there are no subsequent operations.” That was unclear and incompatible with explicit transaction managers.

Now, this method must never commit the transaction.

getInfo(generation)

Return an information string about the evolution that is used to upgrade to the specified generation.

If no information is available, None should be returned.

interface zope.generations.interfaces.IInstallableSchemaManager

Extends: zope.generations.interfaces.ISchemaManager

Manage schema evolution for an application, including installation.

install(context)

Perform any initial installation tasks

The application has never had the application installed before. The schema manager should bring the database to the current generation.

This method should not commit a transaction. The transaction will be committed by the caller if there is no error. The method may create savepoints.

Changed in version 5.0: Previously this documentation contained a provision for committing the transaction “if there are no subsequent operations.” That was unclear and incompatible with explicit transaction managers.

Now, this method must never commit the transaction.

zope.generations.generations

Support for application database generations.

class zope.generations.generations.SchemaManager(minimum_generation=0, generation=0, package_name=None)

Bases: object

Schema manager

Schema managers implement IInstallableSchemaManager using scripts provided as module methods. You create a schema manager by providing mimumum and maximum generations and a package providing modules named evolveN, where N is a generation number. Each module provides a function, evolve that evolves a database from the previous generation.

For the sake of the example, we’ll use the demo package defined in here. See the modules there for simple examples of evolution scripts.

So, if we’ll create a SchemaManager:

>>> manager = SchemaManager(1, 3, 'zope.generations.demo')

and we’ll create a test database and context:

>>> from ZODB.MappingStorage import DB
>>> db = DB()
>>> context = Context()
>>> context.connection = db.open()

Then we’ll evolve the database from generation 1 to 3:

>>> import transaction
>>> tx = transaction.begin()
>>> manager.evolve(context, 2)
>>> manager.evolve(context, 3)
>>> tx.commit()

The demo evolvers simply record their data in a root key:

>>> from zope.generations.demo import key
>>> tx = transaction.begin()
>>> conn = db.open()
>>> conn.root()[key]
(2, 3)

You can get the information for each evolver by specifying the destination generation of the evolver as argument to the getInfo() method:

>>> manager.getInfo(1)
'Evolver 1'
>>> manager.getInfo(2)
'Evolver 2'
>>> manager.getInfo(3) is None
True

If a package provides an install script, then it will be called when the manager’s intall method is called:

>>> tx.abort()
>>> tx = transaction.begin()
>>> del conn.root()[key]
>>> tx.commit()
>>> tx = transaction.begin()
>>> conn.root().get(key)

>>> manager.install(context)
>>> tx.commit()
>>> tx = transaction.begin()
>>> conn.root()[key]
('installed',)

If there is not install script, the manager will do nothing on an install:

>>> manager = SchemaManager(1, 3, 'zope.generations.demo2')
>>> manager.install(context)

We handle ImportErrors within the script specially, so they get promoted:

>>> manager = SchemaManager(1, 3, 'zope.generations.demo3')
>>> manager.install(context)
Traceback (most recent call last):
ImportError: No module named nonexistingmodule

We’d better clean up:

>>> tx.abort()
>>> context.connection.close()
>>> conn.close()
>>> db.close()

evolve(context, generation)

Evolve a database to reflect software/schema changes.

install(context)

Evolve a database to reflect software/schema changes.

getInfo(generation)

Get the information from the evolver function’s doc string.

zope.generations.generations.EVOLVE = 'EVOLVE'

Constant for the how argument to evolve indicating to evolve to the current generation.

See also

evolveSubscriber

zope.generations.generations.EVOLVENOT = 'EVOLVENOT'

Constant for the how argument to evolve indicating to perform no evolutions.

See also

evolveNotSubscriber

zope.generations.generations.EVOLVEMINIMUM = 'EVOLVEMINIMUM'

Constant for the how argument to evolve indicating to evolve to the minimum required generation.

See also

evolveMinimumSubscriber

zope.generations.generations.evolve(db, how='EVOLVE')

Evolve a database

We evolve a database using registered application schema managers. Here’s an example (silly) schema manager:

>>> from zope.generations.interfaces import ISchemaManager
>>> @zope.interface.implementer(ISchemaManager)
... class FauxApp(object):
...
...     erron = None # Raise an error is asked to evolve to this
...
...     def __init__(self, name, minimum_generation, generation):
...         self.name, self.generation = name, generation
...         self.minimum_generation = minimum_generation
...
...     def evolve(self, context, generation):
...         if generation == self.erron:
...             raise ValueError(generation)
...
...         context.connection.root()[self.name] = generation

Evolving a database will cause log messages to be written, so we need a logging handler:

>>> from zope.testing import loggingsupport
>>> loghandler = loggingsupport.InstalledHandler('zope.generations')
>>> def print_log():
...    print(loghandler)
...    loghandler.clear()

Now, we’ll create and register some handlers:

>>> import zope.component
>>> app1 = FauxApp('app1', 0, 1)
>>> zope.component.provideUtility(app1, ISchemaManager, name='app1')
>>> app2 = FauxApp('app2', 5, 11)
>>> zope.component.provideUtility(app2, ISchemaManager, name='app2')

If we create a new database, and evolve it, we’ll simply update the generation data:

>>> from ZODB.MappingStorage import DB
>>> db = DB(database_name='testdb')
>>> evolve(db)
>>> conn = db.open()
>>> root = conn.root()
>>> root[generations_key]['app1']
1
>>> root[generations_key]['app2']
11

>>> print_log()
zope.generations INFO
  testdb: evolving in mode EVOLVE

But nothing will have been done to the database:

>>> root.get('app1')
>>> root.get('app2')

Now if we increase the generation of one of the apps:

>>> app1.generation += 1
>>> evolve(db)

>>> print_log()
zope.generations INFO
  testdb: evolving in mode EVOLVE
zope.generations INFO
  testdb/app1: currently at generation 1, targetting generation 2
zope.generations DEBUG
  testdb/app1: evolving to generation 2
zope.generations DEBUG
  testdb/app2: up-to-date at generation 11

We’ll see that the generation data has updated:

>>> root[generations_key]['app1']
2
>>> root[generations_key]['app2']
11

And that the database was updated for that application:

>>> root.get('app1')
2
>>> root.get('app2')

And that the transaction record got a note

>>> [it.description for it in conn.db().storage.iterator()][-1]
b'app1: evolving to generation 2'

If there is an error updating a particular generation, but the generation is greater than the minimum generation, then we won’t get an error from evolve, but we will get a log message.

>>> app1.erron = 4
>>> app1.generation = 7
>>> evolve(db)

>>> print_log()
zope.generations INFO
  testdb: evolving in mode EVOLVE
zope.generations INFO
  testdb/app1: currently at generation 2, targetting generation 7
zope.generations DEBUG
  testdb/app1: evolving to generation 3
zope.generations DEBUG
  testdb/app1: evolving to generation 4
zope.generations ERROR
  testdb/app1: failed to evolve to generation 4
zope.generations DEBUG
  testdb/app2: up-to-date at generation 11

The database will have been updated for previous generations:

>>> root[generations_key]['app1']
3
>>> root.get('app1')
3

If we set the minimum generation for app1 to something greater than 3:

>>> app1.minimum_generation = 4

Then we’ll get an error if we try to evolve, since we can’t get past 3 and 3 is less than 4:

>>> evolve(db)
Traceback (most recent call last):
...
UnableToEvolve: (4, u'app1', 7)

We’ll also get a log entry:

>>> print_log()
zope.generations INFO
  testdb: evolving in mode EVOLVE
zope.generations INFO
  testdb/app1: currently at generation 3, targetting generation 7
zope.generations DEBUG
  testdb/app1: evolving to generation 4
zope.generations ERROR
  testdb/app1: failed to evolve to generation 4

So far, we’ve used evolve in its default policy, in which we evolve as far as we can up to the current generation. There are two other policies:

EVOLVENOT – Don’t evolve, but make sure that the application is

at the minimum generation

EVOLVEMINIMUM – Evolve only to the minimum generation

Let’s change unset erron for app1 so we don’t get an error when we try to evolve.

>>> app1.erron = None

Now, we’ll call evolve with EVOLVENOT:

>>> evolve(db, EVOLVENOT)
Traceback (most recent call last):
...
GenerationTooLow: (3, u'app1', 4)

>>> print_log()
zope.generations INFO
  testdb: evolving in mode EVOLVENOT
zope.generations ERROR
  testdb/app1: current generation too low (3 < 4) but mode is EVOLVENOT

We got an error because we aren’t at the minimum generation for app1. The database generation for app1 is still 3 because we didn’t do any evolution:

>>> root[generations_key]['app1']
3
>>> root.get('app1')
3

Now, if we use EVOLVEMINIMUM instead, we’ll evolve to the minimum generation:

>>> evolve(db, EVOLVEMINIMUM)
>>> root[generations_key]['app1']
4
>>> root.get('app1')
4

>>> print_log()
zope.generations INFO
  testdb: evolving in mode EVOLVEMINIMUM
zope.generations INFO
  testdb/app1: currently at generation 3, targetting generation 4
zope.generations DEBUG
  testdb/app1: evolving to generation 4
zope.generations DEBUG
  testdb/app2: up-to-date at generation 11

If we happen to install an app that has a generation that is less than the database generation, we’ll get an error, because there is no way to get the database to a generation that the app understands:

>>> app1.generation = 2
>>> app1.minimum_generation = 0
>>> evolve(db)
Traceback (most recent call last):
...
GenerationTooHigh: (4, u'app1', 2)

>>> print_log()
zope.generations INFO
  testdb: evolving in mode EVOLVE
zope.generations ERROR
  testdb/app1: current generation too high (4 > 2)

We’d better clean up:

>>> from zope.testing.cleanup import tearDown
>>> loghandler.uninstall()
>>> conn.close()
>>> db.close()
>>> tearDown()

zope.generations.generations.evolveSubscriber(event)

A subscriber for zope.processlifetime.IDatabaseOpenedWithRoot that evolves all components to their current generation.

If you want to use this subscriber, you must register it.

zope.generations.generations.evolveNotSubscriber(event)

A subscriber for zope.processlifetime.IDatabaseOpenedWithRoot that does no evolution, merele verifying that all components are at the required minimum version.

If you want to use this subscriber, you must register it.

zope.generations.generations.evolveMinimumSubscriber(event)

A subscriber for zope.processlifetime.IDatabaseOpenedWithRoot that evolves all components to their required minimum version.

This is registered in this package’s subscriber.zcml file.

zope.generations.utility

Utility functions for evolving database generations.

zope.generations.utility.findObjectsMatching(root, condition)

Find all objects in the root that match the condition.

The condition is a callable Python object that takes an object as an argument and must return True or False.

All sub-objects of the root will also be searched recursively. All mapping objects providing values() are supported.

Example:

>>> class A(dict):
...     def __init__(self, name):
...         self.name = name
>>> class B(dict):
...     def __init__(self, name):
...         self.name = name
>>> class C(dict):
...     def __init__(self, name):
...         self.name = name
>>> tree = A('a1')
>>> tree['b1'] = B('b1')
>>> tree['c1'] = C('c1')
>>> tree['b1']['a2'] = A('a2')
>>> tree['b1']['b2'] = B('b2')
>>> tree['b1']['b2']['c2'] = C('c2')
>>> tree['b1']['b2']['a3'] = A('a3')

Find all instances of class A:

>>> matches = findObjectsMatching(tree, lambda x: isinstance(x, A))
>>> names = [x.name for x in matches]
>>> names.sort()
>>> names
['a1', 'a2', 'a3']

Find all objects having a ‘2’ in the name:

>>> matches = findObjectsMatching(tree, lambda x: '2' in x.name)
>>> names = [x.name for x in matches]
>>> names.sort()
>>> names
['a2', 'b2', 'c2']

If there is no values on the root, we stop:

>>> root = [1, 2, 3]
>>> found = list(findObjectsMatching(root, lambda x: True))
>>> found == [root]
True

zope.generations.utility.findObjectsProviding(root, interface)

Find all objects in the root that provide the specified interface.

All sub-objects of the root will also be searched recursively.

Example:

>>> from zope.interface import Interface, implementer
>>> class IA(Interface):
...     pass
>>> class IB(Interface):
...     pass
>>> class IC(IA):
...     pass
>>> @implementer(IA)
... class A(dict):
...     def __init__(self, name):
...         self.name = name
>>> @implementer(IB)
... class B(dict):
...     def __init__(self, name):
...         self.name = name
>>> @implementer(IC)
... class C(dict):
...     def __init__(self, name):
...         self.name = name
>>> tree = A('a1')
>>> tree['b1'] = B('b1')
>>> tree['c1'] = C('c1')
>>> tree['b1']['a2'] = A('a2')
>>> tree['b1']['b2'] = B('b2')
>>> tree['b1']['b2']['c2'] = C('c2')
>>> tree['b1']['b2']['a3'] = A('a3')

Find all objects that provide IB:

>>> matches = findObjectsProviding(tree, IB)
>>> names = [x.name for x in matches]
>>> names.sort()
>>> names
['b1', 'b2']

Find all objects that provide IA:

>>> matches = findObjectsProviding(tree, IA)
>>> names = [x.name for x in matches]
>>> names.sort()
>>> names
['a1', 'a2', 'a3', 'c1', 'c2']

zope.generations.utility.ROOT_NAME = 'Application'

The name of the root folder. If zope.app.publication.zopepublication is available, this is imported from there.

zope.generations.utility.getRootFolder(context)

Get the root folder of the ZODB.

We need some set up. Create a database:

>>> from ZODB.MappingStorage import DB
>>> from zope.generations.generations import Context
>>> import transaction
>>> db = DB()
>>> context = Context()
>>> tx = transaction.begin()
>>> context.connection = db.open()
>>> root = context.connection.root()

Add a root folder:

>>> from zope.site.folder import rootFolder
>>> root[ROOT_NAME] = rootFolder()
>>> tx.commit()
>>> tx = transaction.begin()

Now we can get the root folder using the function:

>>> getRootFolder(context) 
<zope.site.folder.Folder object at ...>

We’d better clean up:

>>> tx.abort()
>>> context.connection.close()
>>> db.close()

Configuration

There are two ZCML files included in this package.

configure.zcml

<configure xmlns="http://namespaces.zope.org/zope">

  <!--
  <utility
      name="zope.app"
      provides=".interfaces.ISchemaManager"
      factory=".generations.SchemaManager"
      >

      Provide an *initial* schema manager for zope.

      We can use a factory here, because the generation is 0.

      When we get to generation 1, we'll have to actually create
      a manager instance with the necessary parameters and a package of
      evolution scripts.
  </utility>
  -->

  <!-- Registering documentation with API doc -->
  <configure
      xmlns:apidoc="http://namespaces.zope.org/apidoc"
      xmlns:zcml="http://namespaces.zope.org/zcml"
      zcml:condition="have apidoc">

    <apidoc:bookchapter
        id="generations"
        title="Generations"
        doc_path="README.txt"
        />

  </configure>

</configure>

subscriber.zcml

<configure xmlns="http://namespaces.zope.org/zope">

  <include package="zope.component" file="meta.zcml" />

<!--
<subscriber
    handler=".generations.evolveSubscriber"
    for="zope.processlifetime.IDatabaseOpenedWithRoot"
    >
    Evolve to current generation on startup
</subscriber>
-->

<!--
<subscriber
    handler=".generations.evolveNotSubscriber"
    for="zope.processlifetime.IDatabaseOpenedWithRoot"
    >
    Don't evolve, but check for minimum generations on startup
</subscriber>
-->

<subscriber
    handler=".generations.evolveMinimumSubscriber"
    for="zope.processlifetime.IDatabaseOpenedWithRoot"
    >
    Only evolve to minimum generations on startup
</subscriber>

</configure>