mozilla

Schema validation

Validating requests data using a schema is a powerful pattern.

As you would do for a database table, you define some fields and their type, and make sure that incoming requests comply.

There are many schema libraries in the Python ecosystem you can use. The most known ones are Colander & formencode.

You can do schema validation using either those libraries or either custom code.

Using a schema is done in 2 steps:

1/ linking a schema to your service definition 2/ implement a validator that uses the schema to verify the request

Here’s a dummy example:

def my_validator(request, **kwargs):
    schema = kwargs['schema']
    # do something with the schema

schema = {'id': int, 'name': str}

@service.post(schema=schema, validators=(my_validator,))
def post(request):
    return {'OK': 1}

Cornice will call my_validator with the incoming request, and will provide the schema in the keywords.

Using Colander

Colander (http://docs.pylonsproject.org/projects/colander/en/latest/) is a validation framework from the Pylons project that can be used with Cornice’s validation hook to control a request and deserialize its content into objects.

Cornice provides a helper to ease Colander integration.

To describe a schema, using Colander and Cornice, here is how you can do:

import colander

from cornice import Service
from cornice.validators import colander_body_validator

class SignupSchema(colander.MappingSchema):
    username = colander.SchemaNode(colander.String())

@signup.post(schema=SignupSchema(), validators=(colander_body_validator,))
def signup_post(request):
    username = request.validated['username']
    return {'success': True}

Dynamic schemas

If you want to do specific things with the schema at validation step, like having a schema per request method, you can provide whatever you want as the schema key and built a custom validator.

Example:

def dynamic_schema(request):
    if request.method == 'POST':
        schema = foo_schema
    elif request.method == 'PUT':
        schema = bar_schema
    return schema


def my_validator(request, **kwargs):
    kwargs['schema'] = dynamic_schema(request)
    return colander_body_validator(request, **kwargs)


@service.post(validators=(my_validator,))
def post(request):
    return request.validated

Multiple request attributes

If you have complex use-cases where data has to be validated accross several locations of the request (like querystring, body etc.), Cornice provides a validator that takes an additionnal level of mapping for body, querystring, path or headers instead of the former location attribute on schema fields.

The request.validated hences reflects this additional level.

from cornice.validators import colander_validator

class Querystring(colander.MappingSchema):
    referrer = colander.SchemaNode(colander.String(), missing=colander.drop)

class Payload(colander.MappingSchema):
    username = colander.SchemaNode(colander.String())

class SignupSchema(colander.MappingSchema):
    body = Payload()
    querystring = Querystring()

signup = cornice.Service()

@signup.post(schema=SignupSchema(), validators=(colander_validator,))
def signup_post(request):
    username = request.validated['body']['username']
    referrer = request.validated['querystring']['referrer']
    return {'success': True}

This allows to have validation at the schema level that validates data from several places on the request:

class SignupSchema(colander.MappingSchema):
    body = Payload()
    querystring = Querystring()

    def deserialize(self, cstruct=colander.null):
        appstruct = super(SignupSchema, self).deserialize(cstruct)
        username = appstruct['body']['username']
        referrer = appstruct['querystring'].get('referrer')
        if username == referred:
            self.raise_invalid('Referrer cannot be the same as username')
        return appstruct

Cornice provides built-in support for JSON and HTML forms (application/x-www-form-urlencoded) input validation using the provided colander validators.

If you need to validate other input formats, such as XML, you need to implement your own deserializer and pass it to the service.

The general pattern in this case is:

from cornice.validators import colander_body_validator

def my_deserializer(request):
    return extract_data_somehow(request)


@service.post(schema=MySchema(),
              deserializer=my_deserializer,
              validators=(colander_body_validator,))
def post(request):
    return {'OK': 1}

Using formencode

FormEncode (http://www.formencode.org/en/latest/index.html) is yet another validation system that can be used with Cornice.

For example, if you want to make sure the optional query option max is an integer, and convert it, you can use FormEncode in a Cornice validator like this:

from formencode import validators

from cornice import Service
from cornice.validators import extract_cstruct

foo = Service(name='foo', path='/foo')

def form_validator(request, **kwargs):
    data = extract_cstruct(request)
    validator = validators.Int()
    try:
        max = data['querystring'].get('max')
        request.validated['max'] = validator.to_python(max)
    except formencode.Invalid, e:
        request.errors.add('querystring', 'max', e.message)

@foo.get(validators=(form_validator,))
def get_value(request):
    """Returns the value.
    """
    return {'posted': request.validated}

See also

Several libraries exist in the wild to validate data in Python and that can easily be plugged with Cornice.