Validating requests data using a schema is a powerful pattern.
As you would do for a database table, you define some fields and their type, and make sure that incoming requests comply.
There are many schema libraries in the Python ecosystem you can use. The most known ones are Colander, Marshmallow & formencode.
You can do schema validation using either those libraries or either custom code.
Using a schema is done in 2 steps:
1/ linking a schema to your service definition 2/ implement a validator that uses the schema to verify the request
Here’s a dummy example:
def my_validator(request, **kwargs):
schema = kwargs['schema']
# do something with the schema
schema = {'id': int, 'name': str}
@service.post(schema=schema, validators=(my_validator,))
def post(request):
return {'OK': 1}
Cornice will call my_validator
with the incoming request, and will
provide the schema in the keywords.
Colander (http://docs.pylonsproject.org/projects/colander/en/latest/) is a validation framework from the Pylons project that can be used with Cornice’s validation hook to control a request and deserialize its content into objects.
Cornice provides a helper to ease Colander integration.
To describe a schema, using Colander and Cornice, here is how you can do:
import colander
from cornice import Service
from cornice.validators import colander_body_validator
class SignupSchema(colander.MappingSchema):
username = colander.SchemaNode(colander.String())
@signup.post(schema=SignupSchema(), validators=(colander_body_validator,))
def signup_post(request):
username = request.validated['username']
return {'success': True}
Note
When you use one of colander_body_validator
, colander_headers_validator
,
colander_querystring_validator
etc. it is necessary to set schema which
inherits from colander.MappingSchema
. If you need to deserialize
colander.SequenceSchema
you need to use colander_validator
instead.
Marshmallow (https://marshmallow.readthedocs.io/en/latest/) is an ORM/ODM/framework-agnostic library for converting complex datatypes, such as objects, to and from native Python datatypes that can also be used with Cornice validation hooks.
Cornice provides a helper to ease Marshmallow integration.
To describe a schema, using Marshmallow and Cornice, here is how you can do:
import marshmallow
from cornice import Service
from cornice.validators import marshmallow_body_validator
class SignupSchema(marshmallow.Schema):
username = marshmallow.fields.String(required=True)
@signup.post(schema=SignupSchema, validators=(marshmallow_body_validator,))
def signup_post(request):
username = request.validated['username']
return {'success': True}
If you want to do specific things with the schema at validation step, like having a schema per request method, you can provide whatever you want as the schema key and built a custom validator.
Example:
def dynamic_schema(request):
if request.method == 'POST':
schema = foo_schema()
elif request.method == 'PUT':
schema = bar_schema()
return schema
def my_validator(request, **kwargs):
kwargs['schema'] = dynamic_schema(request)
return colander_body_validator(request, **kwargs)
@service.post(validators=(my_validator,))
def post(request):
return request.validated
In addition to colander_body_validator()
as demonstrated above, there are also three more
similar validators, colander_headers_validator()
, colander_path_validator()
, and
colander_querystring_validator()
(and similarly named marshmallow_*
functions), which validate the given Schema
against the headers, path,
or querystring parameters, respectively.
If you have complex use-cases where data has to be validated accross several locations
of the request (like querystring, body etc.), Cornice provides a validator that
takes an additionnal level of mapping for body
, querystring
, path
or headers
instead of the former location
attribute on schema fields.
The request.validated
hences reflects this additional level.
# colander
from cornice.validators import colander_validator
class Querystring(colander.MappingSchema):
referrer = colander.SchemaNode(colander.String(), missing=colander.drop)
class Payload(colander.MappingSchema):
username = colander.SchemaNode(colander.String())
class SignupSchema(colander.MappingSchema):
body = Payload()
querystring = Querystring()
signup = cornice.Service()
@signup.post(schema=SignupSchema(), validators=(colander_validator,))
def signup_post(request):
username = request.validated['body']['username']
referrer = request.validated['querystring']['referrer']
return {'success': True}
# marshmallow
from cornice.validators import marshmallow_validator
class Querystring(marshmallow.Schema):
referrer = marshmallow.fields.String()
class Payload(marshmallow.Schema):
username = marshmallow.fields.String(validate=[
marshmallow.validate.Length(min=3)
], required=True)
class SignupSchema(marshmallow.Schema):
body = marshmallow.fields.Nested(Payload)
querystring = marshmallow.fields.Nested(Querystring)
@signup.post(schema=SignupSchema, validators=(marshmallow_validator,))
def signup_post(request):
username = request.validated['body']['username']
referrer = request.validated['querystring']['referrer']
return {'success': True}
This allows to have validation at the schema level that validates data from several places on the request:
# colander
class SignupSchema(colander.MappingSchema):
body = Payload()
querystring = Querystring()
def deserialize(self, cstruct=colander.null):
appstruct = super(SignupSchema, self).deserialize(cstruct)
username = appstruct['body']['username']
referrer = appstruct['querystring'].get('referrer')
if username == referrer:
self.raise_invalid('Referrer cannot be the same as username')
return appstruct
# marshmallow
class SignupSchema(marshmallow.Schema):
body = marshmallow.fields.Nested(Payload)
querystring = marshmallow.fields.Nested(Querystring)
@marshmallow.validates_schema(skip_on_field_errors=True)
def validate_multiple_fields(self, data):
username = data['body'].get('username')
referrer = data['querystring'].get('referrer')
if username == referrer:
raise marshmallow.ValidationError(
'Referrer cannot be the same as username')
Cornice provides built-in support for JSON and HTML forms
(application/x-www-form-urlencoded
) input validation using the provided
validators.
If you need to validate other input formats, such as XML, you need to implement your own deserializer and pass it to the service.
The general pattern in this case is:
from cornice.validators import colander_body_validator
def my_deserializer(request):
return extract_data_somehow(request)
@service.post(schema=MySchema(),
deserializer=my_deserializer,
validators=(colander_body_validator,))
def post(request):
return {'OK': 1}
Marshmallow schemas have access to request as context object which can be handy for things like CSRF validation:
class MNeedsContextSchema(marshmallow.Schema):
somefield = marshmallow.fields.Float(missing=lambda: random.random())
csrf_secret = marshmallow.fields.String()
@marshmallow.validates_schema
def validate_csrf_secret(self, data):
# simulate validation of session variables
if self.context['request'].get_csrf() != data.get('csrf_secret'):
raise marshmallow.ValidationError('Wrong token')
FormEncode (http://www.formencode.org/en/latest/index.html) is yet another validation system that can be used with Cornice.
For example, if you want to make sure the optional query option max is an integer, and convert it, you can use FormEncode in a Cornice validator like this:
from formencode import validators
from cornice import Service
from cornice.validators import extract_cstruct
foo = Service(name='foo', path='/foo')
def form_validator(request, **kwargs):
data = extract_cstruct(request)
validator = validators.Int()
try:
max = data['querystring'].get('max')
request.validated['max'] = validator.to_python(max)
except formencode.Invalid, e:
request.errors.add('querystring', 'max', e.message)
@foo.get(validators=(form_validator,))
def get_value(request):
"""Returns the value.
"""
return {'posted': request.validated}
Several libraries exist in the wild to validate data in Python and that can easily be plugged with Cornice.