auxlib.Entity.json() - Code Metrics - Inspection of "improve entity docstrings" - kalefranz/auxlib - Measure and Improve Code Quality continuously with Scrutinizer

Completed

Push — develop ( 1f233d...105c0e )

by Kale

created 2016-03-23 08:58 UTC

auxlib.Entity.json() A

↳ Parent: auxlib.Entity

Complexity

Conditions

Size

Total Lines

Duplication

Lines	0
Ratio	0 %

Metric	Value
cc	1
dl	0
loc	3
rs	10

# -*- coding: utf-8 -*-
"""This module provides facilities for serializable, validatable, and type-enforcing
domain objects.

This module has many of the same motivations as the python Marshmallow package.
<http://marshmallow.readthedocs.org/en/latest/why.html>

Also need to be explicit in explaining what Marshmallow doesn't do, and why this module is needed.
  - Provides type safety like an ORM. And like an ORM, all classes subclass Entity.
  - Provides BUILT IN serialization and deserialization.  Marhmallow requires a lot of code
    duplication.

This module gives us:
  - type safety
  - custom field validation
  - serialization and deserialization
  - rock solid foundational domain objects




Optional Field Properties:
  - validation = None
  - default = None
  - required = True
  - in_dump = True
  - nullable = False

Behaviors:
  - Nullable is a "hard" setting, in that the value is either always or never allowed to be None.
  - What happens then if required=False and nullable=False?
      - The object can be init'd without a value (though not with a None value).
        getattr throws AttributeError
      - Any assignment must be not None.


  - Setting a value to None doesn't "unset" a value.  (That's what del is for.)  And you can't
    del a value if required=True, nullable=False, default=None.  That should raise
    OperationNotAllowedError.


  - Disabling in_dump is a "hard" setting, in that with it disabled the field will never get
    dumped.  With it enabled, the field may or may not be dumped depending on its value and other
    settings.

  - Required is a "hard" setting, in that if True, a valid value or default must be provided. None
    is only a valid value or default if nullable is True.

  - In general, nullable means that None is a valid value.
    - getattr returns None instead of raising Attribute error
    - If in_dump, field is given with null value.
    - If default is not None, assigning None clears a previous assignment. Future getattrs return
      the default value.
    - What does nullable mean with default=None and required=True? Does instantiation raise
      an error if assignment not made on init? Can IntField(nullable=True) be init'd?

  - If required=False and nullable=False, field will only be in dump if field!=None.
    Also, getattr raises AttributeError.
  - If required=False and nullable=True, field will be in dump if field==None.

  - What does required and default mean?
  - What does required and default and nullable mean?
  - If in_dump is True, does default value get dumped:
    - if no assignment, default exists
    - if nullable, and assigned None
  - Can assign None?
  - How does optional validation work with nullable and assigning None?
  - When does gettattr throw AttributeError, and when does it return None?





Examples:
    >>> class Color(Enum):
    ...     blue = 0
    ...     black = 1
    ...     red = 2
    >>> class Car(Entity):
    ...     color = EnumField(Color)
    ...     weight = NumberField(required=False)
    ...     wheels = IntField(default=4, validation=lambda x: 3 <= x <= 4)

    >>> # create a new car object
    >>> car = Car(color=Color.blue, weight=4242.42)

    >>> # it has 4 wheels, all by default
    >>> car.wheels
    4

    >>> # but a car can't have 5 wheels!
    >>> car.wheels = 5
    Traceback (most recent call last):
    ...
    auxlib.exceptions.ValidationError: Invalid value 5 for wheels

    # we can call .dump() on car, and just get back a simple python dict
    >>> type(car.dump())
    <class 'dict'>
    >>> sorted(car.dump().items())
    [('color', 0), ('weight', 4242.42), ('wheels', 4)]

    # and json too
    >>> car.json(sort_keys=True)
    '{"color": 0, "weight": 4242.42, "wheels": 4}'

    # green cars aren't allowed
    >>> car.color = "green"
    Traceback (most recent call last):
    ...
    auxlib.exceptions.ValidationError: 'green' is not a valid Color

    # but black cars are!
    >>> car.color = "black"
    >>> car.color
    <Color.black: 1>

    # car.color really is an enum, promise
    >>> type(car.color)
    <enum 'Color'>

    # let's do a round-trip marshalling of this thing
    >>> same_car = Car.from_json(json.dumps(car.dump()))  # or equally Car.from_json(car.json()))
    >>> same_car == car
    True

    # actually, they're two different instances
    >>> same_car is not car
    True

    # this works too
    >>> cloned_car = Car(**car.dump())
    >>> cloned_car == car
    True

    # now let's get fancy
    >>> class Fleet(Entity):
    ...     cars = ListField(Car)
    ...     boss_car = ComposableField(Car)

    # here's our fleet of company cars
    >>> company_fleet = Fleet(boss_car=Car(color='red'), cars=[car, same_car, cloned_car])
    >>> company_fleet.pretty_json()  #doctest: +SKIP
    {
      "boss_car": {
        "color": 2,
        "wheels": 4
      },
      "cars": [
        {
          "color": 1,
          "weight": 4242.42,
          "wheels": 4
        },
        {
          "color": 1,
          "weight": 4242.42,
          "wheels": 4
        },
        {
          "color": 1,
          "weight": 4242.42,
          "wheels": 4
        }
      ]
    }

    # the boss' car is red of course (and it's still an Enum)
    >>> company_fleet.boss_car.color.name
    'red'

    # and there are three cars left for the employees
    >>> len(company_fleet.cars)
    3

    # because we can
    >>> sum(c.weight for c in company_fleet.cars)
    12727.26



"""
from __future__ import absolute_import, division, print_function

from datetime import datetime
from functools import reduce
from json import loads as json_loads, dumps as json_dumps
from logging import getLogger
import collections
import json

from enum import Enum

from ._vendor.dateutil.parser import parse as dateparser
from ._vendor.five import with_metaclass, items, values
from ._vendor.six import integer_types, string_types
from .collection import AttrDict
from .exceptions import ValidationError, Raise
from .type_coercion import maybecall

log = getLogger(__name__)

KEY_OVERRIDES_MAP = "__key_overrides__"


class Field(object):
    """
    Fields are doing something very similar to boxing and unboxing
    of c#/java primitives.  __set__ should take a "primitve" or "raw" value and create a "boxed"
    or "programatically useable" value of it.  While __get__ should return the boxed value,
    dump in turn should unbox the value into a primitive or raw value.

    Arguments:
        types_ (primitive literal or type or sequence of types):
        default (any, callable, optional):  If default is callable, it's guaranteed to return a
            valid value at the time of Entity creation.
        required (boolean, optional):
        validation (callable, optional):
        dump (boolean, optional):
    """

    def __init__(self, default=None, required=True, validation=None, in_dump=True, nullable=False):
        self._default = default if callable(default) else self.box(default)
        self._required = required
        self._validation = validation
        self._in_dump = in_dump
        self._nullable = nullable
        if default is not None:
            self.validate(self.box(maybecall(default)))

    @property
    def name(self):
        try:
            return self._name
        except AttributeError:
            log.error("The name attribute has not been set for this field. "
                      "Call set_name at class creation time.")
            raise

    def set_name(self, name):
        self._name = name
        return self

    def __get__(self, instance, instance_type):
        try:
            if instance is None:  # if calling from the class object
                val = getattr(instance_type, KEY_OVERRIDES_MAP)[self.name]
            else:
                val = instance.__dict__[self.name]
        except AttributeError:
            log.error("The name attribute has not been set for this field.")
            raise
        except KeyError:
            if self.default is not None:
                val = maybecall(self.default)  # default *can* be a callable
            elif self._nullable:
                return None
            else:
                raise AttributeError("A value for {0} has not been set".format(self.name))
        return self.unbox(val)

    def __set__(self, instance, val):
        # validate will raise an exception if invalid
        # validate will return False if the value should be removed
        instance.__dict__[self.name] = self.validate(self.box(val))

    def __delete__(self, instance):
        instance.__dict__.pop(self.name, None)

    def box(self, val):
        return val

    def unbox(self, val):
        return val

    def dump(self, val):
        return val

    def validate(self, val):
        """

        Returns:
            True: if val is valid

        Raises:
            ValidationError
        """
        # note here calling, but not assigning; could lead to unexpected behavior
        if isinstance(val, self._type) and (self._validation is None or self._validation(val)):
                return val
        elif val is None and self.nullable:
            return val
        raise ValidationError(getattr(self, 'name', 'undefined name'), val)

    @property
    def required(self):
        return self.is_required

    @property
    def is_required(self):
        return self._required

    @property
    def is_enum(self):
        return isinstance(self._type, type) and issubclass(self._type, Enum)

    @property
    def type(self):
        return self._type

    @property
    def default(self):
        return self._default

    @property
    def in_dump(self):
        return self._in_dump

    @property
    def nullable(self):
        return self.is_nullable

    @property
    def is_nullable(self):
        return self._nullable


class IntField(Field):
    _type = integer_types


class StringField(Field):
    _type = string_types


class NumberField(Field):
    _type = integer_types + (float, complex)


class BooleanField(Field):
    _type = bool

    def box(self, val):
        return None if val is None else bool(val)


class DateField(Field):
    _type = datetime

    def box(self, val):
        try:
            return dateparser(val) if isinstance(val, string_types) else val
        except ValueError as e:
            raise ValidationError(val, msg=e)

    def dump(self, val):
        return None if val is None else val.isoformat()


class EnumField(Field):

    def __init__(self, enum_class, default=None, required=True, validation=None,
                 in_dump=True, nullable=False):
        if not issubclass(enum_class, Enum):
            raise ValidationError(None, msg="enum_class must be an instance of Enum")
        self._type = enum_class
        super(self.__class__, self).__init__(default, required, validation, in_dump, nullable)

    def box(self, val):
        if val is None:
            # let the required/nullable logic handle validation for this case
            return None
        try:
            # try to box using val as an Enum name
            return val if isinstance(val, self._type) else self._type(val)
        except ValueError as e1:
            try:
                # try to box using val as an Enum value
                return self._type[val]
            except KeyError:
                raise ValidationError(val, msg=e1)

    def dump(self, val):
        return None if val is None else val.value


class ListField(Field):
    _type = tuple

    def __init__(self, element_type, default=None, required=True, validation=None,
                 in_dump=True, nullable=False):
        self._element_type = element_type
        super(self.__class__, self).__init__(default, required, validation, in_dump, nullable)

    def box(self, val):
        if val is None:
            return None
        elif isinstance(val, string_types):
            raise ValidationError("Attempted to assign a string to ListField {0}"
                                  "".format(self.name))
        elif isinstance(val, collections.Iterable):
            et = self._element_type
            if isinstance(et, type) and issubclass(et, Entity):
                return tuple(v if isinstance(v, et) else et(**v) for v in val)
            else:
                return tuple(val)
        else:
            raise ValidationError(val, msg="Cannot assign a non-iterable value to "
                                           "{0}".format(self.name))

    def unbox(self, val):
        return tuple() if val is None and not self.nullable else val

    def dump(self, val):
        if isinstance(self._element_type, type) and issubclass(self._element_type, Entity):
            return tuple(v.dump() for v in val)
        else:
            return val

    def validate(self, val):
        if val is None:
            if not self.nullable:
                raise ValidationError(self.name, val)
            return None
        else:
            val = super(self.__class__, self).validate(val)
            et = self._element_type
            tuple(Raise(ValidationError(self.name, el, et)) for el in val
                  if not isinstance(el, et))
            return val


class MapField(Field):
    _type = dict
    __eq__ = dict.__eq__
    __hash__ = dict.__hash__


class ComposableField(Field):

    def __init__(self, field_class, default=None, required=True, validation=None,
                 in_dump=True, nullable=False):
        self._type = field_class
        super(self.__class__, self).__init__(default, required, validation, in_dump, nullable)

    def box(self, val):
        if val is None:
            return None
        if isinstance(val, self._type):
            return val
        else:
            # assuming val is a dict now
            try:
                # if there is a key named 'self', have to rename it
                val['slf'] = val.pop('self')
            except KeyError:
                pass  # no key of 'self', so no worries
            return val if isinstance(val, self._type) else self._type(**val)

    def dump(self, val):
        return None if val is None else val.dump()


class EntityType(type):

    @staticmethod
    def __get_entity_subclasses(bases):
        try:
            return [base for base in bases if issubclass(base, Entity) and base is not Entity]
        except NameError:
            # NameError: global name 'Entity' is not defined
            return []

    def __new__(mcs, name, bases, dct):
        # if we're about to mask a field that's already been created with something that's
        #  not a field, then assign it to an alternate variable name
        non_field_keys = (key for key, value in items(dct)
                          if not isinstance(value, Field) and not key.startswith('__'))
        entity_subclasses = EntityType.__get_entity_subclasses(bases)
        if entity_subclasses:
            keys_to_override = [key for key in non_field_keys
                                if any(isinstance(base.__dict__.get(key), Field)
                                       for base in entity_subclasses)]
            dct[KEY_OVERRIDES_MAP] = {key: dct.pop(key) for key in keys_to_override}
        else:
            dct[KEY_OVERRIDES_MAP] = dict()

        return super(EntityType, mcs).__new__(mcs, name, bases, dct)

    def __init__(cls, name, bases, attr):
        super(EntityType, cls).__init__(name, bases, attr)
        cls.__fields__ = dict(cls.__fields__) if hasattr(cls, '__fields__') else dict()
        cls.__fields__.update({name: field.set_name(name)
                               for name, field in cls.__dict__.items()
                               if isinstance(field, Field)})
        if hasattr(cls, '__register__'):
            cls.__register__()

    @property
    def fields(cls):
        return cls.__fields__.keys()


@with_metaclass(EntityType)
class Entity(object):
    __fields__ = dict()
    # TODO: add arg order to fields like in enum34

    def __init__(self, **kwargs):
        for key, field in items(self.__fields__):
            try:
                setattr(self, key, kwargs[key])
            except KeyError:
                # handle the case of fields inherited from subclass but overrode on class object
                if key in getattr(self, KEY_OVERRIDES_MAP):
                    setattr(self, key, getattr(self, KEY_OVERRIDES_MAP)[key])
                elif field.required and field.default is None:
                    raise ValidationError(key, msg="{0} requires a {1} field. Instantiated with "
                                                   "{2}".format(self.__class__.__name__,
                                                                key, kwargs))
            except ValidationError:
                if kwargs[key] is not None or field.required:
                    raise
        self.validate()

    @classmethod
    def create_from_objects(cls, *objects, **override_fields):
        init_vars = dict()
        search_maps = (AttrDict(override_fields), ) + objects
        for key, field in items(cls.__fields__):
            value = find_or_none(key, search_maps)
            if value is not None or field.required:
                init_vars[key] = field.type(value) if field.is_enum else value
        return cls(**init_vars)

    @classmethod
    def from_json(cls, json_str):
        return cls(**json_loads(json_str))

    @classmethod
    def load(cls, data_dict):
        return cls(**data_dict)

    def validate(self):
        # TODO: here, validate should only have to determine if the required keys are set
        try:
            reduce(lambda x, y: y, (getattr(self, name)
                                    for name, field in self.__fields__.items()
                                    if field.required))
        except AttributeError as e:
            raise ValidationError(None, msg=e)
        except TypeError as e:
            if "no initial value" in str(e):
                # TypeError: reduce() of empty sequence with no initial value
                pass
            else:
                raise  # pragma: no cover

    def __repr__(self):
        def _repr(val):
            return repr(val.value) if isinstance(val, Enum) else repr(val)
        return "{0}({1})".format(
            self.__class__.__name__,
            ", ".join("{0}={1}".format(key, _repr(value)) for key, value in self.__dict__.items()))

    @classmethod
    def __register__(cls):
        pass

    def json(self, indent=None, separators=None, sort_keys=False, **kwargs):
        return json_dumps(self.dump(), indent=indent, separators=separators,
                          sort_keys=sort_keys, **kwargs)

    def pretty_json(self, indent=2, separators=(',', ': '), sort_keys=True, **kwargs):
        return self.json(indent=indent, separators=separators, sort_keys=sort_keys, **kwargs)

    def dump(self):
        return {field.name: field.dump(value)
                for field, value in ((field, getattr(self, field.name, None))
                                     for field in self.__dump_fields())
                if value is not None or field.nullable}

    @classmethod
    def __dump_fields(cls):
        if '__dump_fields_cache' not in cls.__dict__:
            cls.__dump_fields_cache = {field for field in values(cls.__fields__) if field.in_dump}
        return cls.__dump_fields_cache

    def __eq__(self, other):
        if self.__class__ != other.__class__:
            return False
        rando_default = 19274656290  # need an arbitrary but definite value if field does not exist
        return all(getattr(self, field, rando_default) == getattr(other, field, rando_default)
                   for field in self.__fields__)

    def __hash__(self):
        return sum(hash(getattr(self, field, None)) for field in self.__fields__)


def find_or_none(key, search_maps, map_index=0):
    """Return the value of the first key found in the list of search_maps,
    otherwise return None.

    Examples:
        >>> d1 = AttrDict({'a': 1, 'b': 2, 'c': 3, 'e': None})
        >>> d2 = AttrDict({'b': 5, 'e': 6, 'f': 7})
        >>> find_or_none('c', (d1, d2))
        3
        >>> find_or_none('f', (d1, d2))
        7
        >>> find_or_none('b', (d1, d2))
        2
        >>> print(find_or_none('g', (d1, d2)))
        None
        >>> find_or_none('e', (d1, d2))
        6

    """
    try:
        attr = getattr(search_maps[map_index], key)
        return attr if attr is not None else find_or_none(key, search_maps[1:])
    except AttributeError:
        # not found in first map object, so go to next
        return find_or_none(key, search_maps, map_index+1)
    except IndexError:
        # ran out of map objects to search
        return None


1			# -- coding: utf-8 --
2			"""This module provides facilities for serializable, validatable, and type-enforcing
3			domain objects.
4
5			This module has many of the same motivations as the python Marshmallow package.
6			<http://marshmallow.readthedocs.org/en/latest/why.html>
7
8			Also need to be explicit in explaining what Marshmallow doesn't do, and why this module is needed.
9			- Provides type safety like an ORM. And like an ORM, all classes subclass Entity.
10			- Provides BUILT IN serialization and deserialization. Marhmallow requires a lot of code
11			duplication.
12
13			This module gives us:
14			- type safety
15			- custom field validation
16			- serialization and deserialization
17			- rock solid foundational domain objects
18
19
20
21
22			Optional Field Properties:
23			- validation = None
24			- default = None
25			- required = True
26			- in_dump = True
27			- nullable = False
28
29			Behaviors:
30			- Nullable is a "hard" setting, in that the value is either always or never allowed to be None.
31			- What happens then if required=False and nullable=False?
32			- The object can be init'd without a value (though not with a None value).
33			getattr throws AttributeError
34			- Any assignment must be not None.
35
36
37			- Setting a value to None doesn't "unset" a value. (That's what del is for.) And you can't
38			del a value if required=True, nullable=False, default=None. That should raise
39			OperationNotAllowedError.
40
41
42			- Disabling in_dump is a "hard" setting, in that with it disabled the field will never get
43			dumped. With it enabled, the field may or may not be dumped depending on its value and other
44			settings.
45
46			- Required is a "hard" setting, in that if True, a valid value or default must be provided. None
47			is only a valid value or default if nullable is True.
48
49			- In general, nullable means that None is a valid value.
50			- getattr returns None instead of raising Attribute error
51			- If in_dump, field is given with null value.
52			- If default is not None, assigning None clears a previous assignment. Future getattrs return
53			the default value.
54			- What does nullable mean with default=None and required=True? Does instantiation raise
55			an error if assignment not made on init? Can IntField(nullable=True) be init'd?
56
57			- If required=False and nullable=False, field will only be in dump if field!=None.
58			Also, getattr raises AttributeError.
59			- If required=False and nullable=True, field will be in dump if field==None.
60
61			- What does required and default mean?
62			- What does required and default and nullable mean?
63			- If in_dump is True, does default value get dumped:
64			- if no assignment, default exists
65			- if nullable, and assigned None
66			- Can assign None?
67			- How does optional validation work with nullable and assigning None?
68			- When does gettattr throw AttributeError, and when does it return None?
69
70
71
72
73
74			Examples:
75			>>> class Color(Enum):
76			... blue = 0
77			... black = 1
78			... red = 2
79			>>> class Car(Entity):
80			... color = EnumField(Color)
81			... weight = NumberField(required=False)
82			... wheels = IntField(default=4, validation=lambda x: 3 <= x <= 4)
83
84			>>> # create a new car object
85			>>> car = Car(color=Color.blue, weight=4242.42)
86
87			>>> # it has 4 wheels, all by default
88			>>> car.wheels
89			4
90
91			>>> # but a car can't have 5 wheels!
92			>>> car.wheels = 5
93			Traceback (most recent call last):
94			...
95			auxlib.exceptions.ValidationError: Invalid value 5 for wheels
96
97			# we can call .dump() on car, and just get back a simple python dict
98			>>> type(car.dump())
99			<class 'dict'>
100			>>> sorted(car.dump().items())
101			[('color', 0), ('weight', 4242.42), ('wheels', 4)]
102
103			# and json too
104			>>> car.json(sort_keys=True)
105			'{"color": 0, "weight": 4242.42, "wheels": 4}'
106
107			# green cars aren't allowed
108			>>> car.color = "green"
109			Traceback (most recent call last):
110			...
111			auxlib.exceptions.ValidationError: 'green' is not a valid Color
112
113			# but black cars are!
114			>>> car.color = "black"
115			>>> car.color
116			<Color.black: 1>
117
118			# car.color really is an enum, promise
119			>>> type(car.color)
120			<enum 'Color'>
121
122			# let's do a round-trip marshalling of this thing
123			>>> same_car = Car.from_json(json.dumps(car.dump())) # or equally Car.from_json(car.json()))
124			>>> same_car == car
125			True
126
127			# actually, they're two different instances
128			>>> same_car is not car
129			True
130
131			# this works too
132			>>> cloned_car = Car(**car.dump())
133			>>> cloned_car == car
134			True
135
136			# now let's get fancy
137			>>> class Fleet(Entity):
138			... cars = ListField(Car)
139			... boss_car = ComposableField(Car)
140
141			# here's our fleet of company cars
142			>>> company_fleet = Fleet(boss_car=Car(color='red'), cars=[car, same_car, cloned_car])
143			>>> company_fleet.pretty_json() #doctest: +SKIP
144			{
145			"boss_car": {
146			"color": 2,
147			"wheels": 4
148			},
149			"cars": [
150			{
151			"color": 1,
152			"weight": 4242.42,
153			"wheels": 4
154			},
155			{
156			"color": 1,
157			"weight": 4242.42,
158			"wheels": 4
159			},
160			{
161			"color": 1,
162			"weight": 4242.42,
163			"wheels": 4
164			}
165			]
166			}
167
168			# the boss' car is red of course (and it's still an Enum)
169			>>> company_fleet.boss_car.color.name
170			'red'
171
172			# and there are three cars left for the employees
173			>>> len(company_fleet.cars)
174			3
175
176			# because we can
177			>>> sum(c.weight for c in company_fleet.cars)
178			12727.26
179
180
181
182			"""
183			from __future__ import absolute_import, division, print_function
184
185			from datetime import datetime
186			from functools import reduce
187			from json import loads as json_loads, dumps as json_dumps
188			from logging import getLogger
189			import collections
190			import json
191
192			from enum import Enum
193
194			from ._vendor.dateutil.parser import parse as dateparser
195			from ._vendor.five import with_metaclass, items, values
196			from ._vendor.six import integer_types, string_types
197			from .collection import AttrDict
198			from .exceptions import ValidationError, Raise
199			from .type_coercion import maybecall
200
201			log = getLogger(__name__)
202
203			KEY_OVERRIDES_MAP = "__key_overrides__"
204
205
206			class Field(object):
207			"""
208			Fields are doing something very similar to boxing and unboxing
209			of c#/java primitives. __set__ should take a "primitve" or "raw" value and create a "boxed"
210			or "programatically useable" value of it. While __get__ should return the boxed value,
211			dump in turn should unbox the value into a primitive or raw value.
212
213			Arguments:
214			types_ (primitive literal or type or sequence of types):
215			default (any, callable, optional): If default is callable, it's guaranteed to return a
216			valid value at the time of Entity creation.
217			required (boolean, optional):
218			validation (callable, optional):
219			dump (boolean, optional):
220			"""
221
222			def __init__(self, default=None, required=True, validation=None, in_dump=True, nullable=False):
223			self._default = default if callable(default) else self.box(default)
224			self._required = required
225			self._validation = validation
226			self._in_dump = in_dump
227			self._nullable = nullable
228			if default is not None:
229			self.validate(self.box(maybecall(default)))
230
231			@property
232			def name(self):
233			try:
234			return self._name
235			except AttributeError:
236			log.error("The name attribute has not been set for this field. "
237			"Call set_name at class creation time.")
238			raise
239
240			def set_name(self, name):
241			self._name = name
242			return self
243
244			def __get__(self, instance, instance_type):
245			try:
246			if instance is None: # if calling from the class object
247			val = getattr(instance_type, KEY_OVERRIDES_MAP)[self.name]
248			else:
249			val = instance.__dict__[self.name]
250			except AttributeError:
251			log.error("The name attribute has not been set for this field.")
252			raise
253			except KeyError:
254			if self.default is not None:
255			val = maybecall(self.default) # default can be a callable
256			elif self._nullable:
257			return None
258			else:
259			raise AttributeError("A value for {0} has not been set".format(self.name))
260			return self.unbox(val)
261
262			def __set__(self, instance, val):
263			# validate will raise an exception if invalid
264			# validate will return False if the value should be removed
265			instance.__dict__[self.name] = self.validate(self.box(val))
266
267			def __delete__(self, instance):
268			instance.__dict__.pop(self.name, None)
269
270			def box(self, val):
271			return val
272
273			def unbox(self, val):
274			return val
275
276			def dump(self, val):
277			return val
278
279			def validate(self, val):
280			"""
281
282			Returns:
283			True: if val is valid
284
285			Raises:
286			ValidationError
287			"""
288			# note here calling, but not assigning; could lead to unexpected behavior
289			if isinstance(val, self._type) and (self._validation is None or self._validation(val)):
290			return val
291			elif val is None and self.nullable:
292			return val
293			raise ValidationError(getattr(self, 'name', 'undefined name'), val)
294
295			@property
296			def required(self):
297			return self.is_required
298
299			@property
300			def is_required(self):
301			return self._required
302
303			@property
304			def is_enum(self):
305			return isinstance(self._type, type) and issubclass(self._type, Enum)
306
307			@property
308			def type(self):
309			return self._type
310
311			@property
312			def default(self):
313			return self._default
314
315			@property
316			def in_dump(self):
317			return self._in_dump
318
319			@property
320			def nullable(self):
321			return self.is_nullable
322
323			@property
324			def is_nullable(self):
325			return self._nullable
326
327
328			class IntField(Field):
329			_type = integer_types
330
331
332			class StringField(Field):
333			_type = string_types
334
335
336			class NumberField(Field):
337			_type = integer_types + (float, complex)
338
339
340			class BooleanField(Field):
341			_type = bool
342
343			def box(self, val):
344			return None if val is None else bool(val)
345
346
347			class DateField(Field):
348			_type = datetime
349
350			def box(self, val):
351			try:
352			return dateparser(val) if isinstance(val, string_types) else val
353			except ValueError as e:
354			raise ValidationError(val, msg=e)
355
356			def dump(self, val):
357			return None if val is None else val.isoformat()
358
359
360			class EnumField(Field):
361
362			def __init__(self, enum_class, default=None, required=True, validation=None,
363			in_dump=True, nullable=False):
364			if not issubclass(enum_class, Enum):
365			raise ValidationError(None, msg="enum_class must be an instance of Enum")
366			self._type = enum_class
367			super(self.__class__, self).__init__(default, required, validation, in_dump, nullable)
368
369			def box(self, val):
370			if val is None:
371			# let the required/nullable logic handle validation for this case
372			return None
373			try:
374			# try to box using val as an Enum name
375			return val if isinstance(val, self._type) else self._type(val)
376			except ValueError as e1:
377			try:
378			# try to box using val as an Enum value
379			return self._type[val]
380			except KeyError:
381			raise ValidationError(val, msg=e1)
382
383			def dump(self, val):
384			return None if val is None else val.value
385
386
387			class ListField(Field):
388			_type = tuple
389
390			def __init__(self, element_type, default=None, required=True, validation=None,
391			in_dump=True, nullable=False):
392			self._element_type = element_type
393			super(self.__class__, self).__init__(default, required, validation, in_dump, nullable)
394
395			def box(self, val):
396			if val is None:
397			return None
398			elif isinstance(val, string_types):
399			raise ValidationError("Attempted to assign a string to ListField {0}"
400			"".format(self.name))
401			elif isinstance(val, collections.Iterable):
402			et = self._element_type
403			if isinstance(et, type) and issubclass(et, Entity):
404			return tuple(v if isinstance(v, et) else et(**v) for v in val)
405			else:
406			return tuple(val)
407			else:
408			raise ValidationError(val, msg="Cannot assign a non-iterable value to "
409			"{0}".format(self.name))
410
411			def unbox(self, val):
412			return tuple() if val is None and not self.nullable else val
413
414			def dump(self, val):
415			if isinstance(self._element_type, type) and issubclass(self._element_type, Entity):
416			return tuple(v.dump() for v in val)
417			else:
418			return val
419
420			def validate(self, val):
421			if val is None:
422			if not self.nullable:
423			raise ValidationError(self.name, val)
424			return None
425			else:
426			val = super(self.__class__, self).validate(val)
427			et = self._element_type
428			tuple(Raise(ValidationError(self.name, el, et)) for el in val
429			if not isinstance(el, et))
430			return val
431
432
433			class MapField(Field):
434			_type = dict
435			__eq__ = dict.__eq__
436			__hash__ = dict.__hash__
437
438
439			class ComposableField(Field):
440
441			def __init__(self, field_class, default=None, required=True, validation=None,
442			in_dump=True, nullable=False):
443			self._type = field_class
444			super(self.__class__, self).__init__(default, required, validation, in_dump, nullable)
445
446			def box(self, val):
447			if val is None:
448			return None
449			if isinstance(val, self._type):
450			return val
451			else:
452			# assuming val is a dict now
453			try:
454			# if there is a key named 'self', have to rename it
455			val['slf'] = val.pop('self')
456			except KeyError:
457			pass # no key of 'self', so no worries
458			return val if isinstance(val, self._type) else self._type(**val)
459
460			def dump(self, val):
461			return None if val is None else val.dump()
462
463
464			class EntityType(type):
465
466			@staticmethod
467			def __get_entity_subclasses(bases):
468			try:
469			return [base for base in bases if issubclass(base, Entity) and base is not Entity]
470			except NameError:
471			# NameError: global name 'Entity' is not defined
472			return []
473
474			def __new__(mcs, name, bases, dct):
475			# if we're about to mask a field that's already been created with something that's
476			# not a field, then assign it to an alternate variable name
477			non_field_keys = (key for key, value in items(dct)
478			if not isinstance(value, Field) and not key.startswith('__'))
479			entity_subclasses = EntityType.__get_entity_subclasses(bases)
480			if entity_subclasses:
481			keys_to_override = [key for key in non_field_keys
482			if any(isinstance(base.__dict__.get(key), Field)
483			for base in entity_subclasses)]
484			dct[KEY_OVERRIDES_MAP] = {key: dct.pop(key) for key in keys_to_override}
485			else:
486			dct[KEY_OVERRIDES_MAP] = dict()
487
488			return super(EntityType, mcs).__new__(mcs, name, bases, dct)
489
490			def __init__(cls, name, bases, attr):
491			super(EntityType, cls).__init__(name, bases, attr)
492			cls.__fields__ = dict(cls.__fields__) if hasattr(cls, '__fields__') else dict()
493			cls.__fields__.update({name: field.set_name(name)
494			for name, field in cls.__dict__.items()
495			if isinstance(field, Field)})
496			if hasattr(cls, '__register__'):
497			cls.__register__()
498
499			@property
500			def fields(cls):
501			return cls.__fields__.keys()
502
503
504			@with_metaclass(EntityType)
505			class Entity(object):
506			__fields__ = dict()
507			# TODO: add arg order to fields like in enum34
508
509			def __init__(self, **kwargs):
510			for key, field in items(self.__fields__):
511			try:
512			setattr(self, key, kwargs[key])
513			except KeyError:
514			# handle the case of fields inherited from subclass but overrode on class object
515			if key in getattr(self, KEY_OVERRIDES_MAP):
516			setattr(self, key, getattr(self, KEY_OVERRIDES_MAP)[key])
517			elif field.required and field.default is None:
518			raise ValidationError(key, msg="{0} requires a {1} field. Instantiated with "
519			"{2}".format(self.__class__.__name__,
520			key, kwargs))
521			except ValidationError:
522			if kwargs[key] is not None or field.required:
523			raise
524			self.validate()
525
526			@classmethod
527			def create_from_objects(cls, objects, *override_fields):
528			init_vars = dict()
529			search_maps = (AttrDict(override_fields), ) + objects
530			for key, field in items(cls.__fields__):
531			value = find_or_none(key, search_maps)
532			if value is not None or field.required:
533			init_vars[key] = field.type(value) if field.is_enum else value
534			return cls(**init_vars)
535
536			@classmethod
537			def from_json(cls, json_str):
538			return cls(**json_loads(json_str))
539
540			@classmethod
541			def load(cls, data_dict):
542			return cls(**data_dict)
543
544			def validate(self):
545			# TODO: here, validate should only have to determine if the required keys are set
546			try:
547			reduce(lambda x, y: y, (getattr(self, name)
548			for name, field in self.__fields__.items()
549			if field.required))
550			except AttributeError as e:
551			raise ValidationError(None, msg=e)
552			except TypeError as e:
553			if "no initial value" in str(e):
554			# TypeError: reduce() of empty sequence with no initial value
555			pass
556			else:
557			raise # pragma: no cover
558
559			def __repr__(self):
560			def _repr(val):
561			return repr(val.value) if isinstance(val, Enum) else repr(val)
562			return "{0}({1})".format(
563			self.__class__.__name__,
564			", ".join("{0}={1}".format(key, _repr(value)) for key, value in self.__dict__.items()))
565
566			@classmethod
567			def __register__(cls):
568			pass
569
570			def json(self, indent=None, separators=None, sort_keys=False, **kwargs):
571			return json_dumps(self.dump(), indent=indent, separators=separators,
572			sort_keys=sort_keys, **kwargs)
573
574			def pretty_json(self, indent=2, separators=(',', ': '), sort_keys=True, **kwargs):
575			return self.json(indent=indent, separators=separators, sort_keys=sort_keys, **kwargs)
576
577			def dump(self):
578			return {field.name: field.dump(value)
579			for field, value in ((field, getattr(self, field.name, None))
580			for field in self.__dump_fields())
581			if value is not None or field.nullable}
582
583			@classmethod
584			def __dump_fields(cls):
585			if '__dump_fields_cache' not in cls.__dict__:
586			cls.__dump_fields_cache = {field for field in values(cls.__fields__) if field.in_dump}
587			return cls.__dump_fields_cache
588
589			def __eq__(self, other):
590			if self.__class__ != other.__class__:
591			return False
592			rando_default = 19274656290 # need an arbitrary but definite value if field does not exist
593			return all(getattr(self, field, rando_default) == getattr(other, field, rando_default)
594			for field in self.__fields__)
595
596			def __hash__(self):
597			return sum(hash(getattr(self, field, None)) for field in self.__fields__)
598
599
600			def find_or_none(key, search_maps, map_index=0):
601			"""Return the value of the first key found in the list of search_maps,
602			otherwise return None.
603
604			Examples:
605			>>> d1 = AttrDict({'a': 1, 'b': 2, 'c': 3, 'e': None})
606			>>> d2 = AttrDict({'b': 5, 'e': 6, 'f': 7})
607			>>> find_or_none('c', (d1, d2))
608			3
609			>>> find_or_none('f', (d1, d2))
610			7
611			>>> find_or_none('b', (d1, d2))
612			2
613			>>> print(find_or_none('g', (d1, d2)))
614			None
615			>>> find_or_none('e', (d1, d2))
616			6
617
618			"""
619			try:
620			attr = getattr(search_maps[map_index], key)
621			return attr if attr is not None else find_or_none(key, search_maps[1:])
622			except AttributeError:
623			# not found in first map object, so go to next
624			return find_or_none(key, search_maps, map_index+1)
625			except IndexError:
626			# ran out of map objects to search
627			return None
628

kalefranz / auxlib

Push — develop ( 1f233d...105c0e )

auxlib.Entity.json() A

Complexity

Size

Duplication

Duplication Side-by-Side

Filter issues like