‘jsondata.jsondata’ - Module¶
Core features for the processing of structured JSON based in-memory data. This comprises the load of a master model from a JSON file - which could be initally empty, and the incremental addition and removal of branches by loading additional JSON modules into the master model. The resulting data could be saved for later reuse. The implementation is based on the standard packages json [json], ujson [ujson], and ‘jsonschema’ [jsonschema].
Additional modification of the data is supported the module jsonpatch analysis and comparison by the module jsondatadiff.
Module¶
Core features for the processing of JSON based structures of in-memory data. Supports RFC4627 and RCF7159, addresses of in-memory-nodes and RFC6901, relative pointer-draft.
JSONData¶
This class provides for the handling of the in-memory data by the main hooks ‘data’, and ‘schema’. This includes generic methods for the advanced management of arbitrary ‘branches’ in extension to RCF6902, and additional methods strictly compliant to RFC6902.
- Attributes:
- data: The data tree of JSON based objects provided
- by the module ‘json’.
- schema: The validator for ‘data’ provided by
- the module ‘jsonschema’.
- Common call parameters provided by the methods of this class are:
- targetnode := addressreference
- The target node of called method. The ‘targetnode’ in general represents the target of the called method. In most cases this has to be a reference to a container for the modification and/or insertion of resulting elements. The methods require the change of contained items, which involves the application of a ‘key’ pointing to the hook in point of the reference to the modification.
- key := key-value
- The hook-in point for references of modified entries within the targetnode container. The following values are supported:
- sourcenode := addressreference
- The in-memory node address of the source branch for the method, e.g. ‘copy’ or ‘move’ operation.
The address references supported in this class refer the resulting in-memory representation of a pointer path. The target is a node within a Python data representation as provided by the package ‘json‘ and compatible packages, e.g. ‘ujson‘. The supported input syntax is one of the following interchangeable formats:
# The reference to a in-memory-node. addressreference := ( nodereference | addressreference-source ) nodereference:= ( <in-memory> | '' ) <in-memory> := "Memory representation of a JSON node, a 'dict' or a 'list'. The in-memory Python node reference has to be located within the document, due to performance reasons this is not verified by default. The 'nodereference' could be converted from the 'addressreference-source' representation." '' := "Represents the whole document in accordance to RFC6901. Same as 'self.data'." # The source of the syntax for the description of the reference # pointer path to a node. This is applicable on paths to be created. addressreference-source := ( JSONPointer ) JSONPointer:="A JSONPointer object in accordance to RFC6901. for additional information on input formats refer to the class documentation. This class provides a fully qualified path pointer, which could be converted into any of the required representations."
For hooks by ‘key-value’ within addressed containers:
key-value:=(None|<list-index>|<dict-key>) None := "When the 'key' parameter is 'None', the action optionally could be based on the keys of the 'sourcenode'. The contents of the branch replace the node contents when the type of the branch matches the hook." <list-index>:=('-'|int) <dict-key>:="Valid for a 'dict' only, sets key/value pair, where present is replace, new is created." '-' := "Valid for a 'list' only, appends to present." int := "Valid for a 'list' only, replaces present when 0 < #int < len(Node)."
In the parameter lists of methods used term ‘pointer’ is either an object of class ‘JSONPointer’, or a list of pointer path entries.
The JSON types ‘object’ and ‘array’ behave in Python slightly different in accordance to RFC6902. The main difference arise from the restrictions on applicable key values. Whereas the ranges are limited logically by the actual container sizes, the object types provide free and unlimited keys. The limit is set by type restriction to unicode and ‘non-nil’ only for keys.
-
class
jsondata.jsondata.
JSONData
(jdata, **kargs)[source]¶ Representation of a JSON based object data tree.
The common node address parameters are defined as:
sourcenode := <commonnode>|<indata> # depends on the interface targetnode := <innode> # target within the represented # JSON structure commonnode := (anydata | indata) anydata := ( # any data hook - within # self.data or external JSONData # - adds to the contained data # of the reference | <json-array-list> # - adds to JSON array | <json-object-dict> # - adds to JSON object ) indata := ( # within self.data JSONPointer # - adds to the node within # self.data [RFC6901]_ | <rfc6901-string> # - pointer string within self.data [RFC6901]_ | <relative-pointer-string>) # - relative pointer within self.data [RELPOINTER]_ )
- REMARK:
- The RFC7159 permits any JSON type as a node, while the RFC4627 permits array and object as node reference only.
Attributes¶
JSONData.data
JSON object data tree. The data tree of JSON based objects provided by the module json/ujson.
JSONData.schema
JSONschema object data tree. The validator for ‘data’ provided by the module jsonschema.
Common Call Parameters¶
- Common call parameters provided by most methods of this class are:
targetnode := addressreference
The target node of the called method. The targetnode in general represents the target of the called method. In most cases this has to be a reference to a container for the modification and/or insertion of resulting elements. The methods require the change of contained items, which involves the application of a key pointing to the hook-in point of the reference for modification.key := key-value
The hook-in point for references of modified entries within the targetnode container.sourcenode := addressreference
The in-memory node address of the source branch for the method, e.g. for the copy and move operation.
The address references supported in this class refer the resulting in-memory representation of a pointer path. The target is a node within a Python data representation as provided by the package ‘json‘ and compatible packages, e.g. ‘ujson‘. The supported input syntax is one of the following interchangeable formats:
# The reference to a in-memory-node. addressreference := ( nodereference | addressreference-source ) nodereference:= ( <in-memory> | '' ) <in-memory> := "Memory representation of a JSON node, a 'dict' or a 'list'. The in-memory Python node reference has to be located within the document, due to performance reasons this is not verified by default. The 'nodereference' could be converted from the 'addressreference-source' representation." '' := "Represents the whole document in accordance to RFC6901. Same as 'self.data'." # The source of the syntax for the description of the reference # pointer path to a node. This is applicable on paths to be created. addressreference-source := ( JSONPointer ) JSONPointer:="A JSONPointer object in accordance to RFC6901. for additional information on input formats refer to the class documentation. This class provides a fully qualified path pointer, which could be converted into any of the required representations."
For hooks by ‘key-value’ within addressed containers:
key-value:=(None|<list-index>|<dict-key>) None := "When the 'key' parameter is 'None', the action optionally could be based on the keys of the 'sourcenode'. The contents of the branch replace the node contents when the type of the branch matches the hook." <list-index>:=('-'|int) <dict-key>:="Valid for a 'dict' only, sets key/value pair, where present is replace, new is created." '-' := "Valid for a 'list' only, appends to present." int := "Valid for a 'list' only, replaces present when 0 < #int < len(Node)."
In the parameter lists of methods used term ‘pointer’ is either an object of class ‘JSONPointer’, or a list of pointer path entries.
The JSON types ‘object’ and ‘array’ behave in Python slightly different in accordance to RFC6902. The main difference arise from the restrictions on applicable key values. Whereas the ranges are limited logically by the actual container sizes, the object types provide free and unlimited keys. The limit is set by type restriction to unicode and ‘non-nil’ only for keys.
Methods¶
__init__¶
-
JSONData.
__init__
(jdata, **kargs)[source]¶ Creates and validates a new object from the provided JSON data. Arbitrary additional JSON data could be added as branches.
- Args:
- jdata:
The initial data of current instance. The accepted formats are in-memory representation of JSON data compatible with the standard library json [json]. The permitted input types vary in accordance to the selected operations mode of either RFC4627 or RFC7159.
type(jdata) := ( list, # json-array | dict, # json-object | JSONData # copy constructor | int # RFC7159 only | float # RFC7159 only | unicode # RFC7159 only )
Note
Further branches could be added to json-objects, and json-arrays only, while values in RFC7159 and RFC8259 mode permit replacement only. Objects support string indexes, arrays integer indexes.
- kargs:
For the complete set of call parameters refer to the method JSONData.setkargs().
- mode:
The mode of JSON processing:
mode := ( MJ_RFC4627 | MJ_RFC7493 # currently not supported, mapped to RFC7159 | MJ_RFC7159 | MJ_RFC8259 | MJ_ECMA404 # same as RFC8259 )
default := MJ_RFC7159
- schema:
A valid in-memory JSONschema.
default:= None
- validator:
Sets schema validator for the data file. Curren release relies on jsonschema, which supports at the time of writing draft-03 and draft-04.
The values are:
validator := ( MS_DRAFT3 | 'draft3' | MS_DRAFT4 | 'draft4' | MS_ON | 'on' | MS_OFF | 'off' | MODE_SCHEMA_DEFAULT | 'default' )
default:= MS_OFF
Returns:
Results in an initialized object.Raises:
NameError
JSONDataValueError
jsonschema.ValidationError
jsonschema.SchemaError
__str__¶
branch_add¶
-
JSONData.
branch_add
(sourcenode, targetnode='', key=None, **kargs)[source]¶ Add a complete branch into a target structure of type object. Present branches are replaced, non-existent branches are added.
The parent of the insertion point has to exist by default, see [RFC6902]. If the target is an array, the source is either appended, or inserted [RFC6902].
- Args:
- sourcenode:
Source branch to be inserted into the target tree. Either from within self-data, or an external source:
sourcenode := <commonnode> # see class header
- targetnode:
Target node within self-data, where the branch is to be inserted.
targetnode := <innode> # see class header
default := ‘’ # the top of the ‘whole document’
- key:
Hook for the insertion within the target node. If not provided the contents of the target node itself are replaced by the source node.
default := None
- kargs:
- copydata:
The type of creation of the added branch.
copydata := ( C_REF # copy the reference only | C_DEEP # call copy.deepcopy() | C_SHALLOW # call copy.copy() )
default := C_DEEP
- Returns:
- When successful returns True, else returns either False, or raises an exception.
- Raises:
- JSONDataNodeError:
- The target node is not contained in current object.
- JSONDataNodeTypeError:
- The types mismatch.
- JSONDataKeyError:
- Key mismatch.
branch_copy¶
-
JSONData.
branch_copy
(sourcenode, targetnode='/', key=None, force=True)[source]¶ Copies the source branch to the target node. The branch_copy is internally mapped to the call branch_add, thus shares basically the same parameters and behavior.
Args:
- sourcenode:
Source branch to be copied into target tree. Either from within self-data, or an external source:
sourcenode := <commonnode> # see class header
- targetnode:
Target node for the branch. Either from within self-data, or an external source:
targetnode := <innode> # see class header
default := ‘/’
- key:
- Optional key for the insertion point within target node, if not provided the target node itself.
- force:
If true present are replaced, else only non-present targets are copied.
default := True
- Returns:
- When successful returns True, else returns either False, or raises an exception.
- Raises:
JSONDataError
pass-through
branch_create¶
-
JSONData.
branch_create
(branchpath, targetnode=None, padding_value=None)[source]¶ Creates an abitrary relative branch from a path description located at targetnode. Intermediate nodes are created automatically when missing.
The requested branch is created as the relative child branch of the provided targetnode. The targetnode must exist, while the child node - branchpath[0] - must not.
Args:
- branchpath:
New branch to be created in the target node. A Pointer address path relative to the targetnode.
branchpath := <commonnode> # see class header
- targetnode:
Base node for the created branch, must exist and located within the data tree of current object.
targetnode := <innode> # see class header
default := “/”
- padding_value:
Optional default value, either an atomic type or a sub-branch itself. This value is only used for a new leaf, in case of an existent node the value is ignored.
default := “null” / None
- Returns:
- When successful returns the created node, else returns either None, or raises an exception.
- Raises:
JSONDataError
JSONDataKeyError
JSONDataNodeTypeError
JSONDataParameterError
JSONDataPathError
branch_move¶
-
JSONData.
branch_move
(sourcenode, targetnode=None, key=None, force=False)[source]¶ Moves a branch to the target node.
Args:
- sourcenode:
Source branch to be moved into the target node. Must be member of the self-data structure, else use either branch_add or branch_copy. Either from within self-data, or an external source:
sourcenode := <innode> # see class header
- targetnode:
Target node for the branch.
targetnode := <innode> # see class header
default := ‘/’
- key:
- Optional key for the insertion point within target node, if not provided the target node itself.
- force:
If true present are replaced, else only non-present are copied.
default := True
- Returns:
- When successful returns True, else returns either None, or raises an exception.
- Raises:
JSONDataError
JSONDataKeyError
branch_remove¶
-
JSONData.
branch_remove
(targetnode, key=None, rfc6902=True)[source]¶ Removes a branch from a contained data in self.
Args:
- targetnode:
Container with item to be removed.
targetnode := <innode> # see class header
- key:
- Key of insertion point within target node, if not provided the target node itself.
- rfc6902:
- If True the removed element has to be present, else non-present is simply ignored.
- Returns:
- When successful returns True, else returns either False, or raises an exception.
- Raises:
JSONDataKeyError
JSONDataNodeTypeError
branch_replace¶
-
JSONData.
branch_replace
(sourcenode, targetnode, key=None, rfc6902=True)[source]¶ Replaces the value of the target node by the copy of the source branch.
Requires in order to RFC6902, all items to be replaced has to be present. Thus fails by default if at least one is missing.
Internally the ‘branch_add()’ call is used with a deep copy. When a swallow copy is required the ‘branch_move()’ has to be used.
- Args:
- sourcenode:
Source branch to be inserted into target tree. Either from within self-data, or an external source:
sourcenode := <commonnode> # see class header
- targetnode:
Target where the branch is inserted.
targetnode := <innode> # see class header
- key:
- Key of insertion point within target node, if not provided the target node itself.
- rfc6902:
- If True the removed element has to be present, else non-present is simply ignored.
- Returns:
- When successful returns True, else returns either False, or raises an exception.
- Raises:
- JSONDataError
branch_superpose¶
-
JSONData.
branch_superpose
(sourcenode, targetnode=None, key=None, **kargs)[source]¶ Superposes a branch recursively on to the current data tree self.data with defined constraints. Provides partial mapping in dependence of parameters and data entries of source and/or target.
The processing is controlled by logic operations on node structures, which maps a logical tree of JSON nodes from the source node onto the subtree of JSON nodes defined by the target target node. The provided logic operators are related to structure, though the types of nodes, not the contents.
For the provided logic operators refer to parameter map.
Args:
- sourcenode:
Value struct to be inserted.
default := None
- targetnode:
Data node within current document to be superposed.
default := None # whole document self.data
- key:
Hook selector within the data tree spanned by the targetnode .
default := None # top of the targetnode
- kargs:
- copy:
Create a copy of the sourcenode.
copy := ( C_DEEP # insert from copy.deepcopy() | C_SHALLOW # insert from copy.copy() | C_REF # insert from the provided parameter )
default := C_REF # no copy, work on input
- depth:
Sets the default behavior for the operators on the data branches. Controls, whether the hook only or the complete branch is processed in depth node-by-node.
0: the hook of the branch #n: the level of the branch as integer, the remaining sub-branch is treated by it's hook -1: the complete branch
default:= 0
- ignore:
Ignores attributes including subtrees contained in the list.
ignore := [<list-of-rfc6901-path-items>]
- map:
Sets the default behavior for the mapping of branches by operators. This is also influenced by the parameter ‘op_depth’.
B_AND: replace corresponding leafs only when any target node is present B_OR: replace corresponding items of source leafs B_XOR: insert only when no target node is present
default:= B_OR
- use:
Considers the listed attributes only as white list.
use := [<list-of-rfc6901-path-items>]
- Returns:
- True for success, else False or raises exception.
- Raises:
JSONDataError
JSONDataIndexError
pass-through
branch_test¶
-
JSONData.
branch_test
(targetnode, value)[source]¶ Tests match in accordance to RFC6902.
- Args:
- targetnode:
Node to be compared with the value. Due to ambiguity the automated conversion is not reliable, thus it has to be valid.
targetnode := <innode> # see class header
- value:
- Expected value for the given node.
- Returns:
- When successful returns ‘True’, else returns ‘False’.
- Raises:
- JSONDataError
deepcopy¶
get_canonical_value¶
-
JSONData.
get_canonical_value
(node)[source]¶ Fetches a copy of the canonical value represented by the node. The actual value could be either an atomic value, a node representing a branch, or a reference to an atomic value. Creates a deep copy, thus references are no longer valid.
- Args:
- value:
Value pointer to be evaluated to the actual value. Valid input types are:
- int,str,unicode:
- Integer, kept as an atomic integer value.
- dict,list:
- Assumed to be a valid node for ‘json’ package, used by reference.
- JSONPointer:
- A JSON pointer in accordance to RFC6901.
- Returns:
- When successful returns the value, else returns either ‘False’, or raises an exception.
- Raises:
- JSONDataError
dump_data¶
-
JSONData.
dump_data
(pretty=0, **kargs)[source]¶ Prints structured data.
Args:
- pretty:
Activates pretty printer, else flat.
format := ( PJ_TREE # tree view JSON syntax | PJ_FLAT # flat print JSON syntax | PJ_PYTREE # tree view Python syntax | PJ_PYFLAT # flat print Python syntax | PJ_REPR # repr() - raw string, Python syntax | PJ_STR # str() - formatted string, Python syntax )
- kargs:
- source:
Prints data within ‘source’.
default:=self.data
- Returns:
- When successful returns ‘True’, else returns either ‘False’, or raises an exception.
- Raises:
- pass-through
dump_schema¶
-
JSONData.
dump_schema
(pretty=True, **kargs)[source]¶ Prints structured schema.
Args:
- pretty:
- Activates pretty printer for treeview, else flat.
- kargs:
- source:
Prints schema within ‘source’.
default:=self.schema
- Returns:
- When successful returns ‘True’, else returns either ‘False’, or raises an exception.
- Raises:
- pass-through
set_schema¶
-
JSONData.
set_schema
(schemafile=None, targetnode=None, **kargs)[source]¶ Sets schema or inserts a new branch into the current assigned schema.
The main schema(targetnode==None) is the schema related to the current instance. Additional branches could be added by importing the specific schema definitions into the main schema. These could either kept volatile as a temporary runtime extension, or stored into a new schema file in order as extension of the original for later combined reuse.
- Args:
- schemafile:
- JSON-Schema filename for validation of the subtree/branch. See also **kargs[‘schema’].
- targetnode:
- Target container hook for the inclusion of the loaded branch.
- kargs:
- schema:
In-memory JSON-Schema as an alternative to schemafile. When provided the ‘schemafile’ is ignored.
default:=None
- validator:
Sets schema validator for the data file. The values are:
validator := ( default = validate, | draft3 = Draft3Validator, | off = None. )
default:= validate
- persistent:
Stores the ‘schema’ persistently into ‘schemafile’ after completion of update including addition of branches. Requires valid ‘schemafile’.
default:=False
- Returns:
- When successful returns ‘True’, else returns either ‘False’, or raises an exception.
Raises:
JSONDataError
JSONDataSourceFileError
JSONDataValueError
setkargs¶
-
JSONData.
setkargs
(**kargs)[source]¶ Sets key arguments.
Args:
kargs:
- copydata:
Controls the assignment policy for JSON data. The schema is copied by reference only, once read.
copydata := ( C_REF # by reference | C_DEEP # by copy.deepcopy() | C_SHALLOW # by copy.copy() )
- debug:
- Displays extended state data for developers. Requires __debug__==True.
- depth:
Sets the default behavior for the operators. Controls, whether the hook only or the complete branch is processed in depth node-by-node.
0: the hook of the branch #n: the level of the branch as integer, the remaining sub-branch is treated by it's hook -1: the complete branch
default:= 0
- indent_str:
Defied the indentation of ‘str’.
default:= 4
- jsonsyntax:
The display syntax for JSON data with __str__.
jsonsyntax := ( JSYN_NATIVE | 'json' | JSYN_PYTHON | 'python' ) JSYN_NATIVE: Native standards syntax. JSYN_PYTHON: Python in-memory syntax.
- mode:
The mode of JSON processing:
mode := ( MJ_RFC4627 | 'rfc4627' | MJ_RFC7493 | 'rfc7493' # currently not supported, mapped to RFC7159 | MJ_RFC7159 | 'rfc7159' | MJ_RFC8259 | 'rfc8259' | MJ_ECMA404 | 'ecma404' # same as RFC8259 )
default := MJ_RFC7159
- rtype:
Sets the data type of the returned result.
rtype := ( data # returns the data record only - self.data | jdata # returns an object of class JSONData | obj # returns an object of own class )
default := obj
- saveContext:
Saves and restores the defined context parameters when entering a context call.
default:= True
- schema:
A valid in-memory JSONschema.
default:= None
- sortstr:
Sort display of ‘str()’ by key.
sortstr := (True | False)
default := True
- validator:
Sets schema validator for the data file. Curren release relies on jsonschema, which supports at the time of writing draft-03 and draft-04.
The values are:
validator := ( MS_DRAFT3 | 'draft3' | MS_DRAFT4 | 'draft4' | MS_ON | 'on' | MS_OFF | 'off' | MODE_SCHEMA_DEFAULT | 'default' )
default:= MS_OFF
- verbose:
- Extends the amount of the display of processing data.
- Returns:
- instance/None
Raises:
NameError:
JSONDataValueError
jsonschema.ValidationError:
jsonschema.SchemaError:
validate¶
-
JSONData.
validate
(data, schema, validator=None)[source]¶ Validate data with schema by selected validator.
Args:
- data:
- JSON-Data.
- schema:
- JSON-Schema for validation.
- validator:
Validator to be applied, current supported:
schema:
In-memory JSON-Schema as an alternative to schemafile. When provided the ‘schemafile’ is ignored.
default:=None
- validator: [default, draft3, draft4, off, on, ]
Sets schema validator for the data file.
- default|MS_ON:
- The current default.
- draft3|MS_DRAFT3:
- The first supported JSONSchema IETF-Draft.
- draft4|MS_DRAFT4:
- The current supported JSONSchema IETF-Draft.
- off|MS_OFF:
- No validation.
default:= MS_DRAFT4
- Returns:
- When successful returns ‘True’, else returns either ‘False’, or raises an exception.
Raises:
JSONDataValidationError
JSONDataSchemaError
JSONDataValueError
Operators¶
Conxtext Manager¶
The context manager provides setting specific parameters for the data processing, which is reset to the state before after finishing.