‘jsondata’ - package

Modular processing of JSON data by trees and branches, pointers and patches.

Sources

‘jsondata.JSONData’ - Module

Constants

Compliance modes

  • MODE_JSON_RFC4927 = 0: Compliant to IETF RFC4927.
  • MODE_JSON_RF7951 = 2: Compliant to IETF RF7951.
  • MODE_JSON_ECMA264 = 10: Compliant to ECMA-264, refer to Chapter 15.12 The JSON Object.
  • MODE_POINTER_RFC6901 = 20: Compliant to IETF RFC6901.
  • MODE_PATCH_RFC6902 = 30: Compliant to IETF RFC6902.
  • MODE_SCHEMA_DRAFT3 = 43: Compliant to IETF DRAFT3.
  • MODE_SCHEMA_DRAFT4 = 44: Compliant to IETF DRAFT4.

Types of validator

  • OFF = 0: No validation.
  • DRAFT4 = 1: Use draft4: jsonschema.validator(Draft4Validator)
  • DRAFT3 = 2: Use draft3:jsonschema.Draft3Validator

The default value is:

  • DEFAULT = DRAFT4 = 1: Default

Match Criteria

Match criteria for node comparison:

  • MATCH_INSERT = 0: for dicts
  • MATCH_NO = 1: negates the whole set
  • MATCH_KEY = 2: for dicts
  • MATCH_CHLDATTR = 3: for dicts and lists
  • MATCH_INDEX = 4: for lists
  • MATCH_MEM = 5: for dicts(value) and lists
  • MATCH_NEW = 6: for the creation of new

Return Sets

  • FIRST: The first match only.
  • ALL: All matches.

JSONData

Attributes

  • JSONData.data: JSON object data tree.
  • JSONData.schema: JSONschema object data tree.

Methods

__init__
__repr__
__str__
branch_add
branch_copy
branch_create
branch_move
branch_remove
branch_replace
branch_test
getData
getPointerPath
getSchema
getTreeDiff
getValueNode
isApplicable
printData
printSchema
pop
setSchema
validate

Operators

‘+’
‘&&’
‘()’
‘S==x’
‘[]’
‘S!=x’
‘+=’
‘=&&’
‘S%x’
*=’
‘||=’
‘-=’
‘^=’
‘%’
‘*’
‘||’
‘S+x’
‘S&&x’
‘S%x’
‘S*x”
‘S||x”
‘S^x”
‘-‘
‘^’

Iterators

__iter__

Exceptions

‘jsondata.JSONDataSerializer’ - Module

JSONDataSerializer

Attributes

  • JSONDataSerializer.data: JSON object data tree.
  • JSONDataSerializer.schema: JSONschema object data tree.

Methods

__init__
__str__
__repr__
json_export
json_import
printData
printSchema
setSchema

‘jsondata.JSONPointer’ - Module

Provides classes for the JSONPointer definition in accordance to RFC6901.

The provided class JSONPointer internally stores and applies pointer data as a list of keys and indexes with the additional cooperative caching of the pointed in-memory node reference for fast access on data provided by the packages ‘json’ and ‘jsonschema’. Requests for the string representation are transformed into a pointer path in accordance to RFC6901.

The JSONPointer class combines fast in-memory operations and pointer arithmetics with standards compliant path strings at the API.

The JSONPointer class by itself is focused on the path pointer itself, though the provided operations do not touch the content value. The pointer provides the hook where the value has to be inserted.

JSONPointer

class jsondata.JSONPointer.JSONPointer(ptr, replace=True, **kargs)[source]

Represents exactly one JSONPointer in compliance with IETF RFC6901. This pointer could be processed by extension, reduction, and general modification with support of provided methods and path arithmetic operators.

The JSONPointer is provided at the API as a utf(-8) string in accordance to RFC6901, including RFC3869.

For enhancement of the processing performance by the underlying packages ‘json’ and ‘jsonschema’, the pointer is stored and applied in two variants.

  • self.raw: Raw input of the pointer string for the logical API.
  • self.ptr: Split elements of the pointer path within a list of keys,
    for the programming interface.

The attribute ‘self.ptr’ contains the path elements in a ‘list’:

ptrlist := (<EMPTY>|plist)
<EMPTY> := "empty list, represents the whole document"
plist := pkey [, plist ]
pkey := (''|int|keyname)
'' := "the empty string is a valid key too"
int := "integer index of an array item, just digits"
keyname := "the valid name of an object/property entry" 

The JSONPointer:

"/address/0/streetName"

is represented as:

['address', 0, 'streetName' ]

The methods and operators are handling the pointer itself, the values referenced by the pointer are not modified.

The methods of this class support for multiple input format of the JSONPointer. The parameter ‘x’ in the operations is defined as a valid JSONPointer fragment. A pointer fragment is a part of a pointer, which could be the complete pointer itself - the all-fragment.

The syntax element could be one of:

'str': A string i accordance to RFC6901. Strings are represented 
    internally as unicode utf-8.
    
    Here either the input parameter 'x' is split into
    a list, or in case of combining operations, the self.ptr
    attribute is 'joined' to be used for the method. 

'int': A numeric value in case of an array index. This value 
    is internally handled for the string representation as a
    unicode utf-8, whereas for the addressing of memory 
    objects the numeric integer value is stored and applied.

'JSONPointer': The attributes of the input object are used 
    with it's peers.

'list': Expects a path list containing:
    - JSON object names
        Names of the json objects.
    - array indexes
        Numeric index for arrays.
    - JSONPointer
        A JSONPointer object, the path is resolved as a section 
        of overall path.

    The self.ptr attribute is applied for operations.

The node reference is cached by the ‘get_node’ and ‘get_node_or_value’ method, thus could be accessed by ‘self.node’, but is not monitored to be valid. Another call of the method reloads the cache by evaluating the pointer value on the document again.

The provided value is internally stored as a raw input value, and a list of keys and indexes for access to in-memory data as provided by the packages ‘json’ and ‘jsonschema’. Requests for the string representation are transformed into a pointer path in accordance to RFC6901. This provides for fast access in case of pointer arithmetics, while providing standards conform path strings at the interface.

Attributes

JSONPointer:

  • JSONPointer.ptr: JSONPointer data.
  • JSONPointer.raw: Raw input string for JSONPointer.

Methods

__init__
JSONPointer.__init__(ptr, replace=True, **kargs)[source]

Converts and stores a JSONPointer as a list.

Processes the ABNF of a JSON Pointer from RFC6901.

Args:
ptr: A JSONPointer to be represented by this object. The
supported formats are:

‘str’: A string i accordance to RFC6901 JSONPointer: A valid object, will be copied

into this, see ‘deep’.
‘list’: expects a path list, where each item
is processed for escape and unquote.

replace: Replaces masked characters. **kargs:

deep: Applies for copy operations on structured data
‘deep’ when ‘True’, else ‘swallow’ only, which is just a link to the data structure. Flat data types are copied by value in any case.

node: Force to set the pointed node in the internal cache. debug: Enable debugging.

Returns:
When successful returns ‘True’, else returns either ‘False’, or raises an exception. Success is the complete addition only, thus one failure returns False.
Raises:
JSONPointerException:
__repr__
JSONPointer.__repr__()[source]

Returns the attribute self.raw, which is the raw input JSONPointer.

__str__
JSONPointer.__str__()[source]

Returns the string for the processed path.

check_node_or_value
JSONPointer.check_node_or_value(jsondata, parent=False)[source]

Checks the existance of the corresponding node within the JSON document.

Args:
jsondata: A valid JSON data node. parent: Return the parent node of the pointed value.
Returns:
True or False
Raises:
JSONPointerException: forwarded from json
copy_path_list
JSONPointer.copy_path_list(parent=False)[source]

Returns a deep copy of the objects pointer path list.

Args:
parent: The parent node of the pointer path.
Returns:
A copy of the path list.
Raises:
none
get_node
JSONPointer.get_node(jsondata, parent=False)[source]

Gets the corresponding node reference for a JSON container type.

This method gets nodes of container types. Container types of JSON are ‘array’ a.k.a. in Python ‘list’, and ‘objects’ a.k.a. in Python ‘dict’.

Due to the special case of RFC6902 ‘append’ by the array index ‘-‘ in combination of the add rules a special exception-treatment is required, for details refer to RFC6902.

The ‘get_node’ method therefore returns only an existing node of a of valid non-ambiguous path pointer. This excludes pointers containing the symbolic index ‘-‘ for an array component.

See also related methods:

get_node_and_child: For Python access to a child node
within a container by the container itself, and the item key.
get_node_exist: For the application of partial valid
pointer paths of new branches.
get_node_or_value: For any type of pointed item, either
a node, or a value.
Args:

jsondata: A valid JSON data node. parent: Return the parent node of the pointed value.

When parent is selected, the pointed child node is not verified.
Returns:
The node reference.
Raises:
JSONPointerException: forwarded from json
get_node_and_child
JSONPointer.get_node_and_child(jsondata)[source]

Returns a tuple containing the parent node and the child.

Args:
jsondata: A valid JSON data node.
Returns:

The the tuple: (n,c): n: Node reference to parent container.

c: Key for the child entry, either an
index ‘int’, or a key (‘str’, ‘unicode’).
Raises:
JSONPointerException: forwarded from json
get_node_exist
JSONPointer.get_node_exist(jsondata, parent=False)[source]

Returns the node for valid part of the pointer, and the remaining part.

This method works similar to the ‘get_node’ method, whereas it handles partial valid path pointers, which may also include a ‘-‘ in accordance to RFC6902.

Therefore the non-ambiguous part of the pointer is resolved, and returned with the remaining part for a newly create. Thus this method is in particular foreseen to support the creation of new sub data structures.

The ‘get_node’ method therefore returns a list of two elements, the first is the node reference, the second the list of the remaining path pointer components. The latter may be empty in case of a fully valid pointer.

Args:
jsondata: A valid JSON data node. parent: Return the parent node of the pointed value.
Returns:
The node reference, and the remaining part. ret:=[ node, [<remaining-path-components-list>] ]
Raises:
JSONPointerException: forwarded from json
get_node_or_value
JSONPointer.get_node_or_value(jsondata, valtype=None, parent=False)[source]

Gets the corresponding node reference or the JSON value of a leaf.

Relies on the standard package ‘json’ by ‘Bob Ippolito <bob@redivi.com>’. This package supports in the current version the following types:

JSON Python
object dict
array list
string unicode
number (int) int, long
number (real) float
true True
false False
null None

It also understands NaN, Infinity, and -Infinity as their corresponding float values, which is outside the JSON spec.

The supported standard value types for Python of get_node_or_value() are mapped automatically as depicted in the following table. Additional bindings may be implemented by sub-classing.

JSONPointer(jsondata) Python-valtype
object (dict) dict
array (list) list
array (tuple) list
string unicode
number (int) int
number (long) long
number (float) float
*number (double) float
number (octal) int
number (hex) int
number (binary) int
number (complex)
  • (custom)
true True
false False
null None

The mappings in detail are:

  • object(dict) => dict:
    {a:b} - native Python dictionary
  • array(list) => list:
    [a,b] - native Python list
  • (*)array(tuple) => list:
    (a,b) - native Python list
  • string(str) => unicode”
    “abc” - native Python unicode string UTF-8
  • number(int) => int:
    1234, −24, 0 - Integers (unlimited precision)
  • number(long) => int:
    1234, −24, 0 - Integers (unlimited precision)
  • number(float) => float:
    1.23, 3.14e-10, 4E210, 4.0e+210, 1., .1 - Floating-point (normally implemented as C doubles in CPython)
  • (*)number(double) => float:
    1.23, 3.14e-10, 4E210, 4.0e+210, 1., .1 - Floating-point (normally implemented as C doubles in CPython)
  • number(octal) => int:
    0o177 - Octal, hex, and binary literals for integers2
  • number(hex) => int:
    0x9ff - Octal, hex, and binary literals for integers2
  • number(binary) => int:
    0b1111 - Octal, hex, and binary literals for integers2
  • number(complex) => <not-supported>(requires custom):
    3+4j, 3.0+4.0j, 3J - Complex numbers
  • true(True) => boolean(True):
    True - native Python boolean
  • false(False) => boolean(False):
    False - native Python boolean
  • null(None) => NoneType(None):
    False - native Python NoneType
Args:
jsondata: A valid JSON data node. valtype: Type of requested value. parent: Return the parent node of the pointed value.
Returns:
The node reference.
Raises:
JSONPointerException: forwarded from json
get_path_list
JSONPointer.get_path_list()[source]

Gets for the corresponding path list of the object pointer for in-memory access on the data of the ‘json’ package.

Args:
none
Returns:
The path list.
Raises:
none
get_path_list_and_key
JSONPointer.get_path_list_and_key()[source]

Gets for the corresponding path list of the object pointer for in-memory access on the data of the ‘json’ package.

Args:
none
Returns:
The path list.
Raises:
none
get_pointer
JSONPointer.get_pointer(forcenotation=None, parent=False)[source]

Gets the objects pointer in compliance to RFC6901.

Args:
forcenotation: Force the output notation to:
None := NOTATION_JSON, NOTATION_JSON = 0, NOTATION_HTTP_FRAGMENT = 1

parent: Get parent of selected node.

Returns:
The pointer in accordance to RFC6901.
Raises:
none
get_raw
JSONPointer.get_raw()[source]

Gets the objects raw 6901-pointer.

Args:
none
Returns:
The raw path.
Raises:
none

Operators

The syntax displayed for provided operators is:

S: self
x: parameter
n: numerical parameter for shift operators.

Thus the position of the opreator and parameteres is defined as follows:

z = S + x: LHS: __add__
z = x + S: RHS: __radd__
S += x:    LHS: __iadd__
‘S+x’
JSONPointer.__add__(x)[source]

Appends a Pointer to self.

Args:
x: A valid JSONPointer fragment.
Returns:
A new object of JSONPointer
Raises:
JSONPointerException:
‘S(x)’
JSONPointer.__call__(x)[source]

Evaluates the pointed value from the document.

Args:
x: A valid JSON document.
Returns:
The pointed value, or None.
Raises:
JSONPointerException
‘S==x’
JSONPointer.__eq__(x)[source]

Compares this pointer with x.

Args:
x: A valid Pointer.
Returns:
True or False
Raises:
JSONPointerException
‘S>=x’
JSONPointer.__ge__(x)[source]

Checks containment(>=) of another pointer within this.

The weight of contained entries is the criteria, though the shorter is the bigger. This is true only in case of a containment relation.

The number of equal path pointer items is compared.

Args:
x: A valid Pointer.
Returns:
True or False
Raises:
JSONPointerException:
‘S>x’
JSONPointer.__gt__(x)[source]

Checks containment(>) of another pointer or object within this.

The number of equal items is compared.

Args:
x: A valid Pointer.
Returns:
True or False
Raises:
JSONPointerException:
‘S+=x’
JSONPointer.__iadd__(x)[source]

Add in place x to self, appends a path.

Args:
x: A valid Pointer.
Returns:
‘self’ with updated pointer attributes
Raises:
JSONPointerException:
‘S<x’
JSONPointer.__le__(x)[source]

Checks containment(<=) of this pointer within another.

The number of equal items is compared.

Args:
x: A valid Pointer.
Returns:
True or False
Raises:
JSONPointerException:
‘S<x’
JSONPointer.__lt__(x)[source]

Checks containment(<) of this pointer within another.

The number of equal items is compared.

Args:
x: A valid Pointer.
Returns:
True or False
Raises:
JSONPointerException:
‘S!=x’
JSONPointer.__ne__(x)[source]

Compares this pointer with x.

Args:
x: A valid Pointer.
Returns:
True or False
Raises:
JSONPointerException
‘x+S’
JSONPointer.__radd__(x)[source]

Adds itself as the right-side-argument to the left.

This method appends ‘self’ to a path fragment on the left. Therefore it adds the path separator on it’s left side only. The left side path fragment has to maintain to be in accordance to RFC6901 by itself.

Once ‘self’ is added to the left side, it terminates it’s life cycle. Thus another simultaneous add operation is handled by the resulting other element.

Args:
x: A valid Pointer.
Returns:
The updated input of type ‘x’ as ‘x+S(x)’
Raises:
JSONPointerException:

Iterators

iter_path
JSONPointer.iter_path(jsondata=None, parent=False, rev=False)[source]

Iterator for the elements of the path pointer itself.

Args:
jsondata: If provided a valid JSON data node, the
path components are successively verified on the provided document. If None the path pointer components are just iterated.

parent: Uses the path pointer to parent node. rev: Reverse the order, start with last.

Returns:
Yields the iterator for the current path pointer component.
Raises:
JSONPointerException: forwarded from json
iter_path_nodes
JSONPointer.iter_path_nodes(jsondata, parent=False, rev=False)[source]

Iterator for the elements the path pointer points to.

Args:
jsondata: A valid JSON data node. parent: Uses the path pointer to parent node. rev: Reverse the order, start with last.
Returns:
Yields the iterator of the current node reference.
Raises:
JSONPointerException: forwarded from json

‘jsondata.JSONPatch’ - Module

Functions

getOp

JSONPatch

Attributes

  • JSONPatch.data: JSONPatch object data tree.

Methods

__init__
__str__
__repr__
apply
get
patch_export
patch_import
repr_export

Operators

‘()’
‘[]’
‘S+x’
‘S==x’
‘S+=x’
‘S-=x’
‘S!=x’
‘S-x’
len

Iterators

__iter__

JSONPatchItem

Methods

__init__
__repr__
__str__
apply
repr_export

Operators

‘()’
‘[]’
‘S==x’
‘S!=x’

JSONPatchItemRaw

Methods

__init__

Class: JSONPatchFilter

Methods

__init__

Operators

‘==’
‘!=’

‘jsondata.JSONTree’ - Module

The JSONTree module provides features for in-memory JSON structures.

The provided features comprise:

  • The construction and printout of tree formatted structures for screen analysis.
  • Comparison of JSON strings as tree structures.

Constants

Operations modes

  • _interactive = False: Activates interactive mode.

Diff Mode

  • DIFF_FIRST = 0: break display of diff after first
  • DIFF_ALL = 1: list all diffs

Displayed Character Set

  • CHARS_RAW = 0: display character set as raw
  • CHARS_RAW = 1: display character set as str
  • CHARS_RAW = 2: display character set as utf/utf-8

Line-Overflow

  • LINE_CUT = 0: force line fit
  • LINE_WRAP = 1: wrap line in order to fit to length

JSONTree

class jsondata.JSONTree.JSONTree(**kargs)[source]

Variables

  • self.difflist : reaulting differences
  • self.scope: scope of differences
  • self.linefit: handle overflow
  • self.linewidth: line width
  • self.charset: character set
  • self.indent: indention steps

Methods

__init__
JSONTree.__init__(**kargs)[source]

Create an object for the tree representation.

Args:

**kargs: Parameter specific for the operation,

scope:

  • all: Display all diffs.
  • first: Display first diff only.

default:=first

charset:

  • raw: Use ‘raw’.
  • str: Use ‘str’.
  • utf: Use ‘utf’.

default:=raw

debug:

Add developer information.

linefit:

  • cut: Cut lines to length.
  • wrap: Split lines to length.

default:=wrap

indent=#numchars:

Number of characters for indentation.

linewidth=#numchars:

Length of lines.

verbose:

Add progress and status dialogue output.
Returns:
When successful returns ‘True’, else raises an exception.
Raises:
passed through exceptions:
printDiff
JSONTree.printDiff()[source]

Prints out the resulting list of differences.

Args:
ffs.
Returns:
When successful returns tree represantation.
Raises:
passed through exceptions:
fetchDiff
JSONTree.fetchDiff(n0, n1, p=[], dl=0)[source]

Recursive tree compare for Python trees as used for the package ‘json’.

Finds diff in native Python trees assembled by the standard package ‘json’ and compatible, e.g. ‘ujson’.

  • leveltop
  • levelbottom
  • delta (for containers)
  • scope(all, first)
  • linewidth
  • displaycharset (str,utf)
  • pathonly

Args:

n0:

JSON string of type ‘str’, or ‘unicode’

n1:

JSON string of type ‘str’, or ‘unicode’

p=[]:

Result entries for each difference:
{'n0':n0,'n1':n1,'dl':dl,'p':p[:]}
  1. first JSON data
  2. second JSON data
  3. diff count increment value
  4. current diff including path

List of differences as of:

  1. non equal types are different: type(n0) != type(n1)

  2. equal types, both list: type(n0) is list

    1. length is different: len(n0.keys()) != len(n1.keys())
    2. at leats one item is different: n1.get(ni) and v != n1[ni]
  3. equal types, both dict: type(n0) is dict and type(n1) is dict

    1. length is different: len(n0.keys()) != len(n1.keys())
    2. at leats one item is different: n1.get(ni) and v != n1[ni]

default:=0

Returns:
When no diffs returns True, else False or raises an exception. The resulting differences are contained in the provided list parameter ‘p’. When not provided the resulting list is suppressed.
Raises:
passed through exceptions:

Exceptions

class jsondata.JSONTree.JSONTreeException[source]

Error in JSONTree.

‘jsondata.Selftest’ - Module

Test of basic features for the user by ‘–selftest’.

This module is used by ‘jsondc’ when the opverify_data_schemaardoced basic functional checks by calling ‘runselftest’.

The display of actions and results could be activated and raised by multiple repetition of the ‘-v’ option.

The following data and schema are applied:
  1. jsondata/data.json + jsondata/schema.jsd
  2. jsondata/datacheck.json + jsondata/datacheck.jsd
The performed process flow is:
  1. load
  2. validate
  3. verify

By default either ‘True’ is returned, or in case of a failed test and/or error condition an exception is raised.

Basic

  • runselftest
jsondata.Selftest.runselftest(appname='selftest', **kargs)[source]

Performs the selftest returns True or False. Executes some the basic runtime test cases for user verification.

Args:
appname: Name of the application. Changing this may break the
selftest. default:=selftest
**kargs:
debug: Displays extended state data for developers.
Requires __debug__==True.
verbose: Extends the amount of the display of
processing data.
_verbose=#levels: Extends the amount of the display
of processing data by given number of levels at once.
Returns:
Selftest object.
Raises:
bypassed subsystems
  • printverbose
jsondata.Selftest.printverbose(lvl, args)[source]

Load Tests

  • load_data
jsondata.Selftest.load_data(appname)[source]

Loads and verifies the self test ‘data.json’.

Therefore the result of the creation of JSONDataSerializer is compared to the load by json.load().

  • load_appname
jsondata.Selftest.load_appname(appname)[source]

Loads and verifies the self test ‘selftest.json’.

Therefore the result of the creation of JSONDataSerializer is compared to the load by json.load().

Validation Tests

  • verify_data_schema
jsondata.Selftest.verify_data_schema(appname)[source]

Loads and validates the self test ‘data.json’ and ‘schema.jsd’.

Therefore the result of the creation of JSONDataSerializer is performed with draft3 validation by jsonschema.validate().

  • verify_appname_schema
jsondata.Selftest.verify_appname_schema(appname)[source]

Loads and validates the self test ‘selftest.json’ and ‘selftest.jsd’.

Therefore the result of the creation of JSONDataSerializer is performed with draft3 validation by jsonschema.validate().

JSONPointer tests

  • jsonpointer_data_schema
jsondata.Selftest.jsonpointer_data_schema(appname)[source]

Loads and verifies by using JSONPointer access ‘data.json’.

  • jsonpointer_selftest_data
jsondata.Selftest.jsonpointer_selftest_data(appname)[source]

Loads and verifies by using JSONPointer access ‘selftest.json’.

  • jsonpointer_selftest_data_schema
jsondata.Selftest.jsonpointer_selftest_data_schema(appname)[source]

Loads and verifies by using JSONPointer access ‘selftest.json’.