JSON Model-to-Model Projection (JM2MP) Documentation

JSON Model-to-Model Projection (JM2MP) Documentation

JM2MP Syntax

Table of Contents

Introduction

In this tutorial, we will explain the syntax of the JSON Model-to-Model Projections (JM2MP) format, which is based entirely on the de jure ECMA-404: The JSON data interchange syntax (2nd edition, December 2017) and de facto RFC 8259: STD 90: The JavaScript Object Notation (JSON) Data Interchange Format standards.

You can find more information about the JSON format at JSON.org.

Basic Terminology

The following concepts are important terms defined as part of JM2MP:

  • Source

    The source is the input JSON document that is transformed by a projection JSON document to obtain the resultant output JSON document.

  • Resultant

    Similarly, the resultant is the output JSON document transformed by the projection.

  • Projection

    It is the algebraic concept of a (macro) function, a JSON document used to transform a source input JSON document into a resultant output JSON document.

    A projection document is composed by at least one named template (called the root template) and which composes the functions (each one called template command) that actually transforms specified input JSON values into the desired output.

  • Template Command

    It is the algebraic concept equivalent to a transformation function. JM2MP defines a number of template commands, each of which performs a specific operation.

  • Named Template

    Just as in most programming languages with functions or procedures, when you want to reuse the functionality provided by the composition of multiple instructions in JM2MP, you can declare named templates.

  • Execution Environment

    The execution environment maintains information about the complete source document, the value of the source document currently being processed, as well as any aliases that may have been defined to simplify the use of template commands and named templates.

    The section execution environment below offers more information about this concept.

Notation

We denote 𝕁 as the complete set of all possible JSON values.

We denote ℙ as the complete set of all template commands.

Due to the literality and homoiconicity of JM2MP with respect to JSON, this means that any JM2MP projection is a valid JSON value, and since its resultant value of such a projection must also be a valid JSON value, we can therefore conclude that ℙ⊆𝕁.

We will denote ⟦Operator⟧ as the named operation to be formalized. When such operation acts over a list of arguments, we will represent the entire operation as ⟦Operator⟧(arguments,…). When the operation is a composition of several operators, we will represent it as ⟦MainOperator(Op1,…,Opn)⟧(arguments,…).

Overview of JM2MP

The JSON Model-to-Model Projection (JM2MP) is designed in two parts: a projection document and a toolkit.

  • The former is a specific document format based on JSON syntax called the projection document, which defines different transformation rules that we will call template commands that can be grouped into named templates.

  • The latter is a set of tools (a toolkit) based on programming modules, capable of both transforming any input JSON document (which we will call the source document) into an arbitrary output JSON document (called the resultant document) following the operations that named templates and template commands describe in the projection document (also based on JSON).

The JM2MP document or projection document is a JSON document that must contain an object as its root value. This object must contain a property named $, called the root template, because as its name implies, it will be the first named template to be invoked as part of the projection process.

To preserve compatibility with the JSON Schema standard, the declaration of a property called $schema is also supported.

Optionally, it may contain another property called $options to allow certain parameters or conditions to be set when processing the projection, making it possible to configure the behavior of the programming module in some way, as we will see later.

If there are other properties in this root object, they will be considered as additional named templates, invocable through their respective names, during the projection process.

Both $schema and $options properties of the root element will always be considered as projection metadata and never named templates.

The only mandatory property in the root object is the root template, with the rest of the properties being entirely optional. So, the only mandatory template for a projection document is the root template.

It will be the author of the projection who, depending on the complexity required and at their sole discretion, determines the appropriate number of templates in each case.

Therefore, the simplest projection that can be constructed is as following:

{ "$" : null }

This projection always generates a JSON document where the output value (the resultant document) is the null literal, regardless of the source document. We will refer to this particular projection as the null projection.

Based in our experience using JM2MP, we recommend to construct projections using a generative style of successive refinement, starting with such null projection and building the required projection guided by the desired output, for instance, using classic Jackson's Structured Programming technics, like decomposition.

Every template in a projection, whether it is the root template or an additional named template, is intended to return a JSON value obtained from the resolution of that template. Therefore, templates can be seen as functions that operate on two set values: the JSON value (a fragment from the projection document) representing the operation itself (which can be arbitrarily complex), and the execution environment (which references the source document, from the root value and from the current value to process, plus additional aliases that can be declared to ease the operations). The result of the template will be a new JSON value that will form part of the resultant document.

As previously stated, the JM2MP.JSlibrary will start to project (to transform) the source document into the resultant document invoking the root template (a kind of applying or executing its equivalent function) over the initial executon environment.

This initial execution environment will consider the root value of the source document as an argument. Such root value will be, usually in any non-trivial JSON documents, a top-level object; but, let's not forget, could be an array or even a simple constant literal.

JM2MP achieves dynamism when generating projections through certain elements called template commands. These commands are actually JSON objects that will be processed by the JM2MP.JS library to obtain their resultant values, calculated according to the corresponding command. These template commands emulate control statements in a programming language or lambda functions that are automatically invoked; in fact, in the JavaScript language from which JSON is derived, there is an idiom called immediately invoked anonymous functions (IIFE).

Note that the projection document of JM2MP is always interpreted as literally as possible, except for template commands. So:

  • Any scalar literal value that appears as part of a projection will be part of the resultant as is (except strings considered as paths, as it will be explained below in the steps and paths section).

  • When an array is part of a projection, all its items will be processed as potential template commands.

  • When an object is part of a projection, all its property values will be processed as potential template commands.

Furthermore, since the process is based on three documents (source, projection, and resultant), all of them in JSON format, both the knowledge and the operations required to construct such projections and templates are greatly simplified.

Is in the nature of JM2MP to treat both source and projection documents as immutable, achieving referential transparency and avoiding side effects, and considering all named templates and template commands as pure functions. So, to put it simply, mathematically speaking, we could say that:

⟦ projection ⟧ ( source , execution_environment ) = resultant

For all these reasons, we can conclude that the JM2MP syntax offers a combination of both declarative and functional paradigms for transforming JSON documents, while also achieving a level of interoperability that a simple script or application might not provide.

Data types

The JSON standard only considers following data types:

  • Scalar values:
    • null
    • Boolean: true and false
    • string
    • number
  • Complex values:
    • Array
    • Object

Current versions of JM2MP and JM2MP.JS are strict handling data types, providing a function to identify the type of a value (see typeof projection below) but not providing functions to implictly cast neither explicitly convert values from one type to another.

Steps and Paths

To locate a specific value within a JSON document, we need to define a mechanism for accessing its structure.

Due to the data types offered by JSON, its information set, we can consider its structure as recursive, because of complex types (arrays and objects). In fact, the information in JSON documents is typically structured as a tree, where the nodes are values and the edges represent membership relationships to arrays and objects. Therefore, all scalar values will be leaf nodes, as will arrays with no elements and objects with no properties. The root node of the tree will thus be the document's root JSON value.

Consequently, to access a specific value in the document (a node within the tree), we need a mechanism to specify how to traverse that tree.

Since the edges (membership relationships) are limited solely to arrays and objects, we can use only: integers to indicate the position of an element in the list (the item’s index in the array) and strings to indicate an object’s property (the name of the key for that property).

With these considerations in mind, we can define a step as an integer or a text string and a path as the finite sequence of steps required to reach and access a specific value.

Currently exist a multitude of mechanisms for locating, searching, and querying JSON documents. For this reason, the JM2MP.JS library facilitates the use of any of these mechanisms, defining an adapter-based interface to make them easy to use within the JM2MP format and incorporating newer ones in the future.

In fact, it is possible to use several of them simultaneously within the same projection, so that you can always choose the most suitable option based on the features offered by that query mechanism.

The template command used for queries is get. By default, JM2MP incorporates a syntax known as native, which simplifies querying by simply using text string literals prefixed in a specific way.

See below the query languages section for more information about this topic.

Template Commands

Template commands are the functions that transforms JSON values.

We divide template commands in different families due of the nature of its intentions inside JM2MP syntax: execution environment, projections, predicates, and operators.

Execution environment

The execution environment, formally defined in JM2MP as an immutable tuple ρ, maintains information about:

  • The root context, represented by the path prefix $, is always a reference to the root value of the entire source document.

  • The current context, represented by the path prefix @, is a reference the value from the source document currently being processed by a template command within a named template.

    Note that two specific template commands enrich their own current context to provide additional capabilities: foldArr and foldObj.

  • Named aliases, represented by the path prefix %, are expressions bound to the current scope in which a template command is being applied (executed), with the intent of simplifying the use of such template command.

As part of any named template, you can use the template commands let and call to define new scopes, specifically to obtain a new execution environment to work with (because, remember, they are immutable).

In its current version, JM2MP.JS will only allow access to the execution environment using these prefixes for the textual variant of the native query language. Thanks to the definition of the QueryAdapter interface, any implementation could propose minor modifications to its represented external query language to allow it to access to this execution environment as well.

let

The let template command is used to create new alias bindings.

An alias is an immutable expression that substitutes for that expression every time it is used, providing referential transparency.

Every let creates a new scope where bindings are defined.

Its JSON form is:

{
  "$op" : "let",
  "$bindings": [
    "<alias_1>" : ℙ,
    "..."       : ℙ,
    "<alias_n>" : ℙ
  ],
  "$in": ℙ
}

All aliases are computed in parallel, so two sibling aliases (defined in same scope) cannot depend on each another.

When you need to create a second alias that depends on a first alias, you must define an inner scope (for the second alias) inside the outer scope (where the first alias is defined), since the bindings clause is evaluated before the in clause.

For example:

{ "$op" : "let",
  "$bindings": [
    "outer_alias" : "<Free expression>"
   ],
  "$in" : {
    "$op" : "let",
    "$bindings" : [
      "inner_alias" : "<Dependant expression of %outer_alias>"
    ],
    "$in" : ℙ
  }
}

call

The call template command is used to invoke an existing named template, just as if it were a function.

Its basic JSON form is:

{
  "$op"  : "call",
  "$ref" : "<Declared named template>"
}

Optionally, it allows to change the current context using its at clause:

{
  "$op"  : "call",
  "$ref" : "<Declared named template>",
  "$at"  :  ℙ
}

Projections

Projections are operations that actually transform an input JSON value (or values) into an output JSON value.

pipe

The pipe template command is the projection that represents the sequential associative composition and, as its name implies, it composes all declared transformations left to right.

Its JSON form is:

{
  "$op"     : "pipe",
  "$stages" : [ ℙ_1 , … , ℙ_n ]
}

where its algebraic equivalence is as follows:

⟦ pipe ( ℙ1 , … , ℙn ) ⟧ ( ρ ) = ℙn ∘ … ∘ ℙ1 ( ρ ) = ℙn ( … ( ℙ1 ( ρ ) ) … )

get

The get template command is the projection that does a selection of values in the specified path and, optionally, from the specified environment (context, root or alias).

Its basic JSON form is:

{
  "$op"   : "get",
  "$path" : "<Path to the specified source values>"
}

It is also possible to define which query language is used (clause syntax), and also change its current context (clause from):

{
  "$op"     : "get",
  "$syntax" : "<Name of the query language used in $path>",
  "$path"   : "<Path to the specified source values>",
  "$from"   : ℙ
}

if

The if template command is the projection that projects its then clause when the cond clause is evaluated as true; otherwise, it projects its else clause.

The if template command uses short-circuit evaluation, meaning that the else clause is only projected if and only if the resultant value from cond projection is false.

Its JSON form is:

{
  "$op"   : "if",
  "$cond" : ℙ,
  "$then" : ℙ,
  "$else" : ℙ
}

where its algebraic equivalence is as follows:

⟦ if ( cond , then , else ) ⟧ ( ρ ) = { ⟦ then ⟧ ( ρ ) , when ⟦ cond ⟧ ( ρ ) = true ⟦ else ⟧ ( ρ ) , otherwise

coalesce

The coalesce template command mixes the if and get projections in order to simplify obtaining a default value only when it is not possible getting the specified one.

Its JSON form is:

{
  "$op"      : "coalesce",
  "$value"   : ℙ,
  "$default" : ℙ
}

foldArr

The foldArr template command is the projection that projects its step clause over all items of array over, from right to the left, starting by init.

It is a catamorphism over ARRAY(𝕁), meaning that step is actually a template command used as a function to reduce, transform or eliminate every item resultant from over clause:

foldArr : A * × B × ( A × B → B ) → B

where: A,B⊆ℙ.

Its JSON form is:

{
  "$op"   : "foldArr",
  "$over" : ℙ,
  "$init" : ℙ,
  "$step" : ℙ
}

The foldArr projection changes the context in every step evaluation, considering the following object as its current context:

const foldArr_step_context = {
  // The current item from OVER array.
  /** @type {*} */ item: over[i],
  // The current value of the accumulator.
  /** @type {*} */ acc,
  // The index of the current element from OVER array.
  /** @type {integer} */ index: i
};

foldObj

Similarly, the foldArr template command is the projection that projects its step clause over all properties of object over, starting by init.

It is a catamorphism over MAP(Σ*⇀𝕁), meaning that step is actually a template command used as a function to reduce, transform or eliminate every property of over:

foldObj : { Σ * ⇀ A } × B × ( Σ * × A × B → B ) → B

where: A,B⊆ℙ.

Becase foldObj operates over objects (the finite set of partial functions from keys to values), its step operation must be commutative respect property names (distinct key-values pairs). It is responsibility of the author of such projection to guarantee this restriction; otherwise, JM2MP.JS cannot guarantee deterministic results.

Its JSON form is:

{
  "$op"   : "foldObj",
  "$over" : ℙ,
  "$init" : ℙ,
  "$step" : ℙ
}

The foldObj projection changes the context in every step evaluation, considering the following object as its current context:

const foldObj_step_context = {
  // The current property's name from OVER object.
  /** @type {string} */ key,
  // The current property's value from OVER object.
  /** @type {*} */ value: obj[key],
  // The current accumulator.
  /** @type {*} */ acc
};

cons

The cons template command is the projection used to create new a item into an existing array, and insert it as its first element.

Its JSON form is:

{
  "$op"   : "cons",
  "$head" : ℙ,
  "$tail" : ℙ
}

insert

The insert template command is the projection used to create new a property into an existing object.

If the existing object into already has a property with name key, its old value will be lost and value will be its new value (overriding it).

Its JSON form is:

{
  "$op"    : "insert",
  "$key"   : ℙ,
  "$value" : ℙ,
  "$into"  : ℙ
}

sort

The sort template command is a projection used to sort the items of an array, using a specified criteria.

The sort template command is actually a derived projection, included by its reduced algorithmic complexity O(n) againts the non-trivial combination of foldArr, if and lt projections of complexity O(n2).

Its JSON form is:

{
  "$op"   : "sort",
  "$over" : ℙ,
  "$by"   : ℙ,
  "$desc" : ℙ
}

where:

  • over is the mandatory list to sort.
  • by is an optional projection to produce the ordering key; if ommited, then the ordering criteria will be to sort by the items themselves.
  • desc is an optional clause that must be evaluated as a logic predicate (Boolean); by default, its value false will order over in ascending order.

lookup

The lookup template command is a projection used to get the value of the property's name specified by the key clause of the object specified by the in clause.

The lookup projection can be derived from the foldObj projection, but with a higher complexitiy of O(n) instead of just O(1); nowadays, objects are hash tables and getting any key is considered a constant-time operation.

Its JSON form is:

{
  "$op"  : "lookup",
  "$key" : ℙ,
  "$in"  : ℙ
}

The lookup projection returns a null value whenever in is evaluated to null or key does not belongs to the resultant (object) value of in.

An EvaluationError will be raised if the resultant value of in were not of object type.

merge

The merge template command is a projection used to merge two objects, left and right, into a newer resultant one, getting all properties from both but, in case of duplicate keys, returning only the values from the right object (applying priority right-to-left, or overriding the left values with the right ones on equally-named properties).

The merge projection can be derived from the foldObj and insert projections, but with a higher complexitiy of O(m2+m·n) instead of just O(m+n); nowadays, objects can be efficiently combined (for instance, using the JavaScript's spread syntax in object literals).

Its JSON form is:

{
  "$op"    : "merge",
  "$left"  : ℙ,
  "$right" : ℙ
}

Predicates and Operators

Because some JM2MP projections evaluates conditions, it is neccesary to consider predicates, that is, functions or operations that obtains a logical value (a Boolean true or false value) in order to take some considerations.

At the same time, it is neccessary to establish mechanisms that allow operations to be performed on scalar values in order to obtain resultant values derived from one or more source values.

In general, their algebraic equivalence is as follows:

⟦ UnaryOperation ( value ) ⟧ ( ρ ) = Operator ( ⟦ value ⟧ ( ρ ) )
⟦ BinaryOperator ( left , right ) ⟧ ( ρ ) = ( ⟦ left ⟧ ( ρ ) Operator ⟦ right ⟧ ( ρ ) )

Predicates and operators are divided in several categories due to its nature: Boolean logic operators, logic predicates, aritmetic (for numbers), operators (for strings), and the last one miscellaneous category.

Boolean logic operators

Although, strictly speaking from an algebraic standpoint, only the not and and operators are needed to combine any predicate in propositional logic, the or operator has also been added for convenience.

Each of them is explained below.

not

The not template command projects the negation of its value clause.

Only projections that resultant in Boolean values are considered, raising an EvaluationError exception otherwise.

Its JSON form is:

{
  "$op"    : "not",
  "$value" : ℙ
}
and

The and template command projects the conjunction of its left and right clauses.

Only projections that resultant in Boolean values are considered, raising an EvaluationError exception otherwise.

Its JSON form is:

{
  "$op"    : "and",
  "$left"  : ℙ,
  "$right" : ℙ
}
or

The or template command projects the disjunction of its left and right clauses.

Only projections that resultant in Boolean values are considered, raising an EvaluationError exception otherwise.

Its JSON form is:

{
  "$op"    : "or",
  "$left"  : ℙ,
  "$right" : ℙ
}

Logic predicates

Once again, although strictly speaking from an algebraic point of view, only the operators eq and lt are needed to combine any predicate in propositional logic that involves a numerical order, the rest of the operators commonly used in these cases have been also defined for the sake of convenience.

Each of them is explained below.

has

Because JM2MP provides an algebraic closure for JSON data types, any undefined operation (such as searching for a value using a path that does not exist) returns a null value.

For this reason, the has projection is provided, which determines whether a key exists in the specified object.

Its JSON form is:

{
  "$op"  : "has",
  "$key" : ℙ,
  "$in"  : ℙ
}
eq

The eq projection provides in JM2MP binary numerical operation equal.

Its JSON form is:

{
  "$op"    : "eq",
  "$left"  : ℙ,
  "$right" : ℙ
}
lt

The lt projection provides in JM2MP binary numerical operation less than.

Its JSON form is:

{
  "$op"    : "lt",
  "$left"  : ℙ,
  "$right" : ℙ
}
gt

The lt projection provides in JM2MP binary numerical operation greater than.

Its JSON form is:

{
  "$op"    : "gt",
  "$left"  : ℙ,
  "$right" : ℙ
}
lte

The lt projection provides in JM2MP binary numerical operation less than or equal.

Its JSON form is:

{
  "$op"    : "lte",
  "$left"  : ℙ,
  "$right" : ℙ
}
gte

The lt projection provides in JM2MP binary numerical operation greater than or equal.

Its JSON form is:

{
  "$op"    : "gte",
  "$left"  : ℙ,
  "$right" : ℙ
}
neq

The neq projection provides in JM2MP binary numerical operation not equal.

Its JSON form is:

{
  "$op"    : "neq",
  "$left"  : ℙ,
  "$right" : ℙ
}

Aritmetic Operators (for Numbers)

The classic arithmetic operations, both binary and unary, have been defined in JM2MP, namely: addition, subtraction, multiplication, division, modulo, negation, and absolute value.

add

The add projection provides in JM2MP binary numerical operation addition.

Its JSON form is:

{
  "$op"    : "add",
  "$left"  : ℙ,
  "$right" : ℙ
}
sub

The sub projection provides in JM2MP binary numerical operation substraction.

Its JSON form is:

{
  "$op"    : "sub",
  "$left"  : ℙ,
  "$right" : ℙ
}
mul

The mul projection provides in JM2MP binary numerical operation multiplication.

Its JSON form is:

{
  "$op"    : "mul",
  "$left"  : ℙ,
  "$right" : ℙ
}
div

The mul projection provides in JM2MP binary numerical operation division.

Its JSON form is:

{
  "$op"    : "div",
  "$left"  : ℙ,
  "$right" : ℙ
}
mod

The mul projection provides in JM2MP binary numerical operation modulo.

Its JSON form is:

{
  "$op"    : "mod",
  "$left"  : ℙ,
  "$right" : ℙ
}
neg

The neg projection provides in JM2MP unary numerical operation negation.

Its JSON form is:

{
  "$op"    : "neg",
  "$left"  : ℙ,
  "$right" : ℙ
}
abs

The abs projection provides in JM2MP unary numerical operation absolute value.

Its JSON form is:

{
  "$op"    : "abs",
  "$left"  : ℙ,
  "$right" : ℙ
}

Operators (for Strings)

Similar to numbers, but in this case for text strings, the classic operations, both binary and unary, have been defined in JM2MP, namely: concatenation, lenght, substring, to-upper-case, and to-lower-case.

concat

The concat projection provides in JM2MP functionality to concatenate several strings into a newer one.

Its JSON form is:

{
  "$op"    : "concat",
  "$parts" : ℙ
}
lenght

The length projection provides in JM2MP the ability to count characters from a string value, but also items from an array.

Its JSON form is:

{
  "$op"    : "length",
  "$value" : ℙ
}
substring

The substring projection provides in JM2MP the ability to select inner characters from a string value, specifying the position of initial character (zero-based) and, optionally, also the position of the final character (but not including it).

Its JSON form is:

{
  "$op"    : "substring",
  "$value" : ℙ,
  "$start" : ℙ,
  "$end"   : ℙ
}
upper

The upper projection provides in JM2MP the ability to transform all characters from a string value into their corresponding upper-case character.

Its JSON form is:

{
  "$op"    : "upper",
  "$value" : ℙ
}
lower

Similarly, the lower projection provides in JM2MP the ability to transform all characters from a string value into their corresponding lower-case character.

Its JSON form is:

{
  "$op"    : "lower",
  "$value" : ℙ
}

Miscellaneous

Th miscellaneous category currently offers only one template command for type identification. In future lines of work this functionality could be expanded to allow, for instance, conversions between data types.

typeof

The typeof template command provides in JM2MP the ability to detect the exact JSON data type of a value, that is: null, boolean, number, string, array or object.

Its JSON form is:

{
  "$op"    : "typeof",
  "$value" : ℙ
}

Note that JSON data types are a subset of JavaScript's data types. In the above data types section, you can examine them all.

Named Templates

Named templates are all properties of the root object from the projection document or projection module whose identifier is neither the root template ($) nor metadata ($options and $schema).

Just as in any programming languages, where a sequence of instructions is given a name and encapsulated in a function or procedure so that it can be called multiple times from different points in the program, named templates are used to group a set of template commands and JSON values to obtain a resultant value each time they are invoked (using call template command), thereby avoiding the repetition of the same structure at different points in the projection.

Since JM2MP treats object properties as commutative sets, the order in which named templates appear in the projection document is irrelevant. However, you must take into account the import rules for projection modules, because when the same named template identifier is used in several different modules, the last ones to be imported might override previously imported named templates.

Also important to note is that call resets aliases to ∅ (the empty set) deliberately: no previously aliases can be used by a invoked named template). Named templates are autonomous functions of their context, so their behaviour is predictable regardless of who invoked them (and when).

Considering the next example, where the source document contains only an integer (number) value:

5

the projection document contains the Gregory-Liebniz (or Madhava-Leibniz) Series formula to calculate an approximation of number Pi (π) in a recursive manner:

π = 4 · ∑ n = 0 ∞ ( ( -1 ) n ( ( 2 · n ) + 1 ) ) 
{
  "$": {
    "$op": "mul",
    "$left": 4,
    "$right": { "$op": "call",
                "$ref": "Pi",
                "$at": 0 }
  },

  "Pi": {
    "$op": "if",
    "$cond": {
      "$op": "lt",
      "$left":  { "$op": "get", "$path": "@" },
      "$right": { "$op": "get", "$path": "$" }
    },
    "$then": {
      "$op": "sub",
      "$left": { "$op": "div",
                 "$left": 1,
                 "$right": { "$op": "add",
                             "$left": { "$op": "mul",
                                        "$left": 2,
                                        "$right": { "$op": "get", "$path": "@" } },
                             "$right": 1 }
      },
      "$right": { "$op": "call",
                  "$ref": "Pi",
                  "$at": { "$op": "add",
                           "$left": { "$op": "get", "$path": "@" },
                           "$right": 1 }
      }
    },
    "$else": 0
  }
}

and the resultant document of such projection will be the π approximation for such number of iterations:

3.33968253968254

Query Languages

A query language is the mechanism used to locate (search for and select) JSON values within a JSON document.

There are several languages designed for this purpose, ranging from the simplest (which simply allow you to locate any element in the document) to the most sophisticated (which offer pattern-matching searches, as well as additional filtering and sorting operations, among others).

The JM2MP syntax provides a mechanism for using any (hypothetical) query language that can be embedded in JSON, either as a string literal or as any other JSON value (tipically arrays or objects). To do this, there are two options:

  1. Any document or projection module can tell the JM2MP.JS library which query language to use by default in that document by declaring its name in the $default-query-language property of the $options object.

  2. The template command get supports the syntax clause to indicate that the query written in the path clause uses such language.

Available Query Languages

The JM2MP.JS library provides built-in support for several query languages widely used in the industry: JMESPath, JSONata, JSONPath, JSON Pointer, and JSON Query. You can find information about each one of them in the external references section, below.

In addition, the JM2MP format offers its own query language syntax, simply called native, which we will briefly describe next.

Native

The JM2MP format offers its own query language syntax, simply named native. Both formats, JM2MP and native, have been designed to be algebraically complete, in the sense that they provide, with mathematical rigor, at least the minimal set of operations necessary to achieve complete location and transformations between JSON documents.

The native query language features two syntactically different but semantically equivalent notations: one based on text strings and another based on JSON syntax.

Please refer to the Native Query Language tutorial for complete details about this query language syntax and how to use it.

External References

The JM2MP.JS library also references several external libraries to enable the use of different query languages as part of JM2MP projections.

At the same time, it is possible to use additional query languages if proper adapter class is developed. Please, refer to any external reference to see how to do so.

JMESPath

See JMESPath external reference for more information.

JSONata

See JSONata external reference for more information.

JSONPath

See JSONPath external reference for more information.

JSON Pointer

See JSON Pointer external reference for more information.

JSON Query

See JSON Query external reference for more information.

Other Query Languages

It is possible to use other query languages not initially referenced by JM2MP.JS... or even create your own! In order to do so, it is required to develop an adapter class that complies with the QueryAdapter interface, same as any other external reference that has already been mentioned.

Remember that any JM2MP module that will use any query language different than native, needs to declare its usage as part of $options root property or as part of any $get template command.

Modularization

In order to simplify large or complex projection documents, modularization can be achieved using an optional $depends-on property inside the $options inner object of the root object; the value of this property must be an array of string items, where each string will be interpreted by JM2MP.JS in different manners (as filenames, as URLs or as literal JSON content) whose content will be imported into current projection document following the exact order of this list.

Each new projection content, in turn, could have its own (ordered) list of_dependencies. It is worth mentioning that, in case of conflict, named templates that already exist in the projection document performing the import will take precedence, overriding (substituting) any named templates with the same name that may exist in the imported document.

  • Projection document and projections modules

    There is no real difference between a projection document and a projection module. It is customary to name document to whatever file (URL or content) that will be used as a starting point to run a projection, considered then as the root projection document.

  • Root and named templates

    Any projection document or module can contain any named template, including the root template. Or any at all, just dependencies.

  • Overriding templates

    Any root or named template can be overriden by any module with higher precedence.

    If a projection module Higher depends on a projection module Lower then both will have a form similar to:

// Higher
{
  "$options": {
    "$depend-on" : [ "Lower", ... ],
    // If no default query language is declared, then NATIVE is considered.
  },
  "$" : "The root template.",
  "CommonNamedTemplate" : "Higher precedence, it will remain."
}

// Lower
{
  "$option" : {
    "$default-query-language": "Language only considered to be default in this LOWER module."
   },
  "CommonNamedTemplate" : "Lower precedence, it will be overriden."
}

  • Metadata

    Both $options and $schema properties of can appear on any projection document or module affecting only to that module and not propagating its respectives values outside that module.

    That means that if some projection module configure its $default-query-language, such configuration only affects to that module and no other one (neither dependant nor imported).

    Please, refear to $options and $schema sections for more information.

Options

The $options property of the root object of any projection is always considered as metadata (not a named template) and is always optional.

It is possible to define certain values at this special object $options as property of the projection's root element, which will be interpreted by JM2MP.JS processors to get: version assurance, simple user's documentation, to set the default query language to be considered, and to declare the dependencies for modularization purposes.

So, $options object for a projection document or module should comply with the following form:

{
  "$options": {
    "$version": "1.0",
    "$annotations": 𝕁,
    "$default-query-language": STRING,
    "$depends-on": [ STRING_1, ..., STRING_n ]
  }
}

Version

An optional $version property can be declared to specify which version of JM2MP may be interpreted; currently only a value of "1.0" is accepted, which is the format described in this documentation.

Annotations

An optional $annotations property can be declared to specify user's documentation; any JSON value can be used, as simple or complex as authors of the projection document or module consider; its value will be fully ignored.

Default Query Language

An optional $default-query-language property can be declared to specify which available query language will be used by default by the JM2MP.JS processor (that is, JM2MP.JS) whenever a text literal query language expression was used instead of a JSON query language expression, as part of any query language syntax used in the projection document or module. If it is not declared, its value will be considered as "native" (see native query language).

Depends On

In order to simplify large or complex projection documents, modularization can be achieved using an optional $depends-on property; the value of this property must be an array of strings, where each string would be interpreted by JM2MP.JS in different manners (as filenames, as URLs or as literal JSON content) whose content will be imported into current projection document following the exact order of this list.

Each new projection content, in turn, could have its own (ordered) list of dependencies. It is worth mentioning that, in case of conflict, named templates that already exist in the projection document performing the import will take precedence, overriding (substituting) any named templates with the same name that may exist in the imported documents.

JSON Schema

The $schema property of the root object of any projection is always considered as metadata (not a named template) and is always optional.

For compatibility reasons, JM2MP supports the declaration of such property in any projection document, in order to comply with the JSON Schema standard. Although currently the JM2MP.JS library neither uses nor validates against this schema, it was considered useful to reserve such possibility from the outset.

The URL https://json-mde.tech/schemas/json-schema/draft--2020-12/jm2mp--1-0-0.json can be used to specifify the current version of JM2MP using JSON Schema (Draft 2020-12).

Examples

Please, refer to the Native Query Language and Examples tutorials, for various examples of how to use the JM2MP format to transform JSON documents.