The product definition is the compilation of configurations, references, and JavaScript code that make up the data product.
The product definition node holds all definitions that make up a product, including how the product is calculated and the datasets the product is comprised of.

A bot may feature multiple products, thus, there may be multiple product definitions. Three main aspects need to be defined in terms of products: the characteristics of the dataset, the composition of each record, and the way records are calculated.

Click to learn more about product definitions

Adding a Product Definition Node

To add a product definition, select Add Product Definition on the bot’s node menu. A product definition is created along with the basic structure of nodes required to make up a product.

Configuring the Product Definition

Select Configure Product on the menu to access the configuration.

{
  "codeName": "New-Product-Name",
  "singularVariableName": "newProductName",
  "pluralVariableName": "newProductNames"
}
  • codeName is the name of the product as it is used within the bot’s code.

  • singularVariableName is the singular form of the name of the variable representing the product, as is used on strategies (i.e.: bollingerChannel.direction, where bollingerChannel is the name of the product as defined in its configuration and direction is the name of the property.

  • pluralVariableName is the plural form of the variable.


Product Definition

Dataset Definition

Record Definition

Record Property

Formula

Calculations Procedure

Procedure Loop

JavaScript Code

Procedure Initialization

JavaScript Code

Data Building Procedure

Procedure Loop

JavaScript Code

Procedure Initialization

JavaScript Code

Dataset Definition

A dataset definition contains a configuration file that defines all aspects of the dataset. There are two types of dataset definitions: Multi-Period-Market and Multi-Period-Daily.

A good part of what makes datasets easy to consume by other bots is the fact that they are standardized in terms of their structure. It is that structure that is specified in the dataset definitions.

Click to learn more about dataset definitions

Adding a Dataset Definition Node

To add a dataset definition, select Add Dataset Definition on the bot’s node menu.

Configuring the Dataset Definition

Select Configure Definition on the menu to access the configuration.

Multi-Period-Market:

{
  "codeName": "Multi-Period-Market",
  "type": "Market Files",
  "validTimeFrames": [ "24-hs", "12-hs", "08-hs", "06-hs", "04-hs", "03-hs", "02-hs", "01-hs" ],
  "filePath": "Data-Mine-Name/Bot-Name/@Exchange/Output/Product-Name/Multi-Period-Market/@Period",
  "fileName": "@BaseAsset_@QuotedAsset.json"
}

Multi-Period-Daily:

{
  "codeName": "Multi-Period-Daily",
  "type": "Daily Files",
  "validTimeFrames": [ "45-min", "40-min", "30-min", "20-min", "15-min", "10-min", "05-min", "04-min", "03-min", "02-min", "01-min" ],
  "filePath": "Data-Mine-Name/Bot-Name/@Exchange/Output/Product-Name/Multi-Period-Daily/@Period/@Year/@Month/@Day",
  "fileName": "@BaseAsset_@QuotedAsset.json",
  "dataRange": {
  "filePath": "Data-Mine-Name/Bot-Name/@Exchange/Output/Product-Name/Multi-Period-Daily",
  "fileName": "Data.Range.@BaseAsset_@QuotedAsset.json"
  }
}
  • codeName is the name of the dataset as used within the code.

  • type refers to the type of dataset; possible values are Market Files and Daily Files.

  • validPeriods refers to the time frames handled by the dataset.

  • filePath sets the path on which files are stored; the proper name of the data mine, the bot and the product need to be entered.

  • fileName sets the name of the files that constitute the dataset.

Record Definition

The record definition is where indicators define how many fields, what fields and in which order those fields will be stored in each record.

Bots store data in the form of arrays of records, in a minimized plain text file and the standard JSON format, although not as objects with named properties, but as arrays.

image

The choice of plain text for storage of large volumes of information has benefits in terms of not requiring any sophisticated technology to store and serve the data (i.e.: no database is required).

Best practice indicates that there needs to be a balance between the amount of data that is stored and calculations that may be performed at runtime.

Storing objects with named properties in the JSON format would be inappropriate as labels would repeat over and over, creating a ridiculous amount of unnecessary information. However, storing arrays in the JSON standard facilitates the handling of files in the context of JavaScript and Node.JS.

Click to learn more about record definitions

Adding a Record Definition Node

To add a record definition node, select Add Record Definition on the product definition node menu. A record definition node is created along with the first record property.

Record Property

A record is the compendium of information stored for each period of time, including the start and end datetime of the period along with the properties that provide the information the indicator is meant to generate.

Records may have as many properties as required.

Click to learn more about record properties

Adding a Record Property Node

To add a record property, select Add Record Property on the bot’s node menu.

Formula

In the context of a record definition, formulas are used to assign a value to the property, usually in the form of a variable declared in the calculation or data building procedure.

A formula may act in slightly different ways depending on the isCaculated attribute in the record property configuration:

  • When a property is calculated ("isCalculated": true), the formula assigns to the property a value that is calculated in the calculations procedure. Even if the property is not stored in the dataset, it is made available as a calculated property to other bots that may have the dataset as input.

  • When a property is not calculated ("isCalculated": false or the attribute is not defined in the configuration) the formula is applied in the data building procedure.

Click to learn more about formulas

Adding a Formula Node

To add a formula, select Add Formula on the record property menu.

Calculations Procedure

Bots store data in the minimized array format as a strategy to build the lightest possible datasets. For the same reason, a bot may choose not to store easily calculable information.

For example, imagine an indicator that stores buy and sell volumes… does it need to store the total volume? Given that it is so easy to calculate, the answer is probably no. IT is up to the developer to decide, but it is certainly not required.

The calculations procedure calculates the properties that the developer chose not to store in the dataset.

Click to learn more about calculations procedures

Adding a Calculations Procedure Node

To add a calculations procedure, select Add Missing Items on the product definition node menu. Items that may be missing are created along with the basic structure of nodes required to define them.

Procedure Loop

The procedure loop holds the JavaScript code that reads an input, processes the information, and generates an output recursively.

This is an example of a procedure loop code snippet, in particular, the code that calculates the Popular SMAs product of the Simple Moving Average (SMA) indicator.

/* Loop Code. */

let candle = record.current // Our main dependencies are candles 
variable.last200.push(candle.close) // Add the current close value to the last 200 array.

if (variable.last200.length > 200) {
    variable.last200.splice(0, 1) // Remove the first element of the array to keep it at a maximun of 200 elements.
}

variable.sma20 = calculateSMA(20)
variable.sma50 = calculateSMA(50)
variable.sma100 = calculateSMA(100)
variable.sma200 = calculateSMA(200)

function calculateSMA(periods) {  // Having a function saves us from duplicating code.
    /* We check we have enough values to make the calculation */
    if (variable.last200.length < periods) { return 0 } // If we dont, we define the value is zero.

    let sum = 0 // Initialize sum variable. 
    for (let i = variable.last200.length - periods; i < variable.last200.length; i++) { // Iterate through the last periods
        sum = sum + variable.last200[i]
    }
    let sma = sum / periods
    return sma
}

Click to learn more about procedure loops

Adding a Procedure Loop Node

To add a procedure loop, select Add Procedure loop on the corresponding procedure node menu. A procedure loop node is created along with the corresponding JavaScript node.

JavaScript Code

The JavaScript code node may hold any snippet of valid JavaScript.

The system allows developers great flexibility as all valid JavaScript code is allowed.

Click to learn more about javascript codes

Adding a JavaScript Code Node

To add a JavaScript code node, select Add Code on the corresponding parent node menu.

Procedure Initialization

The procedure initialization is used to initialize variables used by the procedure loop.

This is an example of a procedure loop code snippet, in particular, the code that initializes the array used in the calculation of the Popular SMAs product of the Simple Moving Average (SMA) indicator.

/* Initialization Code. */
variable.last200 = []

Click to learn more about procedure initializations

Adding a Procedure Initialization Node

To add a procedure initialization, select Add Procedure Initialization on the corresponding procedure node menu. A procedure initialization node is created along with the corresponding JavaScript node.

Data Building Procedure

The data building procedure processes information to be fed to properties that are stored in the output dataset.

This procedure holds the core logic of the bot, as it features the code that performs the calculations required to produce the new data set. The code is split between a procedure initialization and a procedure loop.

Click to learn more about data building procedures

Adding a Data Building Procedure Node

To add a data buildings procedure, select Add Missing Items on the product definition node menu. Items that may be missing are created along with the basic structure of nodes required to define them.