The product definition is the compilation of configurations, references, and JavaScript code that make up the data product.
The product definition node holds all definitions that make up a product, including how the product is calculated and the datasets the product is comprised of.

A bot may feature multiple products, thus, there may be multiple product definitions. Three main aspects need to be defined in terms of products: the characteristics of the dataset, the composition of each record, and the way records are calculated.

Click to learn more about product definitions

Adding a Product Definition Node

To add a product definition, select Add Product Definition on the bot’s node menu. A product definition is created along with the basic structure of nodes required to make up a product.

Configuring the Product Definition

Select Configure Product on the menu to access the configuration.

{
  "codeName": "Product-Name",
  "singularVariableName": "productName",
  "pluralVariableName": "productNames"
}
  • codeName is the name of the product as it is used within the bot’s code.

  • singularVariableName is the singular form of the name of the variable representing the product, as is used on strategies (i.e.: bollingerChannel.direction, where bollingerChannel is the name of the product as defined in its configuration and direction is the name of the property.

  • pluralVariableName is the plural form of the variable.


Product Definition

Dataset Definition

Record Definition

Record Property

Formula

Calculations Procedure

Procedure Loop

JavaScript Code

Procedure Initialization

JavaScript Code

Data Building Procedure

Procedure Loop

JavaScript Code

Procedure Initialization

JavaScript Code

Dataset Definition

A dataset definition contains a configuration file that defines many aspects of the dataset. There are two types of dataset definitions: Multi-Period-Market and Multi-Period-Daily.

A good part of what makes datasets easy to consume by other bots is the fact that they are standardized in terms of their structure. It is that structure that is specified in the dataset definitions.

Click to learn more about dataset definitions

Adding a Dataset Definition Node

To add a dataset definition, select Add Dataset Definition on the bot’s node menu.

Configuring the Dataset Definition

Select Configure Definition on the menu to access the configuration.

Multi-Period-Market:

{
    "codeName": "Multi-Period-Market",
    "type": "Market Files",
    "validTimeFrames": [ "24-hs", "12-hs", "08-hs", "06-hs", "04-hs", "03-hs", "02-hs", "01-hs" ],
    "filePath": "@Exchange/@BaseAsset-@QuotedAsset/@DataMine/@Bot/Output/@Product/@Dataset/@Period",
    "fileName": "Data.json"
  }

Multi-Period-Daily:

{
    "codeName": "Multi-Period-Daily",
    "type": "Daily Files",
    "validTimeFrames": [ "45-min", "40-min", "30-min", "20-min", "15-min", "10-min", "05-min", "04-min", "03-min", "02-min", "01-min" ],
    "filePath": "@Exchange/@BaseAsset-@QuotedAsset/@DataMine/@Bot/Output/@Product/@Dataset/@Period/@Year/@Month/@Day",
    "fileName": "Data.json",
    "dataRange": {
      "filePath": "@Exchange/@BaseAsset-@QuotedAsset/@DataMine/@Bot/Output/@Product/@Dataset",
      "fileName": "Data.Range.json"
    }
  }
  • codeName is the name of the dataset as used within the code.

  • type refers to the type of dataset; possible values are Market Files and Daily Files.

  • validTimeFrames refers to the time frames handled by the dataset.

  • filePath sets the path on which files are stored; the proper name of the data mine, the bot and the product need to be entered.

  • fileName sets the name of the files that constitute the dataset.

Record Definition

The record definition is where indicators define how many properties, what properties and in which order those properties will be made available in each record.

Bots store data in the form of arrays of records, in a minimized plain text file and the standard JSON format, although not as objects with named properties, but as arrays.

Typical dataset file showing records as arrays.

The choice of plain text for storage of large volumes of information has benefits in terms of not requiring any sophisticated technology to store and serve the data (i.e.: no database is required).

Best practice indicates that there needs to be a balance between the amount of data that is stored and calculations that may be performed at runtime.

Storing objects with named properties in the JSON format would be inappropriate as labels would repeat over and over, creating a ridiculous amount of unnecessary information. However, storing arrays in the JSON standard facilitates the handling of files in the context of JavaScript and Node.JS.

Click to learn more about record definitions

Adding a Record Definition Node

To add a record definition node, select Add Record Definition on the product definition node menu. A record definition node is created along with the first record property.

Record Property

A record property is a piece of information that may be stored as part of the dataset or made available as a calculated property.

Records may have as many properties as required.

Click to learn more about record properties

Adding a Record Property Node

To add a record property, select Add Record Property on the bot’s node menu.

Configuring the Record Property

Select Configure Property on the menu to access the configuration.

{
  "codeName": "record-name",
  "isString": false,
  "isCalculated": true
}
  • codeName is the name of the record as it is used in the bot’s code.

  • isString determines if the field is a text string or if, in turn, is a numeric field. Dates are stored in the epoch format, which is numeric. The reason why this configuration is important is that fields that are strings need to be stored between “double quotes”.

  • isCalculated determines if the field is stored in the dataset or if, instead, is calculated at a later stage. A value true means that the record is not stored. In the case isCaculated is not defined in the configuration, it is assumed to be false.

Indicators-Record-Definitions-01

The image above shows a record definition with four record properties.

Formula

In the context of a record definition, formulas are used to assign a value to the property, usually in the form of a variable declared in the calculation or data building procedures.

Formulas may use both the record and the variable objects made available by the calculations procedure and the data building procedure (see procedure loop).

A formula may act in slightly different ways depending on the isCaculated attribute in the record property configuration:

  • When a property is calculated ("isCalculated": true), the formula assigns to the property a value that is calculated in the calculations procedure. Even if the property is not stored in the dataset, it is made available as a calculated property to other bots that may have the dataset as input.

  • When a property is not calculated ("isCalculated": false or the attribute is not defined in the configuration) the formula is applied in the data building procedure.

Click to learn more about formulas

Adding a Formula Node

To add a formula, select Add Formula on the record property menu.

Calculations Procedure

The calculations procedure processes information to be fed to calculated properties. Calculated properties are not stored in the data set; instead, they are calculated in real-time as the data is consumed by others.

Bots store data in the minimized array format as a strategy to build the lightest possible datasets. For the same reason, a bot may choose not to store easily calculable information.

For example, imagine an indicator that stores buy and sell volumes… does it need to store the total volume? Given that it is so easy to calculate, the answer is probably not. It is up to the developer to decide, but it is certainly not required.

Click to learn more about calculations procedures

Adding a Calculations Procedure Node

To add a calculations procedure, select Add Missing Items on the product definition node menu. Items that may be missing are created along with the basic structure of nodes required to define them.

Data Building Procedure

The data building procedure processes information to be fed to properties that are stored in the output dataset.

This procedure holds the core logic of the bot, as it features the code that performs the calculations required to produce the new data set. The code is split between a procedure initialization and a procedure loop.

Click to learn more about data building procedures

Adding a Data Building Procedure Node

To add a data buildings procedure, select Add Missing Items on the product definition node menu. Items that may be missing are created along with the basic structure of nodes required to define them.

Procedure Loop

The procedure loop holds the JavaScript code that reads an input, processes the information, and generates an output recursively.

Click to learn more about procedure loops

Adding a Procedure Loop Node

To add a procedure loop, select Add Procedure loop on the corresponding procedure node menu. A procedure loop node is created along with the corresponding JavaScript node.

JavaScript Code

The JavaScript code node may hold any snippet of valid JavaScript.

The system allows developers great flexibility as all valid JavaScript code is allowed.

Click to learn more about javascript codes

Adding a JavaScript Code Node

To add a JavaScript code node, select Add Code on the corresponding parent node menu.

Procedure Initialization

The procedure initialization is used to initialize variables used by the procedure loop.

This is an example of a procedure loop code snippet, in particular, the code that initializes the array used in the calculation of the Popular SMAs product of the Simple Moving Average (SMA) indicator.

/* Initialization Code. */
variable.last200 = []

Click to learn more about procedure initializations

Adding a Procedure Initialization Node

To add a procedure initialization, select Add Procedure Initialization on the corresponding procedure node menu. A procedure initialization node is created along with the corresponding JavaScript node.