The data mining structure of nodes within the network hierarchy controls the bots you need to run to keep your data feeds up to date.
Data mining is the activity of processing data. You need to process data to feed charts, and so that the trading bot may make decisions based on quality information. In the context of the network hierarchy, the data-mining node groups the task managers handling sensor and indicator bots instances.

Having access to quality information is a crucial element of trading. For that reason, Superalgos strives to give users full control over how, where and when data is processed. You will use this section of the network hierarchy to exert that control.


Data Mining

Task Manager

Task

Sensor Bot Instance

Sensor Process Instance

Market Reference

Key Instance

Indicator Bot Instance

Indicator Process Instance

Market Reference

Task Manager

A task manager is a device used to organize and control any number of tasks, which, in turn, control bot instances. You use a task manager to start or stop several tasks at the same time.

A task manager facilitates the organization of tasks. For example, you may wish to set up a task manager to handle tasks related to a particular set of indicators you use with a certain strategy. Or, for example, to organize multiple backtesting sessions.

Click to learn more about task managers

Adding a Task Manager Node

To add a task manager, select Add Task Manager on the My Computer network node menu. A task manager is added along with a task.

Starting a Task Manager

Select Run All Tasks or Stop All Tasks on the menu to start and stop all tasks respectively.

Task

A task is the device used to control bot instances, that is, to start and stop bots, including sensors, indicators and the trading bot.

Each task controls a single bot. A bot instance running on its own task is independent from other bots at the operating system level, thus, may not be affected by errors ocurring on those other bots.

Click to learn more about tasks

Adding a Task Node

To add a task, select Add Task on the task manager node menu.

Starting a Task

Select Run on the menu to start a task. When a task is started, the process instance of the bot instance attached to the task is started. Also, a visual indication that both the task and the process instance are running appear surrounding the corresponding nodes, in the form of a progress ring.

To stop a task, select Stop on the menu.

Sensor Bot Instance
A sensor bot instance is a reference to a sensor bot as defined in a data mine. The instance of the bot runs the defined processes and generates the defined data products.

A sensor bot is an algorithm that extracts raw data from external sources (i.e.: exchanges, Twitter, etc.) and stores it in a dataset that other bots may consume.

The sensor bot instance holds no definitions as to what the bot does. Instead, its process instance references a process definition in the corresponding data mine. That is how the sensor bot instance obtains the information regarding what it needs to do once it is run.

Click to learn more about sensor bot instances

Adding a Sensor Bot Instance Node

To add a sensor bot instance, select Add Sensor Bot Instance on the task node menu. When a sensor bot instance is added, it is created with one sensor process instance, and a market reference.

Configuring the Sensor Bot Instance

Select Configure Sensor Bot Instance on the menu to access the configuration.

  {
    "startDate": "2020-01-01"
  }
  • startDate is the desired starting date of the data product the sensor bot instance builds, in the YYYY-MM-DD format. The sensor bot instance queries its data source for data starting on the configured startDate.

    • The actual date in which the dataset starts depends on external factors: A. The market may start at a later date. B. The exchange may limit how far in the past data may be retrieved. In both cases, the sensor bot automatically discovers the date closest to the desired starting date that is possible to start with, and proceeds accordingly.

    • In the case the startDate is changed after the sensor bot has started building a data product, either for an earlier or later date, the sensor re-evaluates the feasibility of starting at the new date. The actual date may or may not change; regardless, the sensor bot discards the existing data product and starts over from the newly discovered date. In other words, if the startDate is changed, the sensor bot starts over.

    • Notice that the above starts a chain reaction among all indicator bots that have a data dependency with the sensor bot’s output dataset. Also, if the actual date ends up changing, all indicators that determine the starting date of the market by looking at the date discovered by the sensor bot have to discard their existing data products and start over from the new date.

Starting a Sensor Bot Instance

You do not start or stop a sensor bot instance directly. Instead, you start or stop the corresponding task.

Sensor Process Instance
A sensor process instance is a reference to the process definition of a sensor bot, as defined in a data mine.

For example, in the case of an instance of the Masters data mine Charly sensor bot, the sensor bot process instance references the Historic Trades process definition. Once the reference is established, the sensor process instance adopts the name of the process definition it references.

Click to learn more about sensor process instances

Adding a Sensor Process Instance Node

To add a sensor process instance, select Add Sensor Process Instance on the sensor bot instance node menu. When a sensor process instance is added, it is created with a market reference.

Starting a Sensor Process Instance

You do not start or stop a sensor process instance directly. Instead, you start or stop the corresponding task.

Market Reference
A market reference is a reference to a specific market in a specific exchange, as defined in the Crypto Ecosystem hierarchy. The reference dictates which market the process works with.

In other words, a market reference is the piece of information that lets the process instance know which market of which exchange it needs to process.

Click to learn more about market references

Adding a Market Reference Node

To add a market reference, select Add Market Reference on the sensor, indicator or trading process instance node menu.

Key Instance
The key instance is a reference to an exchange account key as defined in a specific user account, in a specific exchange, on the Crypto Ecosystem hierarchy.

The key instance must be attached to market reference nodes that participate in forward testing and live trading sessions, as that is the scenario in which the user must validate the account with the exchange.

Some exchanges—like Binance—require validating the user even when retrieving data from the exchange. For such reasons, the key instance must also be attached to the market reference of the sensor process instance that connects to such exchanges.

In all cases, the key instance node must reference a valid exchange account key from an account with the exchange, as defined in the Crypto Ecosystem hierarchy.

Click to learn more about key instances

Adding a Key Instance Node

To add a key instance, select Add Key Instance on the market reference node menu.

Indicator Bot Instance
An indicator bot instance is a reference to an indicator bot as defined in a data mine. The instance of the bot runs the defined processes and generates the defined data products.

An indicator bot is an algorithm that processes information that other bots have generated, and produces elaborate datasets for others to consume.

The indicator bot instance holds no definitions as to what the bot does. Instead, its processes instances reference the process definitions in the corresponding data mine. That is how the indicator bot instance obtains the information of what it needs to do once it is run.

Click to learn more about indicator bot instances

Adding an Indicator Bot Instance Node

To add an indicator bot instance, select Add Indicator Bot Instance on the task node menu. When an indicator bot instance is added, it is created with one indicator process instance, and a market reference.

Starting an Indicator Bot Instance

You do not start or stop an indicator bot instance directly. Instead, you start or stop the corresponding task.

Indicator Process Instance
An indicator process instance is a reference to the process definition of an indicator bot, as defined in a data mine.

Indicator bot instances usually require two indicator process instances. One of them references the indicator’s multi-period-market process definition and the second references the multi-period-daily process definition.

Once the reference is established, the indicator process instance adopts the name of the process definition it references.

Click to learn more about indicator process instances

Adding an Indicator Process Instance Node

To add an indicator process instance, select Add Indicator Process Instance on the indicator bot instance node menu. When an indicator process instance is added, it is created with a market reference.

Starting an Indicator Process Instance

You do not start or stop an indicator process instance directly. Instead, you start or stop the corresponding task.