:: Version 2.7.0 ::

Synchronize data from an external application

Prerequisite: You must be familiar with the Data Synchronization Reference.

learning:
Create your first collector
level:
Intermediate
domains:
PHP, DataSynchro, JSON
min version:
2.1.0

In this tutorial you will learn how to write/configure your own collector.
A collector is a PHP standalone application, which get data from a source application and push them within iTop, on a regular basis, for example daily. It's a simple ETL (Extract/Transform/Load) application.
You won't have to write the full application, we have written for you a large part that you can download and enhance.

This tutorial will just describe the steps to perform, for details of the syntax to use refer to Data collector Base

First question: How to get data from your source application. How can it be queried?

  1. Directly querying the database in SQL?
  2. Using REST and webservice, with returned data in JSON format?
  3. Directly using CSV files produced by the application?
  4. Using other sorts of webservices, such as for example ripcord?

For each of the first 3 methods, a predefined collector exist already with the SDK, so writing your own based on those is mainly a matter of writing configuration files. For the last case, there is more work to do in PHP.

Second question: On which server will you install your collector?

  • The collector is a standalone application, which need to be started by a cron on a regular basis.
  • The collector must have access to the data of your source application
  • The Collector must have web access to your iTop webservices, but does not need to be on the same machine.
  • The collector must comply to those Data collector Base - Requirements

In this tutorial, we will consider a situation where you want to feed Locations objects in your iTop with Sites information from your application Blabla which you can get using REST/JSON. In this case the actions to perform are:

Create the Collector

On the Collector server

  • Download and unzip the Data collector Base package on the chosen collector server.
  • On that machine create a PHP file under /collectors/sitescollector.class.inc.php like this:
/collectors/sitescollector.class.inc.php
  <?php
  class SiteCollector extends JsonCollector
  {
  }
  • and another one /collectors/main.php like this:
/collectors/main.php
  <?php
  require_once(APPROOT.'collectors/sitecollector.class.inc.php');
  Orchestrator::AddCollector(1, 'sitecollector');
  • update the file /conf/params.local.xml to set the 3 following parameters:
/conf/params.local.xml
  <itop_url>https://????/</itop_url>
  <itop_login>????</itop_login>
  <itop_password>????</itop_password>

Define the iTop side

On iTop

On your iTop, create a DataSynchro Locations from Blabla to synchronize Sites data from Blabla application. Doing this, ask yourself:

  • What are the available information (fields) on the Site object within the Blabla application?
  • Which one need to be synchronized?
  • Should new Location be created or should it only update existing one?

On the Collector server
Then on your collector server, run this command:

php toolkit/dump_tasks.php --task_name="Location from Blabla" > collectors/SitesCollector.json

It extracts a definition of the Synchro Data Source, which allow the collector to force its existance and behavior on your iTop.

Run the above step again each time you modify your Synchro Data Source on your iTop

The created file collectors/SitesCollector.json is called the JSON definition file. It contains the information required to create or update automatically, the Data Synchro Source on your iTop. Also it is also written in JSON format, do not mix it up with the JSON file which will be coming from the Blabla application.

Define the Mapping

Now you have to write an XML file specifying the mapping logic between you Blabla Sites data format and the iTop Location objects. The exact syntax depends on the format of your source data (OQL, CSV or JSON). Here we use JSON, so we need to define:

Tag Usage
<json_data> Is used to get from the Blabla application the Site objects and for each a given set of fields.
This part depends entirely on REST/JSON interface offered by the Blabla application
<path> Specify how to retrieve the fields code and value within the Blabla returned json response. The supported syntax is limited to tag name separated by /. A tag can be a real label in the json returned file or just a star * for level without label, such as a table.
<fields> specifies for each iTop field, the corresponding Blabla field.
Within that mapping there is a special mandatory tag <primary_key> which must be mapped to a Blabla field which identify uniquely a Site in the Blabla application.
<defaults> allow to specify itop fields for which we want to set fixed values. It can be used in combination with a mapping, for the case where a particular Blabla Site has no value for this field.

Those information must be specified in /conf/params.distrib.xml, file, adding the following entries under <parameters> tag:

/conf/params.distrib.xml
<parameters>
    <!-- Class name of the collector in lowercase -->
    <sitecollector> 
        <jsonurl>http://blabla.demo.com/webservices/rest.php</jsonurl>
        <jsonpost>
            <auth_user>blabla-rest-user</auth_user>
            <auth_pwd>blabla-rest-user-password</auth_pwd>
            <json_data>
        <!--
          This part depends entirely on REST/JSON interface offered by the Blabla application
        -->
            </json_data>
            <version>1.3</version>
        </jsonpost>
        <!-- "path" specify how to retrieve the fields with the JSON response -->
        <path>objects/*/fields</path>  
        <fields> 
            <primary_key>blabla_id</primary_key> <!-- tag primary_key is mandatory -->
            <name>name</name>
            <country>land</country>
            <status>step</status>
         </fields>
         <defaults> <!-- This part allow to specify default values for iTop Location field -->
            <org_id>Demo</org_id>
            <status>active</status>
         </defaults>
     </sitecollector>
</parameters>
2_7_0/customization/synchronize-data.txt · Last modified: 2020/10/26 17:40 (external edit)
Back to top
Contact us