Tutorial : create a custom REST/JSON collector
Introduction
The core folder provides an abstract class
JsonCollector which can serve as the basis for quickly
creating collectors that retrieve their data via a REST/JSON
interface.
To create such a collector you need to:
-
Create a class derived from
JsonCollector -
Create the json definition file for the data synchro source
-
Add a configuration parameter (in
params.local.xml) to define REST/JSON interface to query -
Register your collector in
collectors/main.php
Collector class
- /collectors/sitescollector.class.inc.php
-
<?php class SiteCollector extends JsonCollector { }
Synchro data source definition file
- /collectors/ITopPersonJsonCollector.json
-
{ "name": "Persons from JSON", "description": "", "status": "implementation", "user_id": "$synchro_user$", "notify_contact_id": "$contact_to_notify$", "scope_class": "Person", "database_table_name": "synchro_data_person_1", "scope_restriction": "", "full_load_periodicity": "0", "reconciliation_policy": "use_attributes", "action_on_zero": "create", "action_on_one": "update", "action_on_multiple": "error", "delete_policy": "ignore", "delete_policy_update": "", "delete_policy_retention": "0", "attribute_list": [ { "attcode": "cis_list", "update": "0", "reconcile": "0", "update_policy": "master_locked", "row_separator": "|", "attribute_separator": ";", "value_separator": ":", "attribute_qualifier": "'", "finalclass": "SynchroAttLinkSet", "friendlyname": "cis_list" }, { "attcode": "civility", "update": "0", "reconcile": "0", "update_policy": "master_locked", "finalclass": "SynchroAttribute", "friendlyname": "civility" }, { "attcode": "email", "update": "1", "reconcile": "1", "update_policy": "master_locked", "finalclass": "SynchroAttribute", "friendlyname": "email" }, { "attcode": "employee_number", "update": "0", "reconcile": "0", "update_policy": "master_locked", "finalclass": "SynchroAttribute", "friendlyname": "employee_number" }, { "attcode": "first_name", "update": "1", "reconcile": "1", "update_policy": "master_locked", "finalclass": "SynchroAttribute", "friendlyname": "first_name" }, { "attcode": "function", "update": "1", "reconcile": "0", "update_policy": "master_locked", "finalclass": "SynchroAttribute", "friendlyname": "function" }, { "attcode": "location_id", "update": "0", "reconcile": "0", "update_policy": "master_locked", "reconciliation_attcode": "", "finalclass": "SynchroAttExtKey", "friendlyname": "location_id" }, { "attcode": "manager_id", "update": "0", "reconcile": "0", "update_policy": "master_locked", "reconciliation_attcode": "", "finalclass": "SynchroAttExtKey", "friendlyname": "manager_id" }, { "attcode": "mobile_phone", "update": "0", "reconcile": "0", "update_policy": "master_locked", "finalclass": "SynchroAttribute", "friendlyname": "mobile_phone" }, { "attcode": "name", "update": "1", "reconcile": "1", "update_policy": "master_locked", "finalclass": "SynchroAttribute", "friendlyname": "name" }, { "attcode": "notify", "update": "0", "reconcile": "0", "update_policy": "master_locked", "finalclass": "SynchroAttribute", "friendlyname": "notify" }, { "attcode": "org_id", "update": "1", "reconcile": "1", "update_policy": "master_locked", "reconciliation_attcode": "", "finalclass": "SynchroAttExtKey", "friendlyname": "org_id" }, { "attcode": "phone", "update": "1", "reconcile": "0", "update_policy": "master_locked", "finalclass": "SynchroAttribute", "friendlyname": "phone" }, { "attcode": "picture", "update": "0", "reconcile": "0", "update_policy": "master_locked", "finalclass": "SynchroAttribute", "friendlyname": "picture" }, { "attcode": "projects_list", "update": "0", "reconcile": "0", "update_policy": "master_locked", "row_separator": "|", "attribute_separator": ";", "value_separator": ":", "attribute_qualifier": "'", "finalclass": "SynchroAttLinkSet", "friendlyname": "projects_list" }, { "attcode": "status", "update": "0", "reconcile": "0", "update_policy": "master_locked", "finalclass": "SynchroAttribute", "friendlyname": "status" }, { "attcode": "team_list", "update": "0", "reconcile": "0", "update_policy": "master_locked", "row_separator": "|", "attribute_separator": ";", "value_separator": ":", "attribute_qualifier": "'", "finalclass": "SynchroAttLinkSet", "friendlyname": "team_list" }, { "attcode": "vip", "update": "0", "reconcile": "0", "update_policy": "master_locked", "finalclass": "SynchroAttribute", "friendlyname": "vip" } ], "user_delete_policy": "nobody", "url_icon": "", "url_application": "", "notify_contact_id_obsolescence_flag": "" }
Configuration
Update the file /conf/params.local.xml to set the
needed parameters to connect to your target iTop instance:
- /conf/params.local.xml
-
<itop_url>https://????/</itop_url> <itop_login>????</itop_login> <itop_password>????</itop_password>
Now you have to write an XML file specifying the mapping logic
between you Blabla Sites data format and the iTop
Location objects. The exact syntax depends on the
format of your source data (OQL, CSV or JSON). Here we use JSON, so
we need to define:
| Tag | Usage |
| <json_data> | Is used to get from the Blabla
application the Site objects and for each a given
set of fields. This part depends entirely on REST/JSON interface offered by the Blabla application |
| <path> | Specify how to retrieve the fields code and value within the Blabla returned json response. The supported syntax is limited to tag name separated by /. A tag can be a real label in the json returned file or just a star * for level without label, such as a table. |
| <fields> | specifies for each iTop field, the corresponding
Blabla field.Within that mapping there is a special mandatory tag <primary_key> which must be mapped to a Blabla field which identify uniquely a
Site in the Blabla application. |
| <defaults> | allow to specify itop fields for which we want to
set fixed values. It can be used in combination with a mapping, for
the case where a particular Blabla Site has no value
for this field. |
- /conf/params.local.xml
-
<parameters> <!-- Class name of the collector in lowercase --> <sitecollector> <jsonurl>http://blabla.demo.com/webservices/rest.php</jsonurl> <jsonpost> <auth_user>blabla-rest-user</auth_user> <auth_pwd>blabla-rest-user-password</auth_pwd> <json_data> <!-- This part depends entirely on REST/JSON interface offered by the Blabla application --> </json_data> <version>1.3</version> </jsonpost> <!-- "path" specify how to retrieve the fields with the JSON response --> <path>objects/*/fields</path> <fields> <primary_key>blabla_id</primary_key> <!-- tag primary_key is mandatory --> <name>name</name> <country>land</country> <status>step</status> </fields> <defaults> <!-- This part allow to specify default values for iTop Location field --> <org_id>Demo</org_id> <status>active</status> </defaults> </sitecollector> </parameters>
Registering collector
Finally, in collectors/main.php add the following
lines:
- /collectors/main.php
-
<?php require_once(APPROOT.'collectors/sitecollector.class.inc.php'); Orchestrator::AddCollector(1, 'sitecollector');
Usage
To launch the data collection and synchronization with iTop, run the following command (from the root directory where the application is installed):
php exec.php
extensions/sample-collector-json.txt ยท Last modified:
2023/03/30 14:21 (external edit)