Data Synchronization Reference
If you are not familiar with the concept, start by the Overview
Want to see an example
Want to dig inside to troubleshoot common issues
Data Source Definition
The Data Sources are defined using the menu Admin Tools/Synchronization Data Sources.
Each data source defines:
The target class of the objects to be synchronized
The list of attributes/fields of the objects to synchronize
How to search/reconcile the objects with the objects already existing in iTop
The rules of synchronizing/updating and possibly deleting objects in iTop
How the synchronized objects behave for the iTop end-users (which fields are read-only, are users allowed to delete the objects…)
An optional hyperlink and icon to refer iTop users to the corresponding object in the external application
This section of the Properties defines some general parameters for this Synchro Data Source
|Name||Mandatory||The name which refers to this data source||Servers synchro from SCCS|
|Description||Optional||An additional description, for information only|
|Status||Mandatory||implementation, production or obsolete: for filtering/classifying the data sources||production|
|Target Class||Mandatory||The class of objects to synchronize||Server|
|User||Mandatory||The user allowed to execute this synchro||admin|
|Contact to notify||Optional||The contact to notify (by email) of the results when the synchronization is executed and contains at least one error|
|Icon's hyperlink||Optional||The hyperlink (URL) to an image representing this data source. This icon is shown instead of the default icon in the tooltip appearing on the “Lock” symbol next to the title of the synchronized object in iTop|
|Application's hyperlink||Optional||The hyperlink (URL) to the remote application
from which the synchronized data comes from. The hyperlink can
contain placeholders such as
|Data table||Optional||The name of the MySQL table in which the replica information will be stored. If omitted, the system will generate a unique name for you.||synchro_data_servers_from_sccs|
Because it was required to be able through synchro_import.php to empty fields such as date, the format of date column was changed in release 2.3.4 from
Search & reconciliation
This section of the properties defines
How the information contained in the replica table is matched against the content of the iTop database
What is the behavior of this synchro data source concerning the creation of new iTop objects.
|Action on zero||What to do when the reconciliation did not find
any object in iTop.
|Action on one||What to do when the reconciliation found exactly
one matching object in iTop.
|Action on many||What to do when the reconciliation found several
matching objects in iTop.
This section defines how the deletion of the synchronized objects are handled:
by the end-users in iTop,
by the synchronization system itself.
|Users Allowed||Which end-users are allowed to delete synchronized
objects in iTop.
|Full load interval||The duration between two executions of the synchronization which are considered as having loaded (touched) all replicas. When a replica has not been touched for a duration greater than this value, it is considered as obsolete.|
|Delete Policy||What to do when a replica becomes obsolete.
|Update rules||A list of values separated by semi-colons, of the
|Retention duration||Only used when the
If your set a Full load interval of 7 days, and a Retention duration of 14 days, an object no more provided by the source will be deleted after 3 weeks.
The “Attributes” tab defines:
How to determine whether a new object must be created in iTop or an existing object must be associated with the replica (if the reconciliation rule is “Use Attributes” in the properties)
Which attribute(s) of the associated object must be updated, and how.
|Attribute||It automatically lists each attributes of the DataSynchro “Target class”, with its label and code|
|Reconciliation||Specify if the value in this column should be used to retrieve an existing iTop object. Used for new replica only|
|Update||Specify if that iTop object attribute should be updated by the Datasynchro|
|Update Policy||Specify if the attribute can be edited in iTop or not|
|Reconciliation Key||Required for ExternalKey attributes only. Specify one attribute of the remote object used to retrieve it. It can be the id or a field which would be a unique identifier for the remote class|
The Reconciliation column
When a new record appears in the
table, the synchronization mechanism will try to associate this
record with an existing iTop object. If one or more Attribute is
marked as “Yes” in the “Reconciliation ?” column, these
columns/attributes will be used to search for an iTop object that
matches all these values. If one object is found, it will become
associated with this record. If no object is found, a new iTop
object will be created.
The Update column
The Update column defines if the attribute must be updated or not.
Checked = Yes: the attribute will be set/updated
Unchecked = No: the external source information for this attribute will be ignored, the existing value on the iTop object will be kept as is.
empty stringare not equivalent
Null ⇒ The existing iTop field value is kept as is,
Empty string ⇒ Reset iTop object field
You must use your own feeding mechanism for loading
Null values in
synchro_data_xxx table, as
The Update Policy column
The column Update Policy defines the behavior of the updated attribute when editing the iTop object. The possible values are:
Locked: the attribute will be completely driven by the data synchronization and will appear as read-only (with a small lock icon next to it) in the iTop user interface,
Unlocked: the attribute will be updated by the synchronization (whenever the source data changes) but end-users can still modify the attribute in parallel inside iTop,
Initialize if empty: the synchronization will only set a value of this attribute if the value is empty in iTop. The attribute remains modifiable inside iTop.
The Reconcialiation key column
Finally, the Reconciliation key column defines
how to process attributes which are references to another iTop
object (i.e. Foreign keys). The supplied data can contain either
the actual value of the foreign key (
id) or another
attribute such as the
name of the object; which must
uniquely identify the object.
organization). If such a need arises, then you must use an ETL or a script to perform the desired reconciliation before providing the
The Status tab displays information about the execution of the synchronization. If the synchronization has never been run, the tab displays the following message:
This synchro was never run. No log yet.
Otherwise the tab displays a schema similar to the one below:
On the left of the screen, each execution of the synchro is displayed in an historical list, showing the overall status, the start date and time of the execution.
Click on an element of this list to show the corresponding status in detail, displayed in the two columns on the right.
The column at the left displays the status for the records in
Newrecords are records which were added in the table since the previous execution of the synchro,
Existingrecords are records which were existing before and have been loaded again since the previous execution of the synchro,
Disappearedrecords are records which were not loaded since more than the
Ignoredrecords corresponds to objects no longer existing in iTop.
The column at the right displays the consequences of the synchro, in terms of modifications on the associated objects in iTop.
Newrecord, the consequences can be either:
The creation of the new object in iTop (
The update of an already existing object in iTop (
The simple association with an object in iTop, which is already up-to-date (
Errorwhen trying to either create or update an object in iTop (most probably because of inconsistent data).
Existingrecord, the consequences can be either:
The update of the associated object (
No change at all (
Or an error when trying to update the associated object (
Disappearedrecord, the consequences (depending on the “Deletion rules”) can be either:
The associated object in iTop has been deleted (
The associated object in iTop has been updated (
The associated object in iTop was left unchanged (
Or an error occurred when trying to update or delete the associated object in iTop (
In iTop, the Replicas are used to store the status
information and the relation between the raw data loaded in the
synchro_data_xxx table and the associated iTop
The Replicas for a given data synchronization source can be displayed by clicking on the links at the top of the status tab. A Replica is displayed as shown below:
The content of the replicas is useful to understand the behavior of the data synchronization mechanism (and to get more information about the errors), but its data are maintained by the synchronization and should never be modified manually.
|Fields of a Replica||Purpose|
|Synchro Data Source||Synchro Data Source to which this Replica belongs|
|Destination object (ID)||iTop object id corresponding to that replica. Can be empty when the replica is new and could not be synchronized due to an error|
|Destination type||The Target class of the Synchro Data Source, which is also the class of the Destination object|
|Status||Status of the Replica: Modified, New, Obsolete, Orphan, Synchronized|
|Last seen||When that replica was seen for the last time in
the source data (same
|Object created?||Was the iTop object created by the Data Synchro?|
|Warnings||Warning message when some non blocking error occured|
|Creation date||When the iTop object was created by the Synchro Data Data|
|Last Modified Date||When that replica was modified by the source for the last time|
|New||Newly received replica. Temporary status before syncho_exec.php occurs to create the iTop object|
|Modified||Modified replica. Temporary status before syncho_exec.php occurs to update the iTop object|
|Synchronized||Replica and iTop object are aligned|
|Obsolete||Data is no more in the Source, iTop object was not deleted, otherwise the replica would be as well|
|Orphan||This should not happen. In this situation the synchronized iTop object has disappeared without deleting the replica at the same time. Such replica can safely be manually deleted|
Using MySQL statements to populate the synchro_data_xxxx table
If you populate the table
SQL commands (either with an ETL or a home made script), you
must take care of never creating duplicate
records. Remember that each record in the
synchro_data_xxx table corresponds to exactly one
object in iTop.
In order to avoid such duplicates, you must
first query for an existing record before inserting a new record
into the table. To speedup this query you can either construct a
value that uniquely identifies the “source” object and store this
value in the “primary_key” column of the table (which has a unique
index), or use any combination of the record's column and alter the
definition of the
synchro_data_xxxx table to add your
INSERT … ON DUPLICATE KEY UPDATEto insert records in the
synchro_data_xxxtable. This is not supported by the internal iTop mechanism which relies on the “BEFORE INSERT” trigger.
synchro_data_xxxxtable, do not use the MySQL statement
TRUNCATE TABLE, since this statement will actually drop and reconstruct the table… but not its triggers! Instead use
DELETE FROM synchro_data_xxxx.
Then trigger the execution of the data synchronization between the iTop objects and the replicas of the temporary table by executing the page “synchro/synchro_exec.php” either in command line mode, or by invocating the web page (using wget for example).
Using synchro_import to populate the synchro_data_xxxx table
synchronizeparameter of synchro_import.php trigger synchro_exec.php by default
How to specify a list of related objects (link set)
Some classes have attributes called Link Sets. Example: UserLocal.profile_list
You can synchronize those lists by writing all the information in the corresponding column:
Here is an example specifying that the login will have one profile: 'Portal User'
INSERT login, profile_list INTO data_synchro_userlocal_1 VALUES ('johndoe', 'profileid->name:Portal User;reason:Customer')...;
You can also specify several links (several profiles).
More information in Import a LinkedSet