:: Version 2.6.0 ::

synchro_import.php

DRAFT! FIXME

Purpose

This service is used for data synchronization.

Invoke this service to import data into the data synchronization table and optionally trigger the synchronization as well.

In case you have chosen to import the data directly into the synchronization table, then you just need to trigger the synchronization. Use the service synchro_exec.php.

Arguments

Argument Description Defaut value
param_file Parameters file - see Parameters file -
auth_user User login - CLI mode only -
auth_pwd User password - CLI mode only -
data_source_id Synchro data source id Mandatory!
csvdata In HTTP mode, this argument must contain the data n/a
csvfile In CLI mode, this arguments must point to the file containing the input data n/a
synchronize If set to 1, then the synchronization will be executed right after the data load 1
charset Character set encoding of the CSV data: UTF-8, ISO-8859-1, WINDOWS-1251, WINDOWS-1252, ISO-8859-15 UTF-8
date_format Input date format (used both for dates and datetimes) - Examples: Y-m-d, d/m/Y (Europe) Expected format - no transformation is applied if the argument is omitted
separator column separator in CSV data (1 char, or tab) ;
qualifier test qualifier in CSV data
output retcode to return the count of lines in error, summary to return a concise report, details to get a detailed report (each line listed) summary
max_chunk_size CLI mode only, Limit on the count of records that can be loaded at once while performing the synchronization 0
simulate If set to 1, then the load will not be executed, but the expected report will be produced 0
comment Comment to be added into the change log
no_stop_on_import_error Do not stop the import in case of SQL import error. By default the import will stop at the first error (and rollback all changes). If this flag is set to 1 the import will continue anyway 0
Some issues have been reported with the simulate mode. This feature remains experimental.
Excel often generates CSV using this charset=ISO-8859-1

Example

php -q /var/www/html/itop/synchro/synchro_import.php --auth_user="john" --auth_pwd="trust,no1" 
--data_source_id=2 --csvfile="mydata.csv"

Data Source

Create a Data Source for loading with:

  • target class = Person
  • reconciliation ? = email
  • Update = Yes for name,first_name,org_id,email
  • org_id (reconcialiation key) = Full name

Source file

Loading those data on an iTop with the sample data

primary_key;name;org_id;first_name;email
1;nom1;Demo;person1;person1.nom1@gmail.com
2;nom2;Demo;person2;person2.nom2@gmail.com
3;nom3;Demo;person3;person3.nom3@gmail.com
4;nom4;Demo;person4;person4.nom4@gmail.com
5;nom5;Demo;person5;person5.nom5@gmail.com
6;nom6;Demo;person6;person6.nom6@gmail.com
7;nom7;Demo;person7;person7.nom7@gmail.com
8;nom8;Demo;person8;person8.nom8@gmail.com
9;nom9;Demo;person9;person9.nom9@gmail.com
10;nom10;Demo;person10;person10.nom10@gmail.com
Pay attention to the fact that the colum header cannot be an External Field, it must be the corresponding external key (here org_id), even if the values themselves correspond to another field of the remote object (here org_id→name), that mapping is part of the 'Synchro Data Source' definition ( Reconcialiation key in Attributes tab)
There must be a primary_key column.

Normal execution

#------------------------------------------------------------
# Import phase summary
#------------------------------------------------------------
#Data Source: 2
#Synchronize: 1
#Class: Person
#Separator: ;
#Qualifier: "
#Charset Encoding:UTF-8
#Date and time format: 'Y-m-d H:i:s' 
#Date only format: 'Y-m-d' 
#Data Size: 484
#Data Lines: 10
#Columns: primary_key, name, org_id, first_name, email
#Output format: summary
#Simulate: 0
#Change tracking comment: 
#Issues (before synchro): 0
#Created (before synchro): 0
#Updated (before synchro): 10
#------------------------------------------------------------
# Synchronization phase summary
#------------------------------------------------------------
#Replicas: 10
#Replicas touched since last synchro: 10
#Objects deleted: 0
#Objects deletion errors: 0
#Objects obsoleted: 0
#Objects obsolescence errors: 0
#Objects created: 0 (0 warnings)
#Objects creation errors: 0
#Objects updated: 2 (0 warnings)
#Objects update errors: 2
#Objects reconciled (updated): 0 (0 warnings)
#Objects reconciled (unchanged): 0 (0 warnings)
#Objects reconciliation errors: 0
#Replica disappeared, no action taken: 0

2nd load of the modified source with error

Modified source file with some errors

primary_key;name;org_id;first_name;email
1;nom1;InvalidOrgName;person1;person1.nom1@gmail.com
2;nom2;2;changed-person2;person2.nom2@gmail.com
3;nom3;Demo;changed-person3;person3.nom3@gmail.com
4;nom4;IT Department;person4;person4.nom4@gmail.com
5;nom5;Demo;person5;person5.nom5@gmail.com
6;nom6;Demo;person6;person6.nom6@gmail.com
  • Only 6 rows are provided instead of 10
  • The 2 first rows contain invalid Organization full name generating: #Objects errors in update: 2
  • The rows 3 & 4 have been modified with allowed/valid values generating: #Objects updated: 2

Output with some Errors

#------------------------------------------------------------
# Synchronization phase summary
#------------------------------------------------------------
#Replicas: 10
#Replicas touched since last synchro: 6
#Objects deleted: 0
#Objects deletion errors: 0
#Objects obsoleted: 0
#Objects obsolescence errors: 0
#Objects created: 0 (0 warnings)
#Objects creation errors: 0
#Objects updated: 2 (0 warnings)
#Objects update errors: 2
#Objects reconciled (updated): 0 (0 warnings)
#Objects reconciled (unchanged): 0 (0 warnings)
#Objects reconciliation errors: 0
#Replica disappeared, no action taken: 0

As visible in iTop

<image> </image>

Allowed users

Administrators will always be allowed to execute this page.

Non administrators will be allowed if they are marked as being owner of the data source.

History

Available since 1.1

2_6_0/advancedtopics/synchro_import.php.txt · Last modified: 2020/02/05 11:42 (external edit)
Back to top
Contact us