synchro_import.php
DRAFT!
Purpose
This service is used for data synchronization.
Invoke this service to import data into the data synchronization table and optionally trigger the synchronization as well.
In case you have chosen to
import the data directly into the synchronization table, then you
just need to trigger the synchronization. Use the service synchro_exec.php.
Arguments
Argument | Description | Defaut value |
---|---|---|
param_file | Parameters file - see Parameters file | - |
auth_user | User login - CLI mode only | - |
auth_pwd | User password - CLI mode only | - |
data_source_id | Synchro data source id | Mandatory! |
csvdata | In HTTP mode, this argument must contain the data | n/a |
csvfile | In CLI mode, this arguments must point to the file containing the input data | n/a |
synchronize | If set to 1, then the synchronization will be executed right after the data load | 1 |
charset | Character set encoding of the CSV data: UTF-8, ISO-8859-1, WINDOWS-1251, WINDOWS-1252, ISO-8859-15 | UTF-8 |
date_format | Input date format (used both for dates and datetimes) - Examples: Y-m-d, d/m/Y (Europe) Expected format - no transformation is applied if the argument is omitted | |
separator | column separator in CSV data (1 char, or tab) | ; |
qualifier | test qualifier in CSV data | “ |
output | retcode to return the count of lines
in error, summary to return a concise report,
details to get a detailed report (each line
listed) |
summary |
max_chunk_size | CLI mode only, Limit on the count of records that can be loaded at once while performing the synchronization | 0 |
0 | ||
comment | Comment to be added into the change log | |
no_stop_on_import_error | Do not stop the import in case of SQL import error. By default the import will stop at the first error (and rollback all changes). If this flag is set to 1 the import will continue anyway | 0 |
Some issues have been reported
with the simulate mode. This feature remains
experimental.
Excel often generates CSV
using this charset=ISO-8859-1
Example
php -q /var/www/html/itop/synchro/synchro_import.php --auth_user="john" --auth_pwd="trust,no1" --data_source_id=2 --csvfile="mydata.csv"
Data Source
Create a Data Source for loading with:
-
target class
= Person -
reconciliation ?
= email -
Update
= Yes for name,first_name,org_id,email -
org_id (reconcialiation key)
= Full name
Source file
Loading those data on an iTop with the sample data
primary_key;name;org_id;first_name;email 1;nom1;Demo;person1;person1.nom1@gmail.com 2;nom2;Demo;person2;person2.nom2@gmail.com 3;nom3;Demo;person3;person3.nom3@gmail.com 4;nom4;Demo;person4;person4.nom4@gmail.com 5;nom5;Demo;person5;person5.nom5@gmail.com 6;nom6;Demo;person6;person6.nom6@gmail.com 7;nom7;Demo;person7;person7.nom7@gmail.com 8;nom8;Demo;person8;person8.nom8@gmail.com 9;nom9;Demo;person9;person9.nom9@gmail.com 10;nom10;Demo;person10;person10.nom10@gmail.com
Pay attention to the fact that the
colum header cannot be an External Field, it must be the
corresponding external key (here
org_id
), even if the
values themselves correspond to another field of the remote object
(here org_id→name
), that mapping is part of the
'Synchro Data Source' definition ( Reconcialiation key
in Attributes tab)There must be a
primary_key column.
Normal execution
#------------------------------------------------------------ # Import phase summary #------------------------------------------------------------ #Data Source: 2 #Synchronize: 1 #Class: Person #Separator: ; #Qualifier: " #Charset Encoding:UTF-8 #Date and time format: 'Y-m-d H:i:s' #Date only format: 'Y-m-d' #Data Size: 484 #Data Lines: 10 #Columns: primary_key, name, org_id, first_name, email #Output format: summary #Simulate: 0 #Change tracking comment: #Issues (before synchro): 0 #Created (before synchro): 0 #Updated (before synchro): 10 #------------------------------------------------------------ # Synchronization phase summary #------------------------------------------------------------ #Replicas: 10 #Replicas touched since last synchro: 10 #Objects deleted: 0 #Objects deletion errors: 0 #Objects obsoleted: 0 #Objects obsolescence errors: 0 #Objects created: 0 (0 warnings) #Objects creation errors: 0 #Objects updated: 2 (0 warnings) #Objects update errors: 2 #Objects reconciled (updated): 0 (0 warnings) #Objects reconciled (unchanged): 0 (0 warnings) #Objects reconciliation errors: 0 #Replica disappeared, no action taken: 0
2nd load of the modified source with error
Modified source file with some errors
primary_key;name;org_id;first_name;email 1;nom1;InvalidOrgName;person1;person1.nom1@gmail.com 2;nom2;2;changed-person2;person2.nom2@gmail.com 3;nom3;Demo;changed-person3;person3.nom3@gmail.com 4;nom4;IT Department;person4;person4.nom4@gmail.com 5;nom5;Demo;person5;person5.nom5@gmail.com 6;nom6;Demo;person6;person6.nom6@gmail.com
-
Only 6 rows are provided instead of 10
-
The 2 first rows contain invalid
Organization
full name
generating:#Objects errors in update: 2
-
The rows 3 & 4 have been modified with allowed/valid values generating:
#Objects updated: 2
Output with some Errors
#------------------------------------------------------------ # Synchronization phase summary #------------------------------------------------------------ #Replicas: 10 #Replicas touched since last synchro: 6 #Objects deleted: 0 #Objects deletion errors: 0 #Objects obsoleted: 0 #Objects obsolescence errors: 0 #Objects created: 0 (0 warnings) #Objects creation errors: 0 #Objects updated: 2 (0 warnings) #Objects update errors: 2 #Objects reconciled (updated): 0 (0 warnings) #Objects reconciled (unchanged): 0 (0 warnings) #Objects reconciliation errors: 0 #Replica disappeared, no action taken: 0
As visible in iTop
<image> </image>
Allowed users
Administrators will always be allowed to execute this page.
Non administrators will be
allowed if they are marked as being owner of the data source.
History
Available since 1.1
2_6_0/advancedtopics/synchro_import.php.txt · Last
modified: 2020/02/05 11:42 (external edit)