Import data via CLI
iTop provides a generic web service to import data in itop based on a source csv file. This service can be used locally on the server via the command line interface or remotely via an HTTP post request.
Using this service, you can easily import data in iTop via a batch script that extract data from a third party tool and generate a csv formatted file. For instance a script data extract a list of contact from your Active Directory.
When it comes to import a real huge amount of data, the CLI mode will be a must because you will not reach the limitations of the web server (timeout and memory limits).
Command line interface
All the parameters of the command line are listed in the section “Arguments” here after. The following line is an example of the mandatory argument. All arguments starts with --
php /var/www/itop/webservices/import.php --auth_user=login --auth_pwd=password --class=Organization --csvfile=file --reconciliationkeys="list of keys"
The reconciliation keys are very important to identify uniquely the element to be imported in iTop. The reconciliation keys must be present in the data file. As opposed to the interactive CSV import, you are not limited to the list of reconciliation keys defined by default for the class.
Moreover, the external key will be uniquely identified by all its attributes specified in the reconciliation scheme. In our example, if you consider that Name/Country uniquely identifies the locations, the use both Location→Name and Location→Country in your reconciliation scheme.
|param_file||Parameters file - see Parameters file||-|
|auth_user||User login - CLI mode only||-|
|auth_pwd||User password - CLI mode only||-|
|class||Class of the loaded objects||Mandatory!|
|csvdata||CSV contents (HTTP mode only, data may be posted through this page argument)||Mandatory in HTTP mode|
|csvfile||Path and name of the source file||Mandatory in CLI mode|
|charset||Character set encoding of the CSV data: UTF-8, ISO-8859-1, WINDOWS-1251, WINDOWS-1252, ISO-8859-15||use
|date_format||Input date format (used both for dates and datetimes) - Examples: Y-m-d, d/m/Y (France)||<blank>|
|separator||Column separator in CSV data||, (coma)|
|qualifier||Qualifier in CSV data||“ (double quote)|
|output||retcode to return the count of lines in error, “summary” to return a concise report, “details” to get a detailed report (each line listed)||summary|
|reconciliationkeys||label or code (depends on
|simulate||If set to 1, then the load will not be executed, but the expected report will be produced||0|
|with_archive||If set to 1, then the import will retrieve archived objects as object to update and as valid external key||0|
|comment||Comment to be added into the change log||<blank>|
|no_localize||If set to 0, then column headers, values and reconciliationkeys are supposed to be localized in the language of the logged in user. Set to 1 to use internal attribute codes and values (enums)||0 = label not case sensitive|
The commande line will return a summary report (see below) summarizing the argument used for the import and the result of the import:
Number of elements created
Number of elements updated
Number of elements in error
In case you have specified the option –result=details you will get one line for each element imported with the status of the import (see example below)
#Output format: details #Class: Organization #Separator: ; #Qualifier: \" #Charset Encoding:UTF-8 #Data Size: 26 #Data Lines: 1 #Simulate: 0 #Columns: name, code #Reconciliation Keys: name #Change tracking comment: #Issues: 0 #Warnings: 0 #Created: 1 #Updated: 0 #Unchanged: 0 Line;Status;Object Class;Object Id;name (Name);code (Code) 0;created;Organization;3;\"WorldCompany\";\"WCY\"
ERROR: Missing argument 'class'
Such error is displayed when a mandatory argument is missing
#Unknown class: 'toto'
This error occurs if the class specified does not exists in iTop. Be careful the class name is case sensitive.
#Unknown reconciliationkeys: 'attribute'
This error occurs if at least one column in the reconciliation
keys list is not known.
The reason can be:
a code instead of a label was provided or vice-versa
the reconcialiation key is a External Field
the reconcialiation key is not present in the csv file
org_namecannot be used as a reconciliation key, use instead
#Unknown column: 'Last Nme'
This error occurs if a column is unknown in the csv file
no_localize=0 which is the default, you must use the labels in the CSV file headers and in
reconciliationkeys. For historical reason, in CLI mode, labels are not case sensitive so
reconciliationkeys=“name,owner organization→name” works as well as =“Name,Owner organization→Name” which would be the standard.
Administrators will always be allowed to execute this page.
Non administrators will be allowed if they are allowed to modify objects of the given class.
Importing Images and Files
Some classes contain an image or a file document as one of their field. For example, the Person class, contains a Picture field which holds the picture of this person. When editing a Person interactively, the user can upload an image from her computer to provide the picture of the person. However when performing a CSV import, it is not possible to “upload” such a file or to put its content inside the CSV file itself, since the CSV format does not support this.
In such a case the CSV file must provide - in the appropriate
CSV column - an URL to upload the file
from. This URL must be accessible from the
iTop server and return the expected image (or file document). If
the upload fails or if the format of the uploaded document does not
match the expected format for the field (for example if the
URL does not return
an image for the
picture field), the import/update
will fail for this field (and the whole line of the CSV will be
The URL must point directly to the image to upload (redirections are not supported)
The URL must be accessible from the iTop web server (beware some public websites may deny access to a PHP script - they check the HTTP User Agent string)
The URL must not require authentication, since the upload is done by the iTop server itself
URLs pointing to iTop itself (like the URLs provided by CSV/Excel export) are supported. But only if they point to the same iTop instance (in such a case the import will parse the provided URL and will read the document from the iTop database).
If you use an administrator account to perform the CSV import, you can specify a path to a file located on the iTop server itself (but this works only for administrators).
Use case: importing persons with their corresponding login into iTop
This use case describes how to import persons and their login in iTop. This is one way to load those information from an Active Directory. In that case you just have to write a script that create the two csv files person.csv and login.csv from your Active Directory.
We would like to create two contacts:
Claude Monet who belongs to organization Demo
Gustave Flaubert who belongs to organization IT Department
In this example the two organizations already exist in iTop
The class to import is UserLocal, and the data in csv file (login.csv) are:
Last Name,Status,Organization->Name,Email,Phone,Notification,Function,First Name "Monet","Active","Demo","email@example.com","","yes","","Claude" "Flaubert","Active","IT Department","firstname.lastname@example.org","","yes","","Gustave"
then run the following command line:
php webservices/import.php --auth_user=admin --auth_pwd=admin --csvfile="person.csv" --class="Person" --reconciliationkeys="Email"
Importing user accounts
We would like to create two user account:
support is configuration manager and change supervisor, related to the contact email@example.com
portal is a Porta user related to the contact firstname.lastname@example.org
In this example the login are linked to a contact via the email and we import the Profiles that are linked to the contact. Look at the particular syntax for importing the profiles. For more information about importing link sets refer to Import a linkset
The class to import is UserLocal:
Contact (person)->Email,Login,Language,Password,Profiles "email@example.com","support","EN US",1234,profileid->name:Configuration Manager|profileid->name:Change Supervisor "firstname.lastname@example.org","portal","EN US",1234,profileid->name:Portal user
then run the following command line:
php webservices/import.php --auth_user=login--auth_pwd=password --csvfile="login.csv" --class="UserLocal" --reconcialiationkeys="Login"
"name","org_id->name","country","city" "Bordeaux","Demo","France","Bordeaux" "Grenoble","Demo","France","Eybens" "Paris","Demo","France",""
then run the following command line:
php webservices/import.php --auth_user=login --auth_pwd=password --csvfile="location.csv" --charset="UTF-8" --no_localize=1 --class="Location" --reconcialiationkeys="name,org_id->name"
Limitations / Known issues
External Fields (such as such as
virtualhost_id->org_name) cannot be used as reconciliation keys.
org_namecan be replaced by
Limitation: the arrow operator (
->) cannot be used mutiple times (i.e.
virtualhost_id->org_id->nameis not supported).
0.9: first implementation of this feature
2.0: added the flag no_localize