Collector with multiple sources

When you have multiple sources for the same class of objects, all the mapping logic can be just the same. If it's not easy to merge the different sources before feeding the collector, it seems a bit useless to duplicate the collector and its execution. So to avoid this situation, what you can do is to use a prefix. It's value will be specified within aparam.local.xml file.

Let's take an example, assuming you have 3 data sources: Ganymede, Callisto, Io

They all run the same application and return data of the same type, except that they all collect a separated set of data. It could be 3 Vsphere applications for eg., but it does not have to be Vsphere.

So under your Collector directory, you need to create 3 sub-directories under data directory, like this:

• collectors
• conf
• core
• data
  • data-io
  • data-ganymede
  • data-callisto
• exec.php

Then: in the JSON file describing the DataSynchro(s), you will use a $prefix$ placeholer, in order to create 3 Datasynchros within iTop, each one having a unique name and a different data_table:

                "name": "Serveur from $prefix$",
                "description": "",
                "status": "production",
                "user_id": "$synchro_user$",
                "notify_contact_id": "$contact_to_notify$",
                "scope_class": "Server",
                "database_table_name": "synchro_data_server_$prefix$",
                "full_load_periodicity": "$full_load_interval$",

Create 3 files containing the local parameters, one for each of your data Source

• conf
  • params.distrib.xml
  • params.local.ganymede.xml
  • params.local.callisto.xml
  • params.local.io.xml

And in each param.xml file, you will specify:

• data_path which is the directory where the temporary CSV files will be stored, a different one for each Source.
• prefix which value will replace $prefix$ placeholder in the .JSON definition file, before that file is used to create/update the iTop DataSynchro object.

params.local.ganymede.xml

<?xml version="1.0" encoding="UTF-8"?>
  <parameters>
    ...
    <data_path>%APPROOT%/data/data-ganymede</data_path>
    <prefix>ganymede</prefix>
  </parameters>

params.local.callisto.xml

<?xml version="1.0" encoding="UTF-8"?>
  <parameters>
    ...
    <data_path>%APPROOT%/data/data-callisto</data_path>
    <prefix>callisto</prefix>
  </parameters>

In the above examples, I am not making any assumption on which collector you are using: SQL, CSV, JSON, LDAP, Vsphere or your own…, so it's an incomplete param file. Depending on the collectors, other tags could contain a different value on each file.

This is optional! because doing it means doing it each time you upgrade the Collector source code.

Usually the iTop url and credentials are specified in /conf/params.local.xml.
In this situation, in order to avoid specifying them multiple times in each params.local.xxx.xml, put them in the file /conf/params.distrib.xml

params.distrib.xml

<?xml version="1.0" encoding="UTF-8"?>
  <parameters>
    <itop_url>https://localhost/</itop_url>
    <itop_login>admin</itop_login>
    <itop_password>admin</itop_password>
    <contact_to_notify>itop-admin@demo.com</contact_to_notify>
    <synchro_user>cron-user</synchro_user>
    <json_placeholders type="hash">
      <prefix>$prefix$</prefix>
      <full_load_interval>604800</full_load_interval>
    </json_placeholders> 
    ...
  </parameters>

  php exec.php --config_file='conf/params.local.ganymede.xml'
  php exec.php --config_file='conf/params.local.callisto.xml'
  php exec.php --config_file='conf/params.local.io.xml'

Configure your cron to run each collectors on a regular basis.