Data collector for Graph
- name:
- Data collector for Graph
- description:
- Synchronize a set of Microsoft Graph objects into iTop
- version:
- 1.0.0
- release:
- 2024-04-10
- itop-version-min:
- 2.7.9
- code:
- combodo-graph-data-collector
- state:
- stable
- php-version-max:
- 8.1
This collector enables administrators to automatically feed iTop with relevant and accurate Graph cloud computing information.
Features
The Data collector for Graph is a stand-alone PHP software that connects to Microsoft Graph, retrieves objects from that environment and synchronizes them with iTop's CMDB.
-
Connection to Graph is made through Graph OAUTH2 authentication,
-
Retrieval of objects relies on Graph REST API that extracts data in JSON format,
-
Synchronization follows iTop's built-in Data Synchronization mechanism.
The extension focuses on the following Graph objects:
-
Groups
-
Users
-
Members
They will be mapped to the following iTop classes:
-
Team
-
Person and User
-
link Person / Team
Data collection can be enhanced by adding further Graph classes to this list. Please, refer to the Q&A chapter below.
Revision History
Version | Release Date | Comments |
---|---|---|
1.0.0 | 2024-04-10 | First version |
Limitations
The collector only synchronizes a limited subset of objects from Graph (see above).
-
Neither the Organizations nor the Locations are synchronized.
-
When created by the collector, new users are given the default profile define in the configuration file (portal user by default). No information is retrieved from Graph on profiles.
At this stage, no specific filter is applied during the collect of the groups and therefore, all groups are collected, by default.
Requirements
Usage of the collector requires you to comply with a few points:
-
The Graph objects synchronized by the collector must exist in iTop as CMDB CIs. This is the case with iTop's default data model.
-
In order to browse Graph's computing environment, you'll need an Graph account (see service principal) that allows you to read Graph's resources.
-
From a system standpoint, you'll need to comply with the requirements expressed in the Data collector Base documentation.
Installation
Simply expand the content of the zip archive into a folder on the machine where the collector will be run.
Configuration
The configuration of the application is built by concatenating 4 files:
-
collectors/params.distrib.xml that holds the entries that are specific to the Data collector for Graph. It should not be modified.
-
collectors/extensions/params.distrib.xml that holds entries that have been created and added by the customer to answer its specific needs. This file is optional.
-
conf/params.distrib.xml that is provided by the collector framework, Data collector Base. It should not be modified.
-
conf/params.local.xml where the collector can be adapted to the specific customer needs.
The collectors/params.distrib.xml configuration file holds parameters that must (for some) or can (for others) be changed when configuring the collector, which is done through the conf/params.local.xml file.
<?xml version="1.0" encoding="UTF-8"?><!-- Default values for parameters. Do NOT alter this file, use params.local.xml instead --> <parameters> <!-- Connection parameters--> <!-- iTop Parameters --> <itop_url>http://myitop.com</itop_url> <itop_login>admin</itop_login> <itop_password>admin</itop_password> <!-- Microsoft links --> <microsoft_login_url>https://login.microsoftonline.com/</microsoft_login_url> <microsoft_auth_mode>/oauth2/token</microsoft_auth_mode> <microsoft_resource>https://management.azure.com/</microsoft_resource> <!-- Client's credentials --> <ms_clientid></ms_clientid> <ms_clientsecret></ms_clientsecret> <ms_tenantid></ms_tenantid> <!-- *** Azure Class Parameters *** --> ... List of Graph classes with their parameters ... <!-- Class collection sequence --> ... List of classes to collect with the rank in the collection process ... <!-- Synchro data source parameters --> <contact_to_notify></contact_to_notify> <synchro_user>admin</synchro_user> <name_prefix>Graph</name_prefix> <name_postfix>Discovery</name_postfix> <json_placeholders> <ms_tenantid_short></ms_tenantid_short> <itoplnkpersontoteam_synchro_name>$name_prefix$ lnkPersonToTeam $name_postfix$</itoplnkpersontoteam_synchro_name> ... Placeholders for other classes ... <synchro_status>production</synchro_status> <full_load_interval>604800</full_load_interval><!-- 7 days (in seconds): 7*24*60*60 --> <users_target_class>UserExternal</users_target_class> <!-- Set update_persons_manager to 1 if you want to update persons' manager or to 0 otherwise --> <update_persons_manager>1</update_persons_manager> <!-- Following policies may be: write_if_empty, master_locked or master_unlocked --> <profile_list_update_policy>write_if_empty</profile_list_update_policy> </json_placeholders> <!-- mapping tables --> <user_language_mapping type="array"> <pattern>//EN US</pattern> <pattern>/en/EN US</pattern> <pattern>/en-US/EN US</pattern> <pattern>/.*/%1$s</pattern> </user_language_mapping> </parameters>
The collectors/extensions/params.distrib.xml configuration file holds parameters to collect additional classes required by a customer.
<?xml version="1.0" encoding="UTF-8"?><!-- Default values for parameters. Do NOT alter this file, use params.local.xml instead --> <parameters> <!-- Graph Class Parameters --> ... List of additional Graph classes with their parameters ... <!-- Class collection sequence --> ... List of additional classes to collect with the rank in the collection process ... <!-- Synchronization parameters --> <json_placeholders> <graphmyclass_synchro_name>$name_prefix$ My Class $name_postfix$</graphmyclass_synchro_name> ... Placeholders for other additional classes ... </json_placeholders> </parameters>
Connection parameters
This set of parameters is required to connect to iTop application or to Microsoft Azure environment. Some of them must or may be adjusted to meet customers' own environment.
Parameter | Meaning | Sample value |
---|---|---|
itop_url | URL to the iTop Application | https://localhost/myitop |
itop_login | Login (user account) for connecting to iTop. Must have admin rights with rest profile for executing the data synchro | admin |
itop_password | Password for the iTop account | admin_pwd |
microsoft_login_url | URL to be used by the collector to log into Graph and start the authentication process | https://login.microsoftonline.com/ |
microsoft_auth_mode | Specifies the authentication process - OAUTH2 for the Azure collector | /oauth2/token |
microsoft_resource | URL to the Azure REST API | https://management.azure.com/ |
ms_tenantid | Tenant ID is the unique identifier of your Graph instance. A Tenant ID allows you to register and manage your apps. | 2d74zb091-b7fe-4e14-868b-4e9a3f887a60 |
ms_clientid | Client ID is the unique identifier of your application created in Graph. | 7ed0fdef-abcd-4fd0-8364-da5fde498765 |
ms_clientsecret | Secret key associated to the Client ID | fJ4eQ~rG75DPd4aadrBodZvKxFtxOhnX2XrO |
Synchro data source parameters
These are dedicated to the synchronization data sources that the collector creates. Some of them must or may be adjusted to meet customers' own environment.
Parameter | Meaning | Sample value |
---|---|---|
contact_to_notify | The email address of an existing contact in iTop to be notified of the results of the synchronization. | john.doe@demo.com |
synchro_user | If the user account used for running this synchronization is not an Administrator, then its login must be specified here, since iTop allows only the administrators and the specified user to run the synchronization. | |
name_prefix | String used to prefix the name of all Graph synchro data sources | Azure |
name_postfix | String used postfix the name of all Graph synchro data sources | Discovery |
$name_prefix$ <Azure_Class> $name_postfix$ | Name of the synchro data source for the given Graph Class | Graph Persons Discovery |
ms_tenantid_short | Sub string of the Tenant ID that allows you to
differentiate different synchro data sources working on different
Tenants (1 collector should be used per Tenant - see above). That string is appended to the name of the synchro data source, in iTop. |
4e9a3f887a60 |
full_load_interval | The delay (expressed in seconds) between two complete imports of the data. The objects which have not been detected by the collector during a timespan longer than this interval will be considered as obsolete and marked as such in iTop. Adjust this value depending on the scheduling recurrence. | 604800 |
Mapping tables
This section groups the mapping tables used by the collector.
Parameter | Meaning |
---|---|
user_language_mapping | Mapping table for languages used |
Graph Class Parameters
Next to the core parameters described here above, the collectors/params.distrib.xml file provides the list of all Graph classes that need to be discovered together with their subset of parameters that should be synchronized within iTop. Since the retrieval of Graph data is made in JSON format, this list must be aligned with the requirements expected by the standard JSON collector defined in the data collector base.
Looking, for instance, at the Person class, we have:
<itoppersongraphcollector> <ms_class>User</ms_class> <api_version>1.0</api_version> <jsonfile>data/GraphPerson.json</jsonfile> <path>value</path> <fields> <primary_key>id</primary_key> <email>mail</email> <first_name>givenName</first_name> <function>jobTitle</function> <location_id>officeLocation</location_id> <manager_id>manager_id</manager_id> <mobile_phone>mobilePhone</mobile_phone> <name>surname</name> <org_id>org_id</org_id> <phone>phone</phone> <status>status</status> </fields> <defaults> <status>active</status> <org_id>Demo</org_id> </defaults> </itoppersongraphcollector>
Parameter | Meaning | Sample value |
---|---|---|
ms_class | Name of the Graph class | User |
api_version | Version of the Microsoft REST API used to retrieve class data | 2021-02-01 |
jsonfile | Define the relative path to the JSON file where Graph data have been extracted | data/GraphPerson.json |
path | Path to find the data to synchronize in JSON file | value |
fields | List of objects' fields to be considered by the synchro engine | See above |
defaults | List of default values to be used, if required | See above |
Class collection sequence
This section defines the list of classes that will be collected and in which order. It enables as well the possibility to deactivate the collection of a class.
<collectors_launch_sequence type="array"> <!-- iTop Teams --> <collector> <name>iTopTeamGraphCollector</name> <enable>yes</enable> <rank>1</rank> </collector> <!-- iTop Persons --> <collector> <name>iTopPersonGraphCollector</name> <enable>yes</enable> <rank>2</rank> </collector> <!-- iTop lnkPersonToTeamGraphCollector --> <collector> <name>iToplnkPersonToTeamGraphCollector</name> <enable>yes</enable> <rank>3</rank> </collector> <!-- iTop Users --> <collector> <name>iTopUserGraphCollector</name> <enable>yes</enable> <rank>4</rank> </collector> </collectors_launch_sequence>
Parameter | Meaning | Sample value |
---|---|---|
name | Name of the Graph class collector | iTopPersonGraphCollector |
enable | Enable or disable its collect | yes / no |
rank | Relative rank in the collection | 30 |
Usage
Launch of the Graph collector will be driven by the command and parameters defined in the usage section of iTop Data collector base. Once launched, first action of the collector will be to build its collection plan, based on the list of classes that have been enabled in the configuration file. Then,
-
Configuration files will be consolidated,
-
Synchronisation data sources will be created or updated if required,
-
Authentication to Graph environment will be made (see below),
-
Collection of Graph classes will be made by invoking Graph REST API and extracted data will be stored under the local “data” directory, in JSON format,
-
Synchronisation will run and Graph objects will be pushed to iTop.
-
A Person is attached to a User through the mail attribute conatined in the Graph User class,
-
Reconciliation of the lnkPersonToTeam class is made ont the Person's email and Group's name.
Questions & Answers
Question: how does the authentication mechanism work
?
Answer: the collector relies on OAUTH2 authentication requested by
Graph to retrieve data through its REST API. This is the exact
same mechanism as the one implemented within the Data
collector for Azure. Please, refer to the Q&A section of
that collector for further details.
Question: how can I add Graph classes to the collection
plan
Answer: Other classes than the ones listed by default in the
collector can be added to its collection plan. For that purpose, we
highly recommend to use the extensibility mechanism that is
provided by the Data
Collector base. You'll find the related documentation here.