Sidebar

Combodo

iTop Extensions

Data collector for OCS Inventory

name:
Data collector for OCS Inventory
description:
Data Collector for OCS Inventory NG
version:
2.0.0
release:
2023-03-28
itop-version-min:
2.7
download:
https://store.itophub.io/en_US/products/ocsng-data-collector
code:
ocsng-data-collector
state:
stable
alternate-name:
Data collector for OCS Inventory NG
diffusion:
iTop Hub,Combodo site
php-version-max:
PHP 8.1

This stand-alone application collects information from a single OCS Inventory NG server in order to populate the iTop CMDB with the collected Servers, PCs and Virtual machines. The collector uses the iTop data synchronization engine to synchronize the information between OCS Inventory and iTop.

An additional extension Data model for OCS Inventory is mandatory in iTop. It adds some fields needed for synchronization and displays the contents of the OCS inventory pages directly from iTop, for each synchronized object (Server, PC, Mobile phone or Virtual Machine):

Features

  • Automated inventory of Brand, Model, OS Family, OS Version, OS Licence, Physical Server, PC, Virtual Machine, Mobile Phone, Printer, Software from OCS Inventory server
  • Automated inventory of Network Interfaces
  • The collector can reside on any system with web access to iTop and mysql access to the OCS Inventory server
  • Automatic creation and update of the Synchronization Data Sources in iTop.
  • It is necessary to install the small extension Data model for OCS Inventory which adds some fields and an “OCS Inventory” link on synchronized PCs, Servers, Virtual Machines, Mobile Phones and software. This link opens a new tab with the OCS page of the device.
This collector makes use of iTop's built-in Data Synchronization mechanism. For more information about how the data synchronization works, refer to Data Synchronization Overview and relies on Data collector Base mechanism

Revision History

Date Version Description
2023-01-20 2.0.0 - Add new fields and classes to collector
2022-01-05 1.2.0 - New version with softwares collector
2021-01-05 1.0.7 - Fix compatibility with SSO set as default connection mode
- Configurable timestamp added in the logs
- New option for usage: –help
- The path to the configuration file can be specified via the option --config_file on the command line.
- The location where to store the collected data is now a parameter in the configuration file: data_path.
- Better checking of Data Source definitions to catch missing reconciliation keys
2019-11-13 1.0.6 - Reject invalid characters for database_table_name
- Added the specific class MySQLCollector which forces the DB connection to use UTF-8 characters
- Change all OCS SQLCollectors into MySQLCollectors
2019-10-29 1.0.5 Handles TeemIp :
- Automatically detects if TeemIp (as an iTop module or as a standalone application) is present.
- Optional synchronization of IPv4 addresses
- Optional synchronisation of IP interfaces
2018-09-04 1.0.4 First public version. No longer requires an alteration of the data model.
2018-06-26 1.0.3 New debug trace
2015-05-20 1.0.1 A few bug fixes: support of Windows file names (i.e. c:\Program Files\…). Conversion of SPEED from '100 Mb/s' to '100'. Forcing a NON case-sensitive comparison for 'Laptop' et al.
2015-02-16 1.0.0 First version

Limitations

  • The collector does not retrieve installed software information
  • The collector is not normalizing Model, OS Family and OS Version
  • The collector synchronizes only PCs and Servers which have a valid serial number in OCS
  • If IP collection is enabled, only IPv4 addresses are collected, not IPv6
  • There is no single sign-on between iTop and OCS, so you will need to log in to OCS when you display the OCS tab

Requirements

  • PHP Version 7.2.0
  • An access to the OCS Inventory NG mysql database
  • An access to the iTop web services (REST + synchro_import.php and synchro_exec.php). Note: starting with iTop 2.5.0, the user must have the profile REST Services User in iTop.
  • + Data collector Base requirements.
  • If you have ocs collector v1, you need go to database in order to rename some tables (XX replace id in corresponding datasource in iTop):
    • synchro_data_pc_XX to synchro_data_pc_ocsng
    • synchro_data_server_XX to synchro_data_server_ocsng
    • synchro_data_virtualmachine_XX to synchro_data_virtualmachine_ocsng

Later, if you have some synchronization problem due to table name, you need to rename the table which starts with synchro_data_ and ends with synchro data source id with name seen in synchro data source object.

By example, in this case :

you have to rename the table synchro_data_physicalinterface_171 to synchro_data_pcphysicalinterface_ocsng

Installation

Previous installation of the OCS Inventory integration extension is necessary in order to have all fields for synchronization
  • Expand the content of the zip archive “ocsng-data-collector” in a folder on the machine that will run the collector application. This machine must have an SQL access to the OCS NG database and a web access to the iTop server.
  • Create the file conf/params.local.xml according to your installation, providing the appropriate credentials to connect to OCS NG and iTop.
Do not install this extension in your webserver directories (like apache). otherwise your configuration files (with URLs, credentials) may be accessible from outside

By default this file should contain the values used to connect to the iTop server and the OCS NG server:

<parameters>
  <itop_url>http://localhost/</itop_url>
  <itop_login>admin</itop_login>
  <itop_password>admin</itop_password>
  <contact_to_notify>john.doe@demo.com</contact_to_notify>
  <synchro_user>admin</synchro_user>
  <sql_host>localhost</sql_host>
  <sql_database>ocsweb;charset=UTF8</sql_database>
  <sql_login>root</sql_login>
  <sql_password>root</sql_password>
</parameters>
Parameter Meaning Sample value
itop_url iTop Application URL https://localhost/
itop_login Login (user account) for connection to iTop. Must have administrator rights to run the data sync. admin
itop_password Password for the iTop account.
contact_to_notify The email address of an existing contact in iTop, who will be informed of the results of the synchronisation. john.doe@demo.com
synchro_user iTop user used for synchronization web service admin
sql_host The address to connect to the OCS server database default:localhost
sql_database The database to connect to. You can use ;charset to force the character set default ocsweb
sql_login Login to connect to the OCS database
sql_password Password to connect to the OCS database

Configuration

By default the data collection configuration is defined in the file collectors/params.distrib.xml. Do not modify this file! If you need to adapt the configuration, create a file named params.local.xml in the conf directory and copy/paste the needed definitions into it (the structure of both XMl files is the same). This configuration defines the SQL queries that must be executed on the OCS NG server to retrieve data and synchronise it with iTop.

   <!-- default values -->
    <default_org_id>Demo</default_org_id>
    <default_status>production</default_status>
    <default_active_status>active</default_active_status>
   <!--Filter type for synchronization ( categories or default filters ) -->
    <use_asset_categories>yes</use_asset_categories>
    <use_software_categories>yes</use_software_categories>    
   <!-- Class to collect -->
    <CategoryCollection>yes</CategoryCollection>
    <PCCollection>yes</PCCollection>
    <ServerCollection>yes</ServerCollection>
    <VMCollection>yes</VMCollection>
    <MobilePhoneCollection>yes</MobilePhoneCollection>
    <PrinterCollection>yes</PrinterCollection>
    <SoftwareCollection>yes</SoftwareCollection>

This parameters allow to activate or desactivate a group of collectors. If you want to disable one collector in a group, you can do so in collectors_launch_sequence (cf Data collector Base mechanism).

Principles

  • For each class of object to import into iTop (PC, Server, VM, Brand, Model, OSVersion OSFamily), a Synchro Data Source is created by the collector application.
  • Each collector is associated with an SQL query (via the parameter <name_of_the_collector>_query).
  • This query must return column where aliases (i.e. “name” of the columns) correspond to the expected fields for the corresponding Synchro Data Source.
  • These queries can be adapted to suit your needs (for example to better differentiate PCs and Servers or to change/set some default values for some columns)
  • Since the synchronization of PCs, Servers and Virtual Machines all rely on their own SQL query, it is important to make sure that the results of these 3 queries do no overlap. Otherwise the same “system” will be imported several times in itop (for example as both a PC and a Server).
  • You can use OCS categories in order to synchronize assets and softwares. In this case, you must create categories in OCS and create corresponding objects AssetCategories and SoftwareCategories in iTop. You can use synchronisation in order to help you to create this objects.
Global Parameter Meaning
default_org_id Define the default organization for the synchronized Cis (name of default organization)
default_status Define the default status for the synchronized Cis (implementation, obsolete, production, stock)
default_active_status Define the default status for the synchronized Software instances (active, inactive)
use_asset_categories Define if asset categories defined in iTop are used to get assets in OCS
use_software_categories Define if software categories defined in iTop are used to get softwares in OCS
CategoryCollection Define whether Categories collection is active (yes) or not (no). It is useful if use_asset_categories or use_software_categories is set to yes : Be consistent with these 2 parameters otherwise, you will collect and create Datasources for categories even if you don't use them.
PCCollection Define whether PC collection is active (yes) or not (no)
ServerCollection Define whether Server collection is active (yes) or not (no)
VMCollection Define whether Virtual machine collection is active (yes) or not (no)
MobilePhoneCollection Define whether Mobile phone collection is active (yes) or not (no)
PrinterCollection Define whether Printer collection is active (yes) or not (no)
The “Printer” extension must be installed in OCS
SoftwareCollection Define whether Software collection is active (yes) or not (no)

During the process, some of the following queries are executed depending on your parameters. If use_asset_categories = yes only queries with “with_categories” are executed. If use_asset_categories = no only corresponding queries without “with_categories” are executed.

These queries can be redefined in the file conf/params.local.xml in order to take into account your specific needs (For instance change status of created Server, PC, Virtual Machine, as well as default organisation).

The file params.distrib.xml contains the default values for the parameters. Both files (params.distrib.xml and params.local.xml) use exactly the same format. But params.distrib.xml is considered as the reference and should remain unmodified. Should you need to change the value of a parameter, copy and modify its definition in params.local.xml. The values in params.local.xml have precedence over the ones in params.distrib.xml

Configure Asset collection

Query Parameter Launch if Meaning
OCSAssetCategoryCollector_query CategoryCollection= yes SQL query to retrieve from OCS NG server the assets categories
Usefull only if use_asset_categories is set to yes
OCSBrandCollector_query SQL query to retrieve from OCS NG server the list of known Brands
OCSOSFamilyCollector_query SQL query to retrieve from OCS NG server the list of known OS Families
OCSOSVersionCollector_query SQL query to retrieve from OCS NG server the list of known OS Versions
OCSOSLicenceCollector_query SQL query to retrieve from OCS NG server the list of known Windows Licences
OCSServerModelCollector_query ServerCollection = yes
use_asset_categories = no
SQL query to retrieve from OCS NG server the list of known Physical server models.
Servers are hardware with different type than Mobile, Notebook and Laptop and with different manufacturer than WMWare
OCSServerModelCollector_with_categories_query ServerCollection = yes
use_asset_categories = yes
SQL query to retrieve from OCS NG server the list of known Physical server models.
Servers are found using OCS categories. iTop settings are required.
OCSServerCollector_query ServerCollection = yes
use_asset_categories = no
SQL query to retrieve from OCS NG server the list of physical servers
Servers are hardware with different type than Mobile, Notebook and Laptop and with different manufacturer than WMWare
OCSServerCollector_with_categories_query ServerCollection = yes
use_asset_categories = yes
SQL query to retrieve from OCS NG server the list of physical servers
Servers are found using OCS categories. iTop settings are required.
OCSServerPhysicalInterfaceCollector_query ServerCollection = yes
use_asset_categories = no
SQL query to retrieve from OCS NG server the list of physical network interfaces for servers
Servers are hardware with different type than Mobile, Notebook and Laptop and with different manufacturer than WMWare
OCSServerPhysicalInterfaceCollector_with_categories_query ServerCollection = yes
use_asset_categories = yes
SQL query to retrieve from OCS NG server the list of physical network interfaces for servers
Servers are found using OCS categories. iTop settings are required.
OCSPCModelCollector_query PCCollection = yes
use_asset_categories = no
SQL query to retrieve from OCS NG server the list of known PC models
PCs are hardware with type Notebook or Laptop and with different manufacturer than WMWare
OCSPCModelCollector_with_categories_query PCCollection = yes
use_asset_categories = yes
SQL query to retrieve from OCS NG server the list of known PC models
PCs are found using OCS categories. iTop settings are required.
OCSPCCollector_query PCCollection = yes
use_asset_categories = no
SQL query to retrieve from OCS NG server the list of PCs
PCs are hardware with type Notebook or Laptop and with different manufacturer than WMWare
OCSPCCollector_with_categories_query PCCollection = yes
use_asset_categories = yes
SQL query to retrieve from OCS NG server the list of PCs
PCs are found using OCS categories. iTop settings are required.
OCSPCPhysicalInterfaceCollector_query PCCollection = yes
use_asset_categories = no
SQL query to retrieve from OCS NG server the list of physical interfaces for PCs
PCs are hardware with type Notebook or Laptop and with different manufacturer than WMWare
OCSPCPhysicalInterfaceCollector_with_categories_query PCCollection = yes
use_asset_categories = yes
SQL query to retrieve from OCS NG server the list of physical interfaces for PCs
PCs are found using OCS categories. iTop settings are required.
OCSMobilePhoneCollector_query MobilePhoneCollection = yes
use_asset_categories = no
SQL query to retrieve from OCS NG server the list of Mobile phones
Mobile phones are hardware with type Mobile
OCSMobilePhoneCollector_with_categories_query MobilePhoneCollection = yes
use_asset_categories = yes
SQL query to retrieve from OCS NG server the list of physical interfaces for Mobile phones
Mobile phones are found using OCS categories. iTop settings are required
OCSVirtualMachineCollector_query VMCollection= yes
use_asset_categories = no
SQL query to retrieve from OCS NG server the list of virtual machines
Virtual machines are hardware with VMWare as manufacturer
OCSVirtualMachineCollector_with_categories_query VMCollection= yes
use_asset_categories = yes
SQL query to retrieve from OCS NG server the list of virtual machines
VMs are found using OCS categories. iTop settings are required
OCSLogicalInterfaceCollector_query VMCollection= yes
use_asset_categories = no
SQL query to retrieve from OCS NG server the list of logical network interfaces for virtual machines
Virtual machines are hardware with VMWare as manufacturer
OCSLogicalInterfaceCollector_with_categories_query VMCollection= yes
use_asset_categories = yes
SQL query to retrieve from OCS NG server the list of logical network interfaces for virtual machines
VMs are found using OCS categories. iTop settings are required
OCSPrinterCollector_query PrinterCollection= yes SQL query to retrieve from OCS NG server the list of printers

Collect Assets without using OCS categorization

  • Collector will discover all Cis resulting of SQL queries not ending by _with_categories_query defined in config file.
  • Indicate in configuration file

use_asset_categories = no
<CI>Collection= yes (replace CI by required collected class)

Collect Assets with OCS categorization

If you use synchronisation with assets categories, you have to configure the same assets categories both in OCS and in iTop. You can have many categories for one asset type.

If asset (or software) category change, it can change asset (or software) class in iTop, create duplicate data and break some data synchronization. By example if a PC become a Server, it breaks associated PCModel and software synchronization.
Configure Assets categories in OCS

First configure assets categories in OCS thanks to search menu below Inventory choice

Verify that assets correspond to your request and create your category thanks to this search.

You can see your assets categories in menu:

In order to have the same results as with default params, you can create this categories:

In ocs, categorization may take time to be applied on assets.
If you want more informations about asset categorization in OCS please refer to OCS wiki or to OCS community.
Configure Assets categories in iTop

Asset categories are in typologie menu.

You have to create categories exactly with the same names as in OCS and parameter in which class there are collected. If you use categories collector, you only have to indicate the target class.

As soon as you decided to use categories, ensure you indicate target class otherwise collector won't know where creating your object.

Finaly you have the same list in the both environment

Configure Software collection

Query Parameter Launch if Meaning
OCSSoftwareCategoryCollector_query CategoryCollection= yes SQL query to retrieve from OCS NG server the softwarescategories
Usefull only if use_software_categories is set to yes
OCSSoftware_getListFromItop SoftwareCollection= yes
use_software_categories= no
SQL query to retrieve from iTop the list of existing software
OCSSoftwareCollector_query SoftwareCollection= yes
use_software_categories= no
SQL query to retrieve from OCS NG server the softwares with their differents versions
Only software already in iTop are requested.
OCSSoftwareCollector_with_categories_query SoftwareCollection= yes
use_software_categories= yes
SQL query to retrieve from OCS NG server the softwares with their differents versions.
Software are found using OCS categories. iTop settings are required
OCSPCSoftware_getListFromItop SoftwareCollection= yes
use_software_categories= no
SQL query to retrieve from iTop the list of existing software on PC
OCSPCSoftwareCollector_query SoftwareCollection= yes
use_software_categories= no
SQL query to retrieve from OCS NG server the PC softwares with their version
Only software already in iTop are requested
OCSPCSoftwareCollector_with_categories_getListFromItop SoftwareCollection= yes
use_software_categories= yes
SQL query to retrieve from iTop the list of software categories with type PC Software
OCSPCSoftwareCollector_with_categories_query SoftwareCollection= yes

use_software_categories= yes
SQL query to retrieve from OCS NG server the PC softwares with their version.
Software are found using OCS categories. iTop settings are required
OCSWebServerCollector_getListFromItop SoftwareCollection= yes
use_software_categories= no
SQL query to retrieve from iTop the list of existing software with type WebServer
OCSWebServerCollector_query SoftwareCollection= yes
use_software_categories= no
SQL query to retrieve from OCS NG server the WebServer with their version
Only software already in iTop are requested
OCSWebServerCollector_with_categories_getListFromItop SoftwareCollection= yes
use_software_categories= yes
SQL query to retrieve from iTop the list of software categories with type Web Server
OCSWebServerCollector_with_categories_query SoftwareCollection= yes
use_software_categories= yes
SQL query to retrieve from OCS NG server the WebServer with their version
WebServerare found using OCS categories. iTop settings are required
OCSOtherSoftwareCollector_getListFromItop SoftwareCollection= yes
use_software_categories= no
SQL query to retrieve from iTop the list of existing software with type Other Software
OCSOtherSoftwareCollector_query SoftwareCollection= yes
use_software_categories= no
SQL query to retrieve from OCS NG server the Other softwares with their version
Only software already in iTop are requested
OCSOtherSoftwareCollector_with_categories_getListFromItop SoftwareCollection= yes
use_software_categories= yes
SQL query to retrieve from iTop the list of software categories with type Other Software
OCSOtherSoftwareCollector_with_categories_query SoftwareCollection= yes
use_software_categories= yes
SQL query to retrieve from OCS NG server the Other Softwares with their version
WebServerare found using OCS categories. iTop settings are required
OCSMiddlewareCollector_getListFromItop SoftwareCollection= yes
use_software_categories= no
SQL query to retrieve from iTop the list of existing software with type Middleware
OCSMiddlewareCollector_query SoftwareCollection= yes
use_software_categories= no
SQL query to retrieve from OCS NG server the Middlewares with their version
Only software already in iTop are requested
OCSMiddlewareCollector_with_categories_getListFromItop SoftwareCollection= yes
use_software_categories= yes
SQL query to retrieve from iTop the list of software categories with type Middleware
OCSMiddlewareCollector_with_categories_query SoftwareCollection= yes
use_software_categories= yes
SQL query to retrieve from OCS NG server the Middleware with their version
Software are found using OCS categories. iTop settings are required
OCSDBServerCollector_getListFromItop SoftwareCollection= yes
use_software_categories= no
SQL query to retrieve from iTop the list of existing software with type DBServer
OCSDBServerCollector_query SoftwareCollection= yes
use_software_categories= no
SQL query to retrieve from OCS NG server the DBServer with their version
Only software already in iTop are requested
OCSDBServerCollector_with_categories_getListFromItop SoftwareCollection= yes
use_software_categories= yes
SQL query to retrieve from iTop the list of software categories with type DBServer
OCSDBServerCollector_with_categories_query SoftwareCollection= yes
use_software_categories= yes
SQL query to retrieve from OCS NG server DBServer with their version
Software are found using OCS categories. iTop settings are required

Collect Software without using OCS categorization

  • Create softwares in iTop. Collector will try to discover all defined softwares in iTop.
  • Indicate in configuration file

SoftwareCollection= yes
use_software_categories= no

Collector will collect from OCS software's names defined in iTop

Collect Software using OCS categorization

If you use synchronisation with software categories, you have to configure the same software categories both in OCS and in iTop. You can have many categories for one software type. You will need to

  • define in OCS, categories on all sofwtares you want to collect
  • execute collector to have OCS catgeories synchronised in iTop
  • map target class to each categorie syncrhonised
  • execute again collector to receive categorised OCS CIs.
Configure Software categories in OCS

First configure software categories in OCS thanks to Software Category menu below Manage

Click on New Category and give a name to your category

Then click on Add software. Select your category and add a software name or a regexp

You can add many regexp on a software category. You can seen the list in List of regexp

The list of all software in a category can be seen in List of categories

In ocs, categorization may take time to be applied on softwares.
If you want more informations about software categorization in OCS please refer to OCS wiki or to OCS community
Configure Software categories in iTop

Software categories are in typologie menu.

You have to create categories exactly with the same names as in OCS and parameter in which class there are collected. If you use categories collector, you only have to parameters the corresponding class.

IPs and IP Interfaces collection

The collector automatically detects if TeemIp is present on the remote iTop application (as a module or even as a stand-alone application) and if TeemIp Zone Management extension is installed. Should that be the case, then the following parameters may trigger IP addresses and IP interfaces collection.

<?xml version="1.0" encoding="UTF-8"?>
  <parameters>
    ...
    <!-- TeemIp options -->
    <IPCollection>yes</IPCollection>
    <default_ip_status>allocated</default_ip_status>
    <manage_ipv6>no</manage_ipv6>
    <default_view_name></default_view_name>
    ...
  </parameters>
Global Parameter Meaning Sample value
IPCollection Triggers IP addresses collection yes
default_ip_status Satus of newly created IP addresses allocated
manage_ipv6 Triggers IPv6 collection - not working yet no
default_view_name View of the newly created IP addresses <empty string>
Query Parameter Meaning
OCSIPv4AddressCollector_query SQL query to retrieve from OCS NG server the list of IPv4 address
OCSIPv6AddressCollector_query SQL query to retrieve from OCS NG server the list of IPv6 address
OCSlnkIPInterfaceToIPAddressCollector_query SQL query to retrieve from OCS NG server the list of link between IP interface and IP address
At this stage, IPv6 are not collected. This is due to a limitation of the collector base which cannot handle such objects yet. It will work from iTop 3.1.0

Usage

To launch the data collection and synchronization with iTop, run the following command (from the root directory where the data collector application is installed):

php exec.php

The following (optional) command line options are available:

Option Meaning default value
--console_log_level=<level> Level of output to the console. From -1 (none) to 9 (debug). 6 (info)
--collect_only Run only the data collection, but do not synchronize the data with iTop false
--synchro_only Synchronizes the data previously collected (stored in the data directory) with iTop. Do not run the collection. false
--configure_only Check (and update if necessary) the synchronization data sources in iTop and exit. Do NOT run the collection or the synchronization
--max_chunk_size=<size> Maximum number of items to process in one pass, for preserving the memory of the system. If there are more items to process, the application will iterate. 1000

The execution of the command line will:

  1. Connect to iTop to create the Synchronization Data Sources (or check their definition if they already exist, updating them if needed)
  2. Connect to OCS NG to collect the information about the Servers, PC, Virtual Machines
  3. Upload the collected data into iTop (this is the configuration parameter <name_of_the_collector>_query
  4. Synchronize the collected data with the existing iTop CIs.

Scheduling

Once you've run the data collector interactively, the next step is to schedule its execution so that the collection and import occurs automatically at regular intervals.

The data collector does not provide any specific scheduling mechanism, but the simple command line php exec.php can be scheduled with either cron (on Linux systems) or using the Task Scheduler on Windows.

For optimal results, don't forget to adjust the configuration parameter full_load_interval to make it consistent with the frequency of the scheduling.

Steps of synchronization

extensions/ocsng-data-collector-v2.txt · Last modified: 2023/05/16 09:44 (external edit)
Back to top
Contact us