:: Version 2.7.0 ::

2.6.x to 2.7.0 Migration Notes

The version 2.7.0 of iTop is roughly backward compatible with previous 2.x versions, with a few significative changes.

This document highlights issues which can occur while migrating your iTop to this version.

  • For a focus on new features check What's New
  • For an exhaustive list of changes check the Change Log

Impact on Users

  • OQL queries returning multiple classes of objects, may return less data after the upgrade, to users having “Allowed organizations” set on their user. This is due to a fix in query generation.

Portal behavior changes

  • External key display in forms are now clickable, if you rather keep the previous behavior replace the xxx_id attribute code in your form with xxx_id_friendlyname.
  • We have update Font Awesome from v4 to v5.12.0, check that your icons in portal are still displayed as expected. Some icons names have changed between the 2 versions, old name will still work except if you have used aliases from the version 4, which will need to be changed.
  • For security reasons, LinkedSet displayed in forms, such as for example contacts associated to a User Request, are now by default limited to the scope of the portal user. A typical example there, was that portal user could see on his User Requests, IT department associated contacts, even if in theory his Contact scope was limiting him to contacts of his own organization. If you want to keep the previous behavior, you need to modify the forms in your portal displaying those linksets as explained in the 2.7 new features in the Portal XML reference

Change for Administrators users

  • Admin tools has been modified and some of its entries moved to new Configuration and System menus.
  • Labels of Persons fields in User objects have been changed. This will change what you get in CSV and excel export of Users:
old label new label
Contact (person)->First Name Person->First Name
Contact (person)->Last Name Person->Last Name
Contact (person)->Email Person->Email

To check before upgrading

System

  • Ensure that you have php-gd installed on your iTop server. It is now enforced at setup, as it is mandatory for handling correctly images within caselogs and description.
If you use php-memcache, migrate to php-memcached

DB Views

If you are using the Database Views, your reports will be broken by the update.
  • Starting from iTop 2.7.0, due to an OQL optimisation, some columns in the views were renamed.
  • In that case, deploy this extension to restore the views as they were in 2.6 and before.

Backup & MariaDB

With the following versions of MariaDB, iTop backup won't work:

10.5.x 10.4.x 10.3.x 10.2.x
10.5.7 10.4.16 10.3.26 10.2.35
10.5.8 10.4.17 10.3.27 10.2.36

The symptoms

When you perform this query:

MariaDB [sbacmdb]> select * from view_DBServer;

You get this error:

Error: Failed to execute mysqldump: mysqldump: Couldn't execute 'show create table `view_DBServer`': You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '= 0 and `SoftwareLicence_softwarelicence_id_Licence`.`end_date` < date_format...' at line 1 (1064)

Root cause: This is a MariaDB issue, providing the versions in which the bug was fixed by them.

Strategy: avoid those buggy MariaDB versions. Downgrade to a not impacted version or upgrade to fixed versions when they are available:

10.5.9 10.4.18 10.3.28 10.2.37

Workaround

  • Drop Views before doing Backup, Views are not required by iTop itself.
  • Cautious, a Setup will recreate the Views, so drop them again after that.

Views will be totally removed in 3.0, an optional free extension will be proposed for those still using them

Database size

  • This release will increase the size of your database
  • History tables and event notification tables will increase by 1KB per row
  • For instance if you have 1 million history entries, then you need 1GB of additional disk space just for the History
 SELECT COUNT(*) FROM priv_changeop;  /* History */
 SELECT COUNT(*) FROM priv_event;  /* Notification */

Setup can take time

The upgrade to the 2.7 product version, will generate database changes, which can take time if you have a lot of data.

Underlying changes

Finalclass on intermediate classes

To increase query performance, we are adding on each intermediate table/class within a class hierarchy, a new column to store the final class (Only on intermediate classes as on a root class this column exist already and on a final class, this column is useless as this is the table name).

As a result during Setup, for each of those intermediate classes, iTop runs an ALTER TABLE

Example
ALTER TABLE `priv_changeop_setatt` 
   ADD `optype` VARCHAR(255) CHARACTER 
      SET utf8mb4 COLLATE utf8mb4_unicode_ci 
      DEFAULT 'CMDBChangeOpSetAttribute', 
   ADD INDEX `optype` (`optype` (95))

and just after an UPDATE table

UPDATE `priv_changeop_setatt`,`priv_changeop` 
   SET  `priv_changeop_setatt`.`optype` = `priv_changeop`.`optype` 
   WHERE `priv_changeop_setatt`.`id` = `priv_changeop`.`id` 

Attachments

To store creation date and user having uploaded an attachment, iTop Setup automatically updates the DB schema, like this:

ALTER TABLE `attachment` ADD `creation_date` DATETIME, 
                         ADD `user_id` INT(11) DEFAULT 0, 
                         ADD INDEX `user_id` (`user_id`);

Warning, if you're using the table prefix option (db_subname iTop config file parameter) the table name will vary. For example with db_subname='myprefix' then the table name would be : myprefix_attachment.

Risks

This operation can take a long time if you have a big number of attachments and/or a huge history on your objects.

  • ALTER TABLE execution time is roughly proportional to the table size
  • UPDATE TABLE execution time is mainly impacted by the number of rows and a bit by the table size.

Examples of execution time

Here are examples of the time it took with table size and count:

Table Execution
Size (GB) count Seconds Query (beginning)
57 250000 1486 ALTER TABLE `attachment`
7,3 32000000 212 ALTER TABLE `priv_changeop_setatt`
7,3 32000000 2621 UPDATE `priv_changeop_setatt`,
2,4 1760000 132 ALTER TABLE `priv_changeop_setatt_scalar`
2,4 1760000 465 UPDATE `priv_changeop_setatt_scalar`,
3,1 12667000 96 ALTER TABLE `priv_changeop_links`
3,1 12667000 1076 UPDATE `priv_changeop_links`,
1,7 7531000 69 ALTER TABLE `priv_event_notification`
1,7 7531000 613 UPDATE `priv_event_notification`,
0,8 143000 36 ALTER TABLE `priv_changeop_setatt_longtext`
0,8 143000 54 UPDATE `priv_changeop_setatt_longtext`,

longqueriesmigration27.xlsx

 ALTER TABLE `priv_changeop_setatt` ADD `optype` VARCHAR(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT 'CMDBChangeOpSetAttribute', ADD INDEX `optype` (`optype` (95))  
 ALTER TABLE `priv_changeop_setatt_scalar` ADD `optype` VARCHAR(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT 'CMDBChangeOpSetAttributeScalar', ADD INDEX `optype` (`optype` (95))  
 ALTER TABLE `priv_changeop_setatt_longtext` ADD `optype` VARCHAR(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT 'CMDBChangeOpSetAttributeLongText', ADD INDEX `optype` (`optype` (95))  
 ALTER TABLE `priv_changeop_links` ADD `optype` VARCHAR(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT 'CMDBChangeOpSetAttributeLinks', ADD INDEX `optype` (`optype` (95))  
 ALTER TABLE `priv_event_notification` ADD `realclass` VARCHAR(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT 'EventNotification', ADD INDEX `realclass` (`realclass` (95))  
 ALTER TABLE `attachment` ADD `creation_date` DATETIME, ADD `user_id` INT(11) DEFAULT 0, ADD INDEX `user_id` (`user_id`)  
 UPDATE `priv_changeop_setatt`,`priv_changeop` SET  `priv_changeop_setatt`.`optype` = `priv_changeop`.`optype` WHERE `priv_changeop_setatt`.`id` = `priv_changeop`.`id`  
 UPDATE `priv_changeop_setatt_scalar`,`priv_changeop` SET  `priv_changeop_setatt_scalar`.`optype` = `priv_changeop`.`optype` WHERE `priv_changeop_setatt_scalar`.`id` = `priv_changeop`.`id`  
 UPDATE `priv_changeop_setatt_longtext`,`priv_changeop` SET  `priv_changeop_setatt_longtext`.`optype` = `priv_changeop`.`optype` WHERE `priv_changeop_setatt_longtext`.`id` = `priv_changeop`.`id`  
 UPDATE `priv_changeop_links`,`priv_changeop` SET  `priv_changeop_links`.`optype` = `priv_changeop`.`optype` WHERE `priv_changeop_links`.`id` = `priv_changeop`.`id`  
 UPDATE `priv_event_notification`,`priv_event` SET  `priv_event_notification`.`realclass` = `priv_event`.`realclass` WHERE `priv_event_notification`.`id` = `priv_event`.`id`

Mitigation

To identify tables which can take time, you can use this query, modified with your true itop db name

SELECT TABLE_NAME AS "Tables", 
    round(((data_length + index_length) / 1024 / 1024), 2) AS SIZE 
  FROM information_schema.tables 
  WHERE table_schema =  'itop db name'
  ORDER BY SIZE
Ensure that you have at least the size of the biggest table in free disk space on your iTop server

Then run the upgrade process on a test environment on a copy of your production database. The setup.log file will provide you the exact queries which run.

You can run those SQL queries on your production database before upgrading ! Apart from the performance degradation it can induce during the execution, there is no risk to break iTop as it will just ignore the added columns, until you upgrade it to version 2.7

If you run manually the ALTER TABLE on a given table, run also the UPDATE query, otherwise your database will be incoherent and iTop 2.7 will not work.
There is no UPDATE query for attachment table.

Setup process

If your browser returns a timeout during Setup database upgrade, don't kill your web server nor your database server. As despite this browser message, the Setup is going-on.

You can connect to mysql to check the queries which are on-going, you will see the time already spent on the current query, if you run the command again, the time will increase or show the next query.

php>mysql -u root -p xxxx
mysql>show processlist;

As long as queries are ongoing, the setup is still going on. Then run

tail -f setup.log
 
2020-03-02 13:49:52 | Ok      | <---- Exiting read only mode | SetupLog

to ensure that the setup is ended.

If your browser timeout, then the Setup is not completed. Once you have ensured that the database upgrade is finished, you must run the Setup again, also this time it will be faster as the database upgrade will not be performed again.

Second iTop Upgrade to 2.7

Important thing to know if you have rolled-back your 2.7 iTop upgrade,
and now want to upgrade again the same iTop to 2.7.0 or 2.7.1:

If you upgrade your production instance of iTop to 2.7 version, then later for whatever reason, you downgrade your iTop to any previous version. Everything will work smoothly, but the day you upgrade again to 2.7.0 or 2.7.1, we have a known issue in that case. To fix it you must run the Database integrity menu and fix manually in SQL directly on your MySQL or MariaDB database, the missing finalclass values for objects created in your iTop during the rollback interval.

Extensions incompatibilities

Extensions broken on Portal

Because of the Silex to Symphony migration, the following extensions are not compatible with iTop 2.7 Portal. A new version will be published mid of April 2020 and if you have deployed them on your iTop, you must upgrade those extensions before upgrating your iTop.

If you are a Combodo customer, using an iTop Professional or Essential:
This does not apply to you!

Custom extensions on Portal

If you have custom extensions bringing new bricks in your portal, you will need to get/write a new version of those extensions.

Because, in order to ensure better security, support and sustainability; we migrated the portal's framework from Silex 2.x to Symfony 3.4. Even though we managed to keep a backward compatibility for most of the code, if you made a custom extension for the portal, you are most likely to rework part of it.

Read this page to check if your extension is concerned.

End of Legacy Portal

As announced, the Legacy portal is no more available in iTop 2.7

XML Datamodel

Breaking changes

The XML default datamodel of iTop 2.7, contains a few changes which can collide with changes you may have done already on your iTop datamodel though extensions (or using the ITSM Designer if your are a Combodo customer).

Those conflicts should be handled automatically during iTop Setup, if not read carefully the message, it explains which XML node is failing.

Branding

  • The default theme is now included under /itop_design/branding/themes/theme[@id=“light-grey”], so if you altered the branding node, make sure to put the alteration flag (eg. _delta=“define”) on the subnodes.
itop_design iTop 2.7.0
  <branding>
    <themes>
      <theme id="light-grey">
        ...

No more working:

itop-design in MyExtension
  <branding _delta="define">
    <main_logo>
      <fileref ref="logo-combodo_f737e922334a6d51b0d98e9f590d2295"/>
    </main_logo>
    <login_logo>
      ...
  </branding>

Compatible with 2.7:

itop-design in MyExtension
  <branding>
    <main_logo _delta="define">
      <fileref ref="logo-combodo_f737e922334a6d51b0d98e9f590d2295"/>
    </main_logo>
    <login_logo _delta="define">
      ...
  </branding>

Legacy customers portal

The legacy customers portal was deprecated for a few releases and has now been completly removed. If you had redefined some of its constants or the routing to it, you need to correct your XML.

Constants: Remove all constant nodes starting with “PORTAL_” as they don't exist anymore.

itop_design iTop 2.6 and earlier
<constants>
    <constant id="PORTAL_POWER_USER_PROFILE">...</constant>
    <constant id="PORTAL_SERVICECATEGORY_QUERY">...</constant>
    ...
</constants>

Routing: Remove “legacy_portal” from the portal nodes.

itop_design iTop 2.6 and earlier
<portals>
    <portal id="legacy_portal">
        ...
    </portal>
</portals>

Customers portal

Navigation rules have been added under /itop_design/module_designs/module_design[@id=“itop-portal”]/forms/form[@id=“<FORM_ID>”]/properties for the “ticket-create”, “ticket-edit”, “ticket-reopen” and “ticket-apply-stimulus” forms, so if you altered the properties node, make sure to put the alteration flag (eg. _delta=“define”) on the subnodes.

itop_design / module_designs / module_design / forms iTop 2.7.0
  <form id="ticket-create">
    <properties>
      <navigation_rules>
        <submit>
          <default>go-to-open-requests</default>
        </submit>
        ...

No more working:

itop-design / module_designs / module_design / forms in MyExtension
  <form id="ticket-create">
    <properties _delta="define">
      <always_show_submit>true</always_show_submit>
    </properties>
      ...
  </form>

Compatible with 2.7:

itop-design / module_designs / module_design / forms in MyExtension
  <form id="ticket-create">
    <properties>
      <always_show_submit _delta="define">true</always_show_submit>
    </properties>
      ...
  </form>

Admin Tools menus

The Admin tools sub-menus have been moved under the newly created group-menus Configuration and System. If you have redefine those menus, it may not work as expected.

Other

  • A new GetTicketRefFormat() method is now part of the Ticket|UserRequest|Incident|Problem|Change classes, so if you added a custom method with the same name, you'll have to adjust your code.

Deprecations

While these are not breaking changes, you should avoid using deprecated code. This is a summary of those introduced in the 2.7 version:

  • You should verify if you make use of the deprecated MetaModel::GetNextKey, such use is generally done into /itop_design/classes/class/methods/method[@id='DBInsertNoReload']
  • Portal: Action rules of type goto should be replaced by navigation rules. This means that instead of having an action rule in your brick, you should have a navigation rule in the corresponding form. Search for the following XPath:
    • /itop_design/module_designs/module_design/action_rules/action_rule/submit
    • /itop_design/module_designs/module_design/action_rules/action_rule/cancel

For developers

User rights applied on the User class

Previously PHP code getting User object instances were querying database regardless of the current user rights. This was changed in 2.7.0.

However, some code still needs to access the User class even if the current user doesn't have rights on this class. To do so, you will need to change your code to use the AllowAllData option.

Two examples of “no more working code” and the “appropriate fix”:

MetaModel::GetObject

// before 2.7.0 this was working for non admin users
$oUser = MetaModel::GetObject('User', $iCurrentUserId);
 
// After 2.7.0 to search for objects of the User class 
// You need to explicitly specify to ignore current user rights on the User class
// Forcing the 4st parameter to true, bypass "current user rights"
$oUser = MetaModel::GetObject('User', $iCurrentUserId, true, true); 

Reference: MetaModel::GetObject

DBSearch::AllowAllData

// before 2.7.0 this was working for non admin users
$oFilter = DBSearch::FromOQL('SELECT User WHERE id = :id');
$oSet = new DBObjectSet($oFilter, array(), array('id' => $iCurrentUserId));
$oCurrentUser = $oSet->Fetch();
 
// After 2.7.0 you should add this to get Users objects
// Despite current user doesn't have rights on the User class :
$oFilter = DBSearch::FromOQL('SELECT User WHERE id = :id');
$oFilter->AllowAllData(); // this method call will bypass user access rights
$oSet = new DBObjectSet($oFilter, array(), array('id' => $iCurrentUserId));
$oCurrentUser = $oSet->Fetch();

Reference: DBSearch::AllowAllData

ApplyStimulus changed

If the stimulus is not applicable to the object in its current state, the method ApplyStimulus() now returns false', instead of true'' before

Removed images

Many images have been replaced by Font Awesome icons in the admin. console. Those that were not used anymore have been removed from the iTop package (/images folder). This means that if you were using them in your extensions, you should change your code accordingly.

images/actions_bkg.png
images/actions_left.png
images/asc.gif
images/help.png
images/home.png
images/menu.png
images/mini-arrow-orange-open.gif
images/mini-arrow-orange.gif
images/mini_add.gif
images/mini_search.gif
images/mini_tree.gif
images/newsroom-message.svg
images/newsroom_menu.png
images/on-off-menu.png
images/onOffBtn.png
images/pencil-menu.png
images/printableversion.png
images/reload.png
images/searchBtn.png
images/settings.gif
images/zoom.gif
2_7_0/install/260_to_270_migration_notes.txt · Last modified: 2021/02/15 15:42 (external edit)
Back to top
Contact us