Enhanced global search
- name:
- Enhanced global search
- description:
- Use a faster and more relevant global search with easy-to-filter results
- version:
- 1.2.8
- release:
- 2022-11-15
- itop-version-min:
- 2.7.0
- code:
- combodo-fulltext-search
- state:
- stable
- diffusion:
- Client Store, iTop Hub
- php-version-max:
- PHP 8.1
This extension replace the standard global search with a faster and more accurate search.
Features
-
Users can improve results accuracy by specifying required, optional and denied words.
-
Administrator can fine-tune the results relevance, by changing configuration parameters.
-
It uses MySQL fulltext feature to perform the global search.
Revision History
Release Date | Version | Comments |
---|---|---|
2022-11-10 | 1.2.8 | * N°5400 - Reindex objects modified by SQL
directly * N°5492 - PHP 8 and PHP 8.1 : Add compatibility |
2022-09-07 | 1.2.7 | * N°5503 - Fix indexation (wrong type
checking) * N°5492 - PHP 8 : Add compatibility |
2022-04-08 | 1.2.6 | * N°5041 - Fix indexation for object creation |
2022-04-06 | 1.2.5 | (BROKEN) * N°5041 - Fix unnecessary re-index when modifying n-n links |
2022-01-03 | 1.2.4 | * Fix search needles signs (+, -, “”) * Fix class names handling * Fix CSS on field values (and organization name) |
2021-12-21 | 1.2.3 | * Modification of past search needles * Display results in standard iTop table for a single class * Classes display improvement |
2021-06-02 | 1.2.1 | * Compatibility iTop 2.7 and 3.0 * Fix CSS |
2021-05-25 | 1.2.0 | * Compatibility iTop 2.7 and 3.0 |
2021-05-25 | 1.1.4 | * Prevent indexation concurrency * Fix link for class drill down * fix script refresh-search-index.cli.php to rebuild fully the index * fix script populate-search-index.cli.php to populate the index |
2021-05-04 | 1.1.3 | - broken |
2020-08-25 | 1.1.1 | - Admin console - populate script more robust (memory) - Default configuration moved to XML - Avoid reading blobs from database - Exclude classes by configuration |
2020-06-23 | 1.0.13 | - for iTop 2.7.0 and above - Fix searches with some reserved characters - Search admin menu |
2019-02-04 | 1.0.11 | - Fix broken related link |
2019-01-09 | 1.0.10 | - Fix the friendly name limit - Fix Scheduled task - Fix class name in tests - Add Combodo license |
2019-01-09 | 1.0.9 | - Fix memory limit - Fix Case log indexation - Fix email search - Limit the friendlyname size to fit in the DB |
2018-07-31 | 1.0.8 | - Better indexation - Better search - Support only InnoDB and BOOLEAN mode - Display related objects on search - Display matching fields - Admin page : for re-indexation - Class drill-down - Debug mode |
2018-01-19 | 0.0.3 | - Better indexation when updating data - 'populate_search.php' uses the configuration for the type of table to create (MyISAM or InnoDB) - No constraint on MySQL (MyISAM must be used in version before 5.6) |
2018-01-09 | 0.0.2 | - Better error management. - No indexation at setup (performance issues). - 'populate_search.php' have to be used to create an InnoDB fulltext index. - MySQL >= 5.6 |
2017-12-14 | 0.0.1 | - First version experimental. |
Limitations
-
This extension relies on MySQL full-text feature.
-
It uses the main database configured for iTop.
Requirements
-
MySQL 5.6 or above
-
Mariadb 10.0.5 or above
-
iTop 2.5.0 or above
Installation
Use the Standard installation process for this extension.
<itop_url>/env-production/combodo-fulltext-search/script/populate-search-index.cli.php
Configuration
Parameter | Type | Description | Default value |
---|---|---|---|
object_weight_factor | array | Weight factor to apply for a given object name (format: 'classname' ⇒ 'value') the values can vary from 0 to 10 generally (0 means no results for this class). The default is 1 for every class not specified. The modification of this parameter needs a complete re-indexation of the database. |
array( 'Organization' => 2.0, 'Person' => 1.5, 'Location' => 1.2, 'SLT' => 0.8, ) |
sentence_weight_factor | float | Weight factor for searches with multiple words matching exactly the list of words in that order. | 1000.0 |
mandatory_weight_factor | float | Weight factor for searches with all the words of the search matching at least once. | 50.0 |
start_with_weight_factor | float | Weight factor for searches with one word matching the beginning. | 0.5 |
max_interactive_index_update_time_in_sec | integer | Time in seconds allowed for direct indexation update on change. | 5 |
background_index_refresh_period_in_min | integer | Period in minutes for background indexation. | 1 |
background_max_indexation_time_in_min | integer | Execution time limit in minutes for background indexation. | 0.8 |
background_index_full_rebuild_enabled | boolean | Allow daily rebuild of the index. Only useful if automatic archiving is done. | false |
background_index_full_rebuild_time | hour | Starting hour of the full rebuild of the index. | '01:30' |
background_index_full_rebuild_week_days | weekdays | List of days for index rebuild. | 'sunday' |
excluded_classes | array | List of classes excluded from search. It has to be the finalclass (i.e CustomerContract or ProviderContract but not Contract | [] |
- configuration example 1
-
'combodo-fulltext-search' => array( 'object_weight_factor' => array( 'Organization' => 2.0, 'Person' => 1.5, 'Location' => 1.2, 'SLT' => 0.8, ), 'excluded_classes' => array( 'Attachment', ), ),
- configuration example 2
-
'combodo-fulltext-search' => array ( 'object_weight_factor' => array ( 'Organization' => 2.0, 'Person' => 5.0, 'Location' => 1.2, 'SLT' => 0.8, ), 'sentence_weight_factor' => 10.0, 'mandatory_weight_factor' => 2.0, 'start_with_weight_factor' => 0.5, // Max delay to re-index in realtime, // above this it flags index entries as out of date, // they will be computed by the background task 'max_interactive_index_update_time_in_sec' => 5, // Background task periodicity, should be more than iTop cron periodicity 'background_index_refresh_period_in_min' => 5, // Background task max time per run 'background_max_indexation_time_in_min' => 1, // identical to run refresh-search-index.cli.php in a system cron, // useful if you don't have system access 'background_index_full_rebuild_enabled' => false, // deactivated by default 'background_index_full_rebuild_time' => '11:00', 'background_index_full_rebuild_week_days' => 'wednesday', ),
Background behavior
Create, update or delete object
When ever a user create, update or delete an object in iTop, the
extension will recompute the fulltext search indexes. If it cannot
do it within max_interactive_index_update_time_in_sec
second, it flags out-of-date indexes and they will be processed by
the itop-cron using a background task.
You may see that warning message when you change an organization name for example, as it is present in most iTop objects, so a lot of indexes will have to be updated, more than what can be processed in a user acceptable delay
Searching for one word When searching for a single word (example: demo) the following searches are done:
-
The word: demo
-
The words starting with the search: demo*
Searching for multiple words
When searching for multiple words, (example: user-level lock) the following searches are done:
-
The exact sentence entered: “user_level lock”
-
All the words are present: +user_level +lock
-
At least one word is present: user_level lock
As -
is a keyword to exclude a word, it is replaced
by _
which is another keyword to represent any single
character.
Computing weight
Each search is bringing points calculated by MySQL fulltext search, which are then weighted with the configuration parameters set.
What values to set
To ensure that the search will be efficient within your iTop, there are additional actions for Administrator users:
-
Global search management action which allow to rebuild the indexes after a change of configuration parameter:
object_weight_factor
. -
Debug action which display details on how the result ordering is generated, based on weight.
-
user_level lock has a MySQL provided weight
-
“user_level lock” has the above weight *
sentence_weight_factor
-
Total Score is the sum of the various scores *
object_weight_factor
Debug
action on those “sampling
searches”.Global Search Management
There is an administrator menu, which allow to
-
see on-going status of re-indexation processing
-
run re-indexation tasks for those who do not have access to their iTop system (Use with caution!!!)
Read carefully the text near the checkbox, it is refreshed based on the check status.
-
First checkbox delete all indexes.
-
Second checkbox alone is similar to using the /script/refresh-search-index.cli.php on your iTop system
-
Third checkbox, will request the cron to do the work in background, rather than doing it real-time interactively. Background computation is performed every
background_index_refresh_period_in_min
minutes for a maximum duration ofbackground_max_indexation_time_in_min
minutes before stopping until the next run.
Consequences of your choices:
-
Checking the first one, without the second one, leads to no more index at all
-
Checking the first and second without the third one, leads to timeout.
-
Checking the 3 may work, but it's much better to use the /script/populate-search-index.cli.php on your server. It will be faster and more efficient than truncating the tasks in multiple small pieces of work.
-
Interactive in most cases will timeout and fail (apart from empty test databases).
Scripts
A better alternative to using this administrative page within iTop, is to run in CLI the below scripts:
populate-search-index.cli.php --verbose=0/1 --step-limit=<numer of second>
-
It puts your iTop under maintenance
-
Drop the index table
-
Reopen access to your iTop (also global search will not respond)
-
Rebuild the fulltext index table
wait_timeout
is around 28
800 before running
populate-search-index.cli.php
!To prevent situation where the script and the cron would both try to recompute indexes at the same time, we have used a MySQL lock.
Some distribution, overwrite the default MySQL/MariaDB value for
the wait_timeout
parameter and put 600 (seconds). That
delay is not enough to populate indexes for a big database, as a
result, the script will return “MySQL has gone
away” which is the answer to the unlock request done by
the script when it finishes. In reality, the work was done, indexes
have been recomputed, the error is misleading, it's just the lock
which could not be released, because after
wait_timeout
seconds of inactivity on that session,
MySQL/MariaDB has decided it was useless to wait more, and released
the lock.
⇒ To get rid of that error, just restore default value for
wait_timeout
(28 800).
refresh-search-index.cli.php --verbose=0/1 --step-limit=<numer of second>
-
It does not put iTop under maintenance
-
It takes every existing index, and re-index them one by one
-
Missing index are not created. So the script is not adequate for an existing database on which you just installed the extension.
-
It is useful to remove entries corresponding to recently archived objects
Usage
Just type the words in the global search field,
then enter
or click on the magnifier icon to get the
result :
Tips to specify your search
+
at the
beginning of required wordUserRequest:+PHP error
will search for User Requests which contain mandatorily 'PHP' and maybe 'error'
-
at the
beginning of excluded wordUserRequest:PHP -closed
will search for User Requests which contain 'PHP' but not 'closed' (this exclude all UserRequest with status=closed)
*
at the
end of an incomplete wordOrganization:Combo* Grenoble
will search for Organization which contain a word starting with 'Combo' and the exact word 'Grenoble'. This is useless in single word search, as it does it automatically.
Restricting the search to a class
By default the search searchs on all Classes with category searchable, including abstract classes, such as Ticket, Contact or FunctionalCI.
<class-name>:
at the beginning
of a search pattern to limit the search to objects of this classe
or one of its descendantsFunctionalCI:combodo
will search for any FunctionalCI which contains “combodo”, so the returned objects can be Server, PC, Enclosure,…
The class name can be the internal name (like FunctionalCI) of the display name in the current language used (like CI fonctionnel).