To get the most out of the HighQ AI engine, you need to actively train it. Training means that you help the engine to understand what types of files you are uploading (i.e. helping it understand and recognise a type of document).
When you enable AI training and assign a classifier to files, the files in the selected folder are mapped to the selected classifier name in the AI hub. The AI Hub automatically uses this mapping to train a new File Classifier. The new file classifier is then available as a classification type across the whole HighQ instance (unless you create a 'Private' classifier, which is only used in a single site). If the HighQ AI finds any examples of this file anywhere in the system, it will label it with the new classification.
In addition to classifiers, you can train your own clauses, or re-train HighQ provided clauses.
The information below will help you train your AI engines to better understand file metadata.
We highly recommend you only turn on AI training for only one or two sites. This makes it easier to administer training and to manage the files that are used as training examples in a single place, rather than have them scattered across many sites.
Enabling AI engine training
To enable AI engine training, navigate to Admin > AI Hub > Configure and scroll down to Advanced settings:
Select the Enable AI engine training checkbox and click Save.
This training is for the HighQ AI engine. To train other AI engines, please refer to how to use their training capabilities on their platforms. It is not currently possible to train other AI engines (such as Kira) using the HighQ AI Hub.
Manage your AI training
Files that belong to a classification can be used to train the HighQ AI so it is better equipped to recognise similar files.
Training HighQ AI from the Files module (select multiple files)
You can train the AI with one or more files on your site. This allows you to target a set of new or relevant files for analysis.
Open the Files module.
Select the files to use to train a classifier. Click Action, Assign AI, then select Train HighQ AI.
The Assign AI > Train HighQ AI and Assign contract template options are not available if a folder is selected.
Training HighQ AI from the Files module (select the contents of a folder)
You can train the HighQ AI with the contents of a folder. This allows you to use every file in a folder as an example for training.
Select the folder and click More actions > Edit details:
In the Edit folder screen, navigate to the Settings tab and click Select... (under the AI Training heading) to open the Classifiers screen (see below).
Selecting an existing classification
The Classifiers window displays the names of all available classifiers, with a short description and a tag to show the language used by the classifier.
Select the classifier you want to associate with the folder and click Done.
See Classifiers provided by HighQ AI for a description of the classifiers packaged with the HighQ AI engine.
Adding new classifiers
If no classifier matches your needs, you can create your own, using your own documents for analysis.
Click New classifier:
The New classifier screen opens:
Enter a name and description for your classifier and also choose the scope of the classifier: if you choose a private scope, the AI will only search for documents in that specific site and not instance wide.
You have the option to create a new classifier in English or German. Please contact your account manager should you wish to add classifiers in German.
Once you have entered the details, click Add to save your new classifier. Select your new classifier in the Classifiers window and click Done.
Training the AI with new files
If you assigned a folder to train the AI, any files uploaded to the folder will qualify as 'training files' for the assigned classification in the AIML (AI Modelling Language) database. This means that the AI understands that the type of document uploaded to the folder is a certain type, and will learn to identify this type if it sees it again at a later date.
When a new classification is created or a new version of an existing classification is added, the system administrator should check the list of HighQ AI models to ensure that the best AI versions are being deployed to their sites. They do this in the System Administration for Document Analysis, by managing the HighQ AI.
Training the AI
Once you have set your classifier for your folder, you can start to train the AI. You can train the AI from the Files module, an iSheet or the Admin module.
Manage AI training in the Admin module
Navigate to Admin > AI Hub > Configure:
In the HighQ section, click More actions > Manage Training:
The Manage window allows you to manage training for both classifiers and clauses.
It shows information about each classifier and the current version used. Click Train to update the trained version with any new files or folders that the classifier that been assigned to.
Click Choose version to see versions that have been trained - in this screen, you can revert to an older version of the trained classifier (click More actions, then Rollback to here), or delete a version (click More actions, then Delete).
A system administrator can use rollback to delete multiple versions at the same time.
Only a system administrator can delete a version created by another user.
Click Done and Save if you made any changes.
Show training examples for HighQ AI training
As of version 5.6, the train button for a classifier will not be enabled until enough training examples have been provided.
This makes it easier to see how many more examples are required before a new version of the AI model can be trained.
To access this, within your site, click Site admin:
The Site admin screen will be displayed. Within Site admin, navigate to AI Hub > Configure:
The list of all available AI will be shown within the AI Hub. Navigate to HighQ AI and click More actions. The new Manage training option will be displayed:
Click Manage training. The Manage - HighQ AI screen will be displayed. Within the Manage - HighQ AI screen, click the Classifier tab:
You can now see many more examples are required to train a new version of the AI model.
Please note that due to our "Examples cumulative" enhancement, all newly trained or out of the box AI models will show the label "Add 1 example" to train, as they can be trained by adding a single new example
Please also note that when you have trained a new classifier or clause, you must enable it in any site that you want to use it. It is not automatically enabled, in case you are not ready to make use of that classifier or you do not wish to use it in other sites
Minimum number of example documents
If you do not have enough examples of documents to train the AI, you will receive the following message:
You must ensure there is at least a minimum of 50 new files in folders where the same classifier is applied, instance wide (but see the note below). The AI engine needs a minimum number of examples, and more is better. Once you have checked that there are at least 50 new files of the correct classification, navigate back to the admin panel and click More actions > Train AI.
By default, if you do not have 50 files for all the classifiers - the AI engine will only create new versions where the AI Hub can find 50 new examples.
The default and recommended number of examples to train the HighQ AI Engine classification type is 50 documents. This can be reduced via ASP admin upon client request to the support team. The lower limit is 5 examples, but caution should be exercised if the limit is reduced.
As fewer examples are used, training accuracy will decrease; appropriate discussions should take place prior to making the change with the solutions consultant, client success resource or account manager.
Please note that as of version 5.6, the requirement for 50 new files has been reduced to one file. This change means that you only need to find one more example in order to train a new version of a model, as the new version will be trained on the original examples plus any additional (new) examples.
Using the information
Once you have set up your AI engine and have trained it, you can insert it into a file metadata iSheet, and use the data in data visualisation.
