After you have trained your AI, you may be interested in determining how it’s performing and what images within your data set it might be having problems identifying. To do this you can use the ‘AI Examination’ option, available via right clicking a label within ‘Label images’ screen.
Lucid AI Right click popup menu option for label examination by the AI
The ‘AI Examination’ will be available on the pop-up context menu. After selecting this option, the AI will go through your training set and report on each region. Once the AI has examined all the regions a notification will be sent to let you know it can be reviewed.
Lucid AI notification example of completed AI examination of a label
To review the images the AI is having difficulties with use the search bar filter option ‘AI Unrecognized’. The search will then return all the images the AI doesn’t recognize for its defined region labels.
Lucid AI filter images containing unrecognized region labels
Lucid AI Examination results filtered to those images containing regions that weren’t recognized. In this example, 15 regions were unrecognized out of 309.
Click on an image (as shown above) and then a region (as shown below once the image has loaded). The identification statistics are then displayed. The statistics label will show the top entity label recognized along with the percentage match for the defined label (what is should have recognized it as).
Lucid AI Example of a poorly recognized whole image region label. In this particular case the image is blurry and the region captures too much unwanted background.
From this data you may see a trend that tells you don’t have enough images/regions of a particular state and so the AI is regularly failing to recognize them. For example, you may only have a few regions defined for the seed pod for the plant in question. The AI may consistently identify other parts of the plant such as leaves and flowers, but not the pods. Therefore, to improve the recognition of the plant you need to add more images/regions of the pods. Or as in the example above, your image may be of poor quality and/or the region label is not capturing the most appropriate part(s) of the image.
Once the task of annotating images with entity label regions has been completed training the AI can begin. Starting the AI training process is a simple as clicking the ‘Start training’ button. The AI training process is an extremely intensive computing task. Depending on the number of entity label regions defined and level of image augmentation this training process can take anywhere from a few hours to multiple days of computing time. Once the training process has begun it can’t be started again until the current training session has completed. You can continue to manage your AI project such as add images and create further annotations while a training session is underway. Any new additions to the project will not be included until the next training session is started.
It’s important to understand that each time you train your AI it’s not a cumulative process. The training session resets and retrains your AI from scratch. However, the AI from the last training session can still be used while undergoing its next training session. It is also possible to rollback your AI to its previous build, if you are unhappy with the latest trained edition. See the Rolling back your AI help for more information on this topic.
When your AI training session is started the trainer will give you an estimated completion time which is calculated on the amount of training data to be processed. This estimate isn’t a guaranteed time since several other factors can affect the training time, particularly the current computing load. That is how many users are working on the project and accessing the AI during the training session. Since the training process doesn’t have a precise end time and will most likely take longer than a user session (the average length of time working on a project). The user who triggered the training session, along with other project members listed in the project notification settings will be emailed once the training session has completed and the new AI has become available for use.
Due to the amount of time needed to train an AI it is possible on the rare occasions the service is interrupted due to system maintenance and updates. If this does occur you will be automatically notified, via email, of the interruption and you will need to restart the training again when next logged in.
Training analysis
During your AI training it possible to undertake additional analysis of how well the AI is expected to perform. This analysis is optional as it can easily double the training time needed. Analysis is generally recommended when the AI is first trained on an image set or the training image set changes significantly or additional entity labels are introduced. Training analysis is not necessary when small changes to the image set are made. The analysis of your AI will give you a performance percentage for each entity label when using the image data held back during the training process. The closer your entity label performance percentages are to 100% the better your AI is at recognizing image data for that entity label.
It is unlikely the AI training will achieve high 90th percentile range across your entire entity label set. Anything above the mid-75th percentile should achieve very good recognition results, assuming the training data set is a match for the images the AI will encounter. If you are seeing many of your entities with percentile ranges lower than this, then it is likely you either don’t have enough image regions defined for that label or the image regions for the entity label are possibly too dissimilar. For example, training your AI to recognise a plant species based on its flowers and seeds using the one species name label. Flower and seeds are very different looking and would be much better split into their own separate entity labels such as ‘Species name flower’ and ‘Species name seed’.
The training analysis will be emailed to you, if selected, as a part of the training completed notification.
Testing your AI
Lucid AI Testing the AI example
After the AI has been trained it is important to try it out. The testing interface is very similar to how entity label regions are annotated on an image. To begin testing you first must provide an image for testing. For a true test you should provide an image that is not part of the image data set. Once the image has been selected, one or more regions of interest can be applied to the image. As with training data, it’s not always desirable to have the AI try and distinguish between multiple species within the one region or excessive background. Regions waiting testing will be labelled ‘To be determined’. Once the regions have been submitted, the AI will review each of the defined regions and return the top five entity label probabilities for each, ordered from the most likely to the least likely. The highest probable entity labels will be applied to the annotated regions. For convenience, a random selection of images from the training data set will be available, if the result entity label is clicked, as a reminder of what the entity is and the type of images that represent that label.
For faster visual acuity the results of each region are colour coded and a success level icon assigned. For example, when the AI is very sure of its prediction (80% or higher) then the label colour will be green. If the top prediction is 80% or greater than a gold trophy icon will also be displayed.
Lucid AI Test AI button options
Choose image
An image must be provided for the AI to test against. Either click the ‘Choose image’ button to browse for suitable JPEG or PNG image file on your local device. Or drag and drop one from your file system to the panel that contains the ‘Choose image’ button. Alternatively, you can simply paste a copied image from your system clipboard (E.g., Windows – Ctrl + v or for OSX – Command + v) left click on the panel to give it focus, then paste, if the clipboard image doesn’t update as the selected image.
Clear regions
Clears regions from the selected image.
Create whole image region
Automatically creates a whole image region.
Zoom image
Zoom in or out on the image. Large images may exceed the available space within the interface, zoom out will make it fully visible. The mouse wheel can also be used to zoom.
Adding testing regions
To annotate test regions right click on the image and select either the rectangle or polygon region option from the pop-up context menu. If the polygon region type is selected use the mouse right click pop-up context menu to complete the polygon drawing.
Tip
After the result of the region(s) are returned you can adjust one or more regions for retesting.
Test
Click the test button to submit the region(s) to the AI to analyse.
Note
If no region is defined, then a whole image region is assumed for the test.
Results
The right-hand side results panel will list each region along with its top five predictions. Each set of results will be numbered, along with the corresponding region label, including the top prediction entity label, to indicate region match result. Clicking on a result will load a random sample of images from the corresponding training set as shown in the screen shot below.
Lucid AI view sample images related to the results
