running a macro on invisible images

misbrit2128

Hello,

For some background,  I am relatively new writing macros and have been building all of my macros from recordings and then modifying as I learn more. I have found little documentation to learn from other than this community. If there is a good source to learn more about writing macros in Image Pro, I would like to know where it is!

I have a macro that flattens an image, counting objects, collect measurement data in data collector, generates a mask of the objects for an image, saves the mask, and then closes the mask and original image. I want to batch process this macro on 1200 images. When the batch processing starts the macro takes about 3 seconds to run, however, by the end of the batch the macro takes about 13 seconds to run on an image. The macro only works on visible images at the moment and I suspect that there could be some hidden images that it holds on too but I haven't figured out how to use the debugging macros to diagnose the problem. I have also read on here that processing on hidden images is faster but I am unclear about how to do that. Is there help document that walks through the process of setting up a macro to work on invisible images? I am hoping that working entirely on hidden images perhaps reduce the chance Image Pro is keeping other hidden images in memory. 

Thanks!

YuriG

Hi misbrit2128,

The way you create the macros is correct, but there are many different nuances in optimizing the recorded macros and editing them. The documentation about this is included into the help, though it could be improved. You can also use the pre-installed macros (in the Scripts folder) and the apps, which could be downloaded form our app center, as examples. (look at https://www.mediacy.com/imagepro/learn/media and search for "macro")
 I would also recommend checking the Forum for specific programming issues, where we try to address all programmers' questions.

Regarding running the batch processing on invisible images: we already have several topics on the forum that discuss it (search for "Invisible"), but here is the summary:

You can modify the recorded macro to make it a LoopMacro for batch processing adding an argument (doc), which is the invisible document passed by batch macro:

Public Sub LoopMacro(doc As Object)
...

Set the Visible property to False for commands that create new images, in you case it's Mask:

        With Measure.Measurements.CreateCommands.Mask(Counting_3)
            .Visible=False 'don't show the mask
            .Run(doc1, image1)'image1 is the mask
        End With

Some commands, such as QuickSaveSelectedForPublication, do not work with invisible images, so you have to use alternatives.

I've modified your macro to work in with batch processing with "Display Documents" off:

    'process invisible image from the Batch
    'doc - loaded image document
    Public Sub LoopMacro(doc As Object)
        Dim Counting_3 = New SimpleScript
        Dim doc1 As IMcDocument, image0, image1 As McImage, doc2

        'use the document passed from the Batch macro
        doc1=doc

        With Process.Filter.EnhancementCommands.Flatten(Counting_3)
            .BrightOnDark = False
            .FeatureWidth = 20
            .Run(doc1, image0)
        End With

        With Measure.ThresholdToolCommands.Open(Counting_3)
            .FileName = ThisApplication.Path(mcPathType.mcptConfigurationFiles) & "Threshold, 6-21 vent-S7.rge"
            .FilterIndex = 1
            .Run(doc1)
        End With

        With Measure.MeasurementsCommands.ExecuteCount(Counting_3)
            .Run(doc1)
        End With

        With Measure.Data.CollectorCommands.Collect(Counting_3)
            .Run(doc1)
        End With

        With Measure.Measurements.CreateCommands.Mask(Counting_3)
            .Visible=False 'don't show the mask
            .Run(doc1, image1)'image1 is the mask
        End With

        'QuickSaveSelectedForPublication doesn't work with hidden images        
        'save mask directly
        Dim fName As String=doc1.FileName
        'use default QuickSaveForPublications path
        Dim outPath As String=ThisApplication.Settings("QuickSave", "Publication").Get("Path", ThisApplication.Path(mcPathType.mcptWritableImages))
        'generate mask name, save it to the same folder as the original image, but with _Mask suffix in JPG format
        Dim maskName As String=outPath & System.IO.Path.GetFileNameWithoutExtension(fName) & "_Mask" & ".jpg"
        image1.SaveAs(maskName)
        image1.Close'close the image
End Sub

I've also attached the complete project.

I can recommend other ways to improve batch performance, such as Data Collection. In your macro you collect data from every image, I don't know what you collect, but if you just use the default Measurements table and your images may contain thousands of object, data collection will take longer time with every iteration, as multiple rows will get merged in Data Collector tables. In this case I recommend collecting only Statistical values of the measurements, use Measurement Stats table and collect Count, Mean (and other stats) of your values, so you get only 1 row of data per image. You may also create a separate "RunBefore" macro to setup and initialize data collector. (check https://www.mediacy.com/imagepro/learn/media and search for "Data Collector").

Best regards,

Yuri

misbrit2128

Hi Yuri,

So I used your macro and it is working on invisible images which is great. This significantly shorten the time to complete a batch. However, I am finding that sometimes particles are not getting counted unless the display documents is on. Suggestions as to why Measure thresholding and counting would be different?

Thanks!

YuriG

Hi,

There should not be difference between count on invisible image and on visible. It could be a problem in a macro that one of the operation work with visible image, e.g. loads segmentation options (IQO file) to ActiveImage (or ActiveDocument) instead of the invisible image. Please check your macro that all of the function use the invisible image as input parameter (doc or derived from doc images). Let me know if you cannot find the problem.

Yuri

misbrit2128

Hi Yuri,

I was not able to find the error. Below is the marco/subroutine I was using in batch processor. I have also included the image I noticed the error with. All of the particles with a pixel value below 146 should be counted however none are counted when the image is invisible.

Thanks!
misBrit2128

'process invisible image from the Batch
    'doc - loaded image document
    Public Sub LoopMacro(doc As Object)
        Dim Counting_3 = New SimpleScript
        Dim doc1 As IMcDocument, image0, image1 As McImage, doc2, varList1

        doc1=doc

        With Process.Filter.EnhancementCommands.Flatten(Counting_3)
        .BrightOnDark = False
        .FeatureWidth = 20
        .Run(doc1, image0)
        End With

'        With Measure.MeasurementsCommands.Options(Counting_3)
'            .Segmentation.AutoFindPhase = MediaCy.IQL.Features.mcFindPhase.mcfpManual
'            .Segmentation.SegmentationType = McMMOptions.mcmmSegmentationType.mcmmstThresholdSegmentation
'            .Run(doc1)
'        End With

        With Measure.ThresholdToolCommands.Open(Counting_3)
        .FileName = ThisApplication.Path(mcPathType.mcptConfigurationFiles) & "Mono 146.rge"
        .FilterIndex = 1
        .Run(doc1)
        End With

        With Measure.MeasurementsCommands.ExecuteCount(Counting_3)
        .Run(doc1)
        End With

        With Measure.Data.CollectorCommands.Collect(Counting_3)
        '.Image = .GetImage("Measure.Data.Collector.Collect")
        .Run(doc1)'New List({doc1}))
        End With

        With Measure.Measurements.CreateCommands.Mask(Counting_3)
            .Visible=False 'don't show the mask
            .Run(doc1, image1)'image1 is the mask
        End With

        'QuickSaveSelectedForPublication doesn't work with hidden images        
        'save mask directly
        Dim fName As String=doc1.FileName
        'use default QuickSaveForPublications path
        Dim outPath As String=ThisApplication.Settings("QuickSave", "Publication").Get("Path", ThisApplication.Path(mcPathType.mcptWritableImages))
        'generate mask name, save it to the same folder as the original image, but with _Mask suffix in JPG format
        Dim maskName As String=outPath & System.IO.Path.GetFileNameWithoutExtension(fName) & "_Mask" & ".jpg"
        image1.SaveAs(maskName)
        image1.Close'close the image
End Sub

YuriG

I see that you use "Mono 146.rge" in your macro. Can you please attach it too?

misbrit2128

How do I do that? I tried to attach it in the previous post and the file format is not allowed.

YuriG

Zip it first, then it can be attached. You can also add the test image to the Zip as well.

misbrit2128

Here are the files. I also included the mask images that are generated when the image is visible or not. 

YuriG

Thanks for the images and the macro. I tested it and found that threshold ranges (RGE file) are not loaded on an invisible image, so the count was executed with default threshold 128.
I recommend loading the options file instead (IQO), it includes threshold ranges and also all other parameters (filter ranges, count options, etc). It works properly on invisible images.

Here is the modified version of the macro.

    'process invisible image from the Batch
    'doc - loaded image document
    Public Sub LoopMacro(doc As Object)
        Dim Counting_3 = New SimpleScript
        Dim doc1 As IMcDocument, image0, image1 As McImage, doc2, varList1

        doc1=doc

        With Process.Filter.EnhancementCommands.Flatten(Counting_3)
        .BrightOnDark = False
        .FeatureWidth = 20
        .Run(doc1, image0)
        End With

        'Load measurement options that set Threshold 146 and all other parameters
        With Measure.Measurements.OptionsCommands.Open(Counting_3)
            .FileName = ThisApplication.Path(mcPathType.mcptConfigurationFiles) & "Mono146.iqo"
            .FilterIndex = 1
            .Run(doc1)
        End With


        With Measure.MeasurementsCommands.ExecuteCount(Counting_3)
        .Run(doc1)
        End With

        With Measure.Data.CollectorCommands.Collect(Counting_3)
        '.Image = .GetImage("Measure.Data.Collector.Collect")
        .Run(doc1)'New List({doc1}))
        End With

        With Measure.Measurements.CreateCommands.Mask(Counting_3)
            .Visible=False 'don't show the mask
            .Run(doc1, image1)'image1 is the mask
        End With

        'QuickSaveSelectedForPublication doesn't work with hidden images        
        'save mask directly
        Dim fName As String=doc1.FileName
        'use default QuickSaveForPublications path
        Dim outPath As String=ThisApplication.Settings("QuickSave", "Publication").Get("Path", ThisApplication.Path(mcPathType.mcptWritableImages))
        'generate mask name, save it to the same folder as the original image, but with _Mask suffix in JPG format
        Dim maskName As String=outPath & System.IO.Path.GetFileNameWithoutExtension(fName) & "_Mask" & ".jpg"
        image1.SaveAs(maskName)
        image1.Close'close the image
End Sub

I've also attached the IQO file (zipped).
Let me know if it works for you.

Regards,

Yuri

misbrit2128

Yuri,

Thank you for the help! Everything is working now. 

Brittany