Home Image-Pro Automation (Macros, Apps, Reports)

How can I reduce [managed memory usage] ?

Hi guys,
We developed a macro  for image batch analysis on IP-10 version 10.0.7.

The content of the macro is roughly [Open Image],[Apply Calibration] ,[Use 2D Filter], [Count], [Mask New Image], etc.

The images are all set to [visible=false], and after each image is counted, it will be closed.
Follow : https://forums.mediacy.com/discussion/709/batch-processing-limitations

At the beginning, it takes about 2 seconds to analyze an image, and after a while, it takes about 10 seconds for an image, and the [managed memory usage] cannot be reduced.

After testing, I found that the memory accumulate in [Apply Calibration], [Duplicate], [CreakMask], etc.
So I wrote a simple macro test.


[Apply Calibration]:


[Duplicate]:


After processing 14 images:



What should I add to the macro to reduce the usage rate of [managed memory usage]?
Attached is the simple test macro.

Thanks!

Best Answers

  • 2020-09-14-100842

    Kyle1996 --

    I have never understood the benefits of the SIMPLESCRIPT feature in IMAGE-PRO.

    The existence of SIMPLESCRIPT routines in MODULES and APPS has sometimes bitten me because they seem to want to run whenever an image is opened by IMAGE-PRO.

    Normally I rip the references required by SIMPLE SCRIPT and make the routines only active when I call them specifically.

    I do this by removing the

        As SimpleScript

    from the routine declaration and by changing the reference to the ROUTINE in the code (ex "_2D_FILTERS") to "NOTHING" within the CODE like

        With Process.Filter.MorphologicalCommands.Close(Nothing)

    To understand the difference, you might want to put a DEBUG.PRINT into the routines before you make the change to see when they are being triggered.

    I think you will find that a

        DEBUG.PRINT "Hello from _2D_FILTERS"
        DEBUG.PRINT ThisApplication.ActiveDocument.DisplayName

    after the PUBLIC FUNCTION statement shows that it is running more often than you expect.

    I hope this information is helpful.

    Please let us know.

    -- Matt
  • Answer ✓
    Kyle1996

    I checked your macro and it looks like a combination of running async background process with commands and scripts cause image leaks. (Image leaks could be detected by using a demo "Debugging" macro project and running CountImages macro from the Leaks group). 

    I modified your macros to handle invisible images without leaks.
    The ApplyImage macro:
        Public Function Applyimage() As SimpleScript
    
            If FirstLoad = True Or IsStop = True Then
                Exit Function
            End If
    
            'open invisible image
            visible_OG=ThisApplication.Images.Open(Load_ImageName,mcImageCreateFlags.mcicfNoAddToCollection Or mcImageCreateFlags.mcicfNotVisible)
    
            ApplyCalibration
            _2D_Filters
    
            visible_OG.Close 'close image
            visible_OG=Nothing
        End Function
    

    I've also disabled "New SimpleScript" lines to avoid accumulating references in script objects, so it will be equivalent to using "Nothing":

        Public Function ApplyCalibration() As SimpleScript
                '
            'ApplyCalibration = New SimpleScript
    
            Dim var1 = CalibrationSelected, spcal1
    
            If visible_OG Is Nothing And ThisApplication.ActiveImage Is Nothing Then'
                Exit Function
            End If
    
            If CalibrationSelectTrue =False Then
                Exit Function
            End If
    
                With Measure.Calibration.SpatialCommands.Define(ApplyCalibration)
                    .Run(var1, spcal1)
                End With
    
    
                With Measure.Calibration.SpatialCommands.Apply(ApplyCalibration)'this Code Pile up memory
                    .Run(visible_OG, spcal1)
                End With
    
        End Function
    

    and in _2D_Filters (also set global variable to Nothing in the end)

        Public Function _2D_Filters() As SimpleScript
            '_2D_Filters = New SimpleScript
    
    
            With Adjust.ImageCommands.Duplicate(_2D_Filters)
                .Visible =False
                .Name = "scratch_imp"
                .Run(visible_OG, Imp_image)
            End With
    
            With Process.Filter.MorphologicalCommands.Close(_2D_Filters)
                .Passes = 3
                .Shape = MediaCy.IQL.Filters.mcMorphoShape.mcms2x2Square
                .Run(Imp_image, Imp_image)
            End With
    
            'Imp_image.Clear
            Imp_image.Close
            Imp_image=Nothing
    
        End Function
    

    In my tests the batch macro doesn't produce leaks anymore.
    Please test it.

    Yuri

Answers

  • Hi guys,It work,but there was some problem.

    If I processing non-stop,memory usage will build up until the end.
    After all images processing, the [managed memory usage] will return to 0%.
    We need Processing approximately 2000 images,and this will be a problem.

    [Processing approximately 500 images]    >    [After process]


    ------------------------------------------------------------------------------------------------------------------------------------------------

    And I try this function to avoid [SpatialCommands.Apply] step.
    It work,it can reduce memory usage while processing.


    But there has a problem when I use the same method in [_2D_Filters].
    I use [Imp_image=visible_OG] to avoid [Duplicate] step,and close image.


    But now I can't close [visible_OG],it will increase memory usage until the end and return to 0%.


    How to reduce memory usage during processing or is there a better way to do this?
    Attached is the simple test macro.

    Thanks !
  • edited September 2020
    Kyle1996,

    The references are accumulated in command inputs, which get automatically released when the macro finishes. To release the references in the loop you have to add ThisApplication.ProcessAll, so no memory will be accumulated:
        Public Function Applyimage() As SimpleScript
            If FirstLoad = True Or IsStop = True Then
                Exit Function
            End If
            'open invisible image
            visible_OG=ThisApplication.Images.Open(Load_ImageName,mcImageCreateFlags.mcicfNoAddToCollection Or mcImageCreateFlags.mcicfNotVisible)
    
            ApplyCalibration
            _2D_Filters
    
            visible_OG.Close 'close image
            visible_OG=Nothing
           ThisApplication.ProcessAll
        End Function

    If you want to optimize your macro further, you can use low-level IQL functions instead of commands, though you have to look up the functions in the Automation Reference Help (also intellisense can guide you).
    For example, the command:
            With Process.Filter.MorphologicalCommands.Close(_2D_Filters)
                .Passes = 3
                .Shape = MediaCy.IQL.Filters.mcMorphoShape.mcms2x2Square
                .Run(Imp_image, Imp_image)
            End With
    
    can be replaced by 
            Imp_image.Morpho.Close(3,MediaCy.IQL.Filters.mcMorphoShape.mcms2x2Square)
    

    Yuri



  • Thanks!  That works.
Sign In or Register to comment.