how add 2d filters before using smart segmentation?

Louis

Hi Yuri,

Good day!

I'm writing to ask your help of how to add filters before using smart segmentation? I want to first process the image using some filter tech to better identity attributes. like convert grey of the image, -->morphological close 2x2 square 3 passes-->high gaussian to strengthen, canny to detect edge ....   Is this function in the recipe option? how to preview the effect after adding filters? how to decide the order? and how to mark the objects and background after applying the filter? how to add back those counted objects back to the original image if count only can be done in filter processed image? 

lots of questions....    thank you very much for your always strong support!



Best, Louis

YuriG

Hi Louis,

It can be implements as a part of smart segmentation recipe using Custom channels:


You can prototype the sequence of the filters using normal filters in Image-Pro applying them consecutively to a test image, and then create custom filters based on it using the list of predefined channel types (filters, arithmetical operations, stats,...):


If you define a sequence of filters, set the output of one filter as the input for another:


And after you built the chain, activate only the last filter in the Custom filter dropdown. The description of the recipe will show all the steps of pre-processing ("My Sobel Filter" is the last filter in this chain:


You can add a couple of reference objects and then preview the image used for segmentation using the Create Image button:


Here is the example of the image created from smart recipe based on the Sobel filter:


Regards,


Yuri

Louis

Hi Yuri,

Got it!  i tried immediately, but it seems quite different of the output in the recipe option than 2D filter. for a simple example, firstly i want to turn the image to grey without scale. but it is scaled to best by default in the recipe and find no where to adjust it.

How to deal with this?

Thanks a lot
Louis

YuriG

Hi Louis,

The output image is not the same as you get directly from the Filters, as it's indeed uses scaling based on the reference objects.  The goal of the output image is to check how the segmentation mask was created (and if there is a way to improve it). If you segment the output image at the level 100, it will give you the segmentation outlines matching the preview mask of smart segmentation. Also note, that the output image has floating point type, so you can set any range there.

Yuri

Louis

Hi YuriG,

not quite understand what you mean by "If you segment the output image at the level 100, it will give you the segmentation outlines matching the preview mask of smart segmentation. Also note, that the output image has floating point type, so you can set any range there."

and i found that the input of first recipe can only be the ones from first three channel(color, background, morphology..) and the output of those channel seems very difficult to predict as they are based on reference objects as you mentioned and has no place to fine tune. So it is not intuitive as those 2D filters. and sometime not easy to understand their functions. it is so different output compare with 2D filters. As a result, the formula i successfully worked out from 2D filter is difficult to rebuild in smart segment. It seems making the process more difficult for user rather than simply applying several filters then point out objects and background on filtered image and then label back on to original image.

did i used it in a wrong way?

BR,
Louis

YuriG

Hi Louis,

Smart segmentation generates all these filter channels in absolutely the same way as you would run 2D filters consecutively. The only difference is the thresholding in the end. With 2D filters you set threshold in the Threshold dialog, but in Smart segmentation threshold is calculated automatically based on the reference objects. If you use 1 reference object for Class1 and 1 reference object for background, you will see the values of these reference intensities in the Recipe window (158.5 and 10.5) and the threshold is calculated in the middle (158.5 + 10.5)/2=84.5 :


So, if you want to get result exactly the same as after the filter operations and thresholding, set only 2 reference objects (1 class and 1 background) to achieve the average value the same as the manual threshold. Try experimenting on simple filters to validate it.

Regards,

Yuri

Louis

Hi Yuri,

threshold is only useful when come to the last step for counting, right? but for the many filters used before counting how to ensure it is the same as when doing in 2D filter? as for each step it is automatically adjusted according to objects and background.

For my problem, i want to identify those pores in below pictures. My plan is ->normal gray ->morphology close to get rid of hair ->morphological open to stregthen pore -> edge minus to enhance the pore then use smart segment to count based on filtered image then label and mark pore objects back to original image.  

Two difficulties i met is 1. difficult to get the same result in smart segment as using 2D filters 2. have no idea of how the label and mark back to original image.

Could you kindly help to guide me? thanks a lot


Louis

YuriG

Hi Louis,

Can you record a macro that includes all the filters and threshold level, which you would use to segment pores on that image and send it to me? I will try to convert your macro to a recipe.

Did you know that you can record a recipe and then edit it in the scripting workbench? Here is the macro that was recorded from the recipe in my first comment:

    Public Function SmartSeg() As SimpleScript
        SmartSeg = New SimpleScript
        Dim doc1

        With Application.DocumentCommands.Active(SmartSeg)
            .Run(doc1)
        End With

        With New MediaCy.Addins.LearningSegmentation.SetRecipeCommand(SmartSeg)
            .Options.AccumulatedClass = False
            .Options.ObjectSize = 15
            .Options.AutoWeight = True
            .Options.UseAllChannels = False
            .Options.NumberOfTopChannels = 2
            .Options.Classes = New System.Collections.Generic.List(Of MMClassDescr)
                .Options.Classes.Add(New MMClassDescr("Class 1",System.Drawing.Color.Blue))
            .Options.BackgroundClass = New MMClassDescr("Background",System.Drawing.Color.FromArgb(255,255,128))
            .ObjectClasses = New Integer() {1,0}
            .UserChannels = New UserRecipeChannels
                .UserChannels.Add(New UserChannel(MediaCy.Addins.LearningSegmentation.rcpCh.Custom1,"My Close 3x3","Process.Filter.Morphological.Close3D","Gray","Width='3',Height='3',Passes='2'"))
                .UserChannels.Add(New UserChannel(MediaCy.Addins.LearningSegmentation.rcpCh.Custom2,"My High-Pass 15x15","Process.Filter.Large.HighPassLarge","Custom1","Width='15',Height='15',Strength='3',Passes='1'"))
                .UserChannels.Add(New UserChannel(MediaCy.Addins.LearningSegmentation.rcpCh.Custom3,"My Sobel Filter ","Process.Filter.Edge.Sobel","Custom2",""))
            .RecipeChannels = New RecipeChannels
                .RecipeChannels.Add(New RecipeChannel(MediaCy.Addins.LearningSegmentation.rcpCh.Custom3, 1))
                    .RecipeChannels.Last.ReferenceObjects.Add(New RefObjStats(1, 101, 0))
                    .RecipeChannels.Last.ReferenceObjects.Add(New RefObjStats(1, 99, 0))
            .FilterInput = True
            .Run(doc1)
        End With

    End Function

With this script you can set the final threshold directly in the macro, then you apply the recipe to the image and can save it to IQO file.
The threshold in this example is defined by intensities of reference objects (101+99)/2=100


So, please send me the macro and I will try converting it to a recipe.

Regards,

Yuri

Louis

Hi Yuri,

i copy the macro as below. These are the filters i used. But without count or segment. because i did know how to proper combine these filters before smart segment.

could you kindly help to see how it works

Thanks a lot
Louis

Public Function pores() As SimpleScript
        pores = New SimpleScript
        Dim image1, doc1, window1, image2, doc2, window2

        With Application.DocumentCommands.ActiveImage(pores)
            .Run(image1)
        End With

        With Process.Filter.MorphologicalCommands.Close(pores)
            .Passes = 2
            .Shape = MediaCy.IQL.Filters.mcMorphoShape.mcms3x3Cross
            .Run(image1, image1)
        End With

        With Application.DocumentCommands.Define(pores)
            .Run(image1, doc1)
        End With

        With Process.Filter.MorphologicalCommands.Open(pores)
            .Passes = 5
            .Shape = MediaCy.IQL.Filters.mcMorphoShape.mcms2x2Square
            .Run(doc1, image1)
        End With

        With Process.Filter.LargeCommands.EdgeMinusLarge(pores)
            .Passes = 14
            .Strength = 5
            .Width = 9
            .Height = 9
            .Run(doc1, image1)
        End With

        With Application.WindowCommands.Define(pores)
            .Run(doc1, window1)
        End With

        With View.ImageViewCommands.Options(pores)
            .View.Pan = 0
            .View.Scroll = 3349
            .View.Magnification = 0.168779015006199R
            .Run(window1)
        End With

        With Application.RibbonCommands.SelectRibbonTab(pores)
            .TabName = "Adjust"
            .Run()
        End With

        With Adjust.ImageCommands.Convert(pores)
            .Destination = Image.ConvertTypes.MonoFloat
            .ConvertOption = Image.ConvertOptions.Direct
            .ApplyBestFit = False
            .Visible = True
            .UseTiles = False
            .Run(doc1, image2)
        End With

        With Application.DocumentCommands.Activate(pores)
            .Run(image2, doc2)
        End With

        With Application.WindowCommands.Define(pores)
            .Run(doc2, window2)
        End With

        With View.ImageViewCommands.Options(pores)
            .View.Magnification = 0.300044982803705R
            .View.Scroll = 0
            .View.Pan = 0
            .Run(window2)
        End With

YuriG

Hi Louis,

I checked your macro and created a recipe that replicates that. Here is the macro that can set the recipe to the active image:

    Public Sub ApplyPoresRecipeToActiveImage()
        Dim doc1

        With Application.DocumentCommands.Active(Nothing)
            .Run(doc1)
        End With

        Dim threshold As Double = 10' set segmentation threshold level

        With New MediaCy.Addins.LearningSegmentation.SetRecipeCommand(Nothing)
            .Options.AccumulatedClass = False
            .Options.ObjectSize = 15
            .Options.AutoWeight = True
            .Options.UseAllChannels = False
            .Options.NumberOfTopChannels = 2
            .Options.Classes = New System.Collections.Generic.List(Of MMClassDescr)
                .Options.Classes.Add(New MMClassDescr("Pores", System.Drawing.Color.Blue))
            .Options.BackgroundClass = New MMClassDescr("Background",System.Drawing.Color.FromArgb(255,255,128))
            .ObjectClasses = New Integer() {1,0}
            .UserChannels = New UserRecipeChannels
                .UserChannels.Add(New UserChannel(MediaCy.Addins.LearningSegmentation.rcpCh.Custom1,"My Close","Process.Filter.Morphological.Close3D","Gray","Width='3',Height='3',Passes='2'"))
                .UserChannels.Add(New UserChannel(MediaCy.Addins.LearningSegmentation.rcpCh.Custom2,"My Open","Process.Filter.Morphological.Open3D","Custom1","Width='3',Height='3',Passes='3'"))
                .UserChannels.Add(New UserChannel(MediaCy.Addins.LearningSegmentation.rcpCh.Custom3,"My Edge Minus","Process.Filter.Large.EdgeMinusLarge","Custom2","Width='9',Height='9',Strength='5',Passes='14'"))
            .RecipeChannels = New RecipeChannels
                .RecipeChannels.Add(New RecipeChannel(MediaCy.Addins.LearningSegmentation.rcpCh.Custom3, 1))
                    .RecipeChannels.Last.ReferenceObjects.Add(New RefObjStats(1, threshold + 1, 0))
                    .RecipeChannels.Last.ReferenceObjects.Add(New RefObjStats(1, threshold - 1, 0))
            .FilterInput = True
            .Run(doc1)
        End With
    End Sub

I created a separate variable "threshold", which can be modified if necessary, the default value is 10.
You can also modify filter parameters, if necessary.

I've attached Pores.iqo file that contains the recipe, you can just open an image, load IQO file and click Count.

The result should look like this:



Regards,

Yuri

Louis

Hi Yuri,

Thank you much for your help.

How to load the IQO file. It seems no where to load in version 11.1....  Is there any guidance or video to teach how to write the scrip to use filters before smart segment. not familiar with MediaCy.Addins.LearningSegmentation namespace.

and one more question, i noticed that in the end of code you creat an object of RecipeChannels then add filtered channel custom3 to the collection and refer objects for machine learning. my question is how to add more reference objects and background in the filtered channel before counting. in this case, you just simply set a threshold.

Thanks again, hopefully the software can add more GUI to guide use filters before smart segment or even deep learning.

Best,
Louis

Louis

one more question when you add the first channel how you make it not scale while use gray as input.

.UserChannels.Add(New UserChannel(MediaCy.Addins.LearningSegmentation.rcpCh.Custom1,"My Close","Process.Filter.Morphological.Close3D","Gray","Width='3',Height='3',Passes='2'"))

YuriG

Hi Louis,

An IQO file is the measurement options file, which also includes smart segmentation recipe. You can load IQO files from the Count/Size ribbon:


Regarding the filters chain building:
The custom channel is defined with the following syntax: ("Name","Command","Input","Parameters").
Note, that you can use ANY image-Pro filter commands in custom filters! (e.g. Morpho, Large, ...) 
When you build a sequence of filters it's important that the next filter uses the previous filter ID (e.g. Cusom1) as  the "Input". And you use ONLY THE LAST filter in the chain as the recipe channel (Custom3):



When you add new custom channels, they are not scaled  (scaling is only applied when you use the Create Image button, but you should not worry about it when you execute the recipe) , so you can use the same threshold level on the recipe as you would do on the normal filtered image.

Let me know if you have more questions.

Yuri