how add 2d filters before using smart segmentation?

Louis · February 19

Hi Yuri,

Good day!

I'm writing to ask your help of how to add filters before using smart segmentation? I want to first process the image using some filter tech to better identity attributes. like convert grey of the image, -->morphological close 2x2 square 3 passes-->high gaussian to strengthen, canny to detect edge .... Is this function in the recipe option? how to preview the effect after adding filters? how to decide the order? and how to mark the objects and background after applying the filter? how to add back those counted objects back to the original image if count only can be done in filter processed image?

lots of questions.... thank you very much for your always strong support!

Best, Louis

Image: https://us.v-cdn.net/5020878/uploads/editor/kd/k8na230b3je6.png

YuriG · February 19

Hi Louis,

It can be implements as a part of smart segmentation recipe using Custom channels:

Image: https://us.v-cdn.net/5020878/uploads/editor/k2/lvve9opv0qk5.png

You can prototype the sequence of the filters using normal filters in Image-Pro applying them consecutively to a test image, and then create custom filters based on it using the list of predefined channel types (filters, arithmetical operations, stats,...):

Image: https://us.v-cdn.net/5020878/uploads/editor/6b/qhg350a5gxhe.png

If you define a sequence of filters, set the output of one filter as the input for another:

And after you built the chain, activate only the last filter in the Custom filter dropdown. The description of the recipe will show all the steps of pre-processing ("My Sobel Filter" is the last filter in this chain:

Image: https://us.v-cdn.net/5020878/uploads/editor/ka/t892wbphfbhk.png

You can add a couple of reference objects and then preview the image used for segmentation using the Create Image button:

Image: https://us.v-cdn.net/5020878/uploads/editor/g6/7czkiruccwil.png

Here is the example of the image created from smart recipe based on the Sobel filter:

Image: https://us.v-cdn.net/5020878/uploads/editor/6v/lhbn5wqup0v4.png

Regards,

Yuri

Louis · February 19

Hi Yuri,

Got it! i tried immediately, but it seems quite different of the output in the recipe option than 2D filter. for a simple example, firstly i want to turn the image to grey without scale. but it is scaled to best by default in the recipe and find no where to adjust it.

How to deal with this?

Thanks a lot
Louis

YuriG · February 19

Hi Louis,

The output image is not the same as you get directly from the Filters, as it's indeed uses scaling based on the reference objects. The goal of the output image is to check how the segmentation mask was created (and if there is a way to improve it). If you segment the output image at the level 100, it will give you the segmentation outlines matching the preview mask of smart segmentation. Also note, that the output image has floating point type, so you can set any range there.

Yuri

Louis · February 19

Hi YuriG,

not quite understand what you mean by "If you segment the output image at the level 100, it will give you the segmentation outlines matching the preview mask of smart segmentation. Also note, that the output image has floating point type, so you can set any range there."

and i found that the input of first recipe can only be the ones from first three channel(color, background, morphology..) and the output of those channel seems very difficult to predict as they are based on reference objects as you mentioned and has no place to fine tune. So it is not intuitive as those 2D filters. and sometime not easy to understand their functions. it is so different output compare with 2D filters. As a result, the formula i successfully worked out from 2D filter is difficult to rebuild in smart segment. It seems making the process more difficult for user rather than simply applying several filters then point out objects and background on filtered image and then label back on to original image.

did i used it in a wrong way?

BR,
Louis

YuriG · February 19

Hi Louis,

Smart segmentation generates all these filter channels in absolutely the same way as you would run 2D filters consecutively. The only difference is the thresholding in the end. With 2D filters you set threshold in the Threshold dialog, but in Smart segmentation threshold is calculated automatically based on the reference objects. If you use 1 reference object for Class1 and 1 reference object for background, you will see the values of these reference intensities in the Recipe window (158.5 and 10.5) and the threshold is calculated in the middle (158.5 + 10.5)/2=84.5 :

Image: https://us.v-cdn.net/5020878/uploads/editor/2a/c3iydso71ntb.png

So, if you want to get result exactly the same as after the filter operations and thresholding, set only 2 reference objects (1 class and 1 background) to achieve the average value the same as the manual threshold. Try experimenting on simple filters to validate it.

Regards,

Yuri

Louis · February 19

Hi Yuri,

threshold is only useful when come to the last step for counting, right? but for the many filters used before counting how to ensure it is the same as when doing in 2D filter? as for each step it is automatically adjusted according to objects and background.

For my problem, i want to identify those pores in below pictures. My plan is ->normal gray ->morphology close to get rid of hair ->morphological open to stregthen pore -> edge minus to enhance the pore then use smart segment to count based on filtered image then label and mark pore objects back to original image.

Two difficulties i met is 1. difficult to get the same result in smart segment as using 2D filters 2. have no idea of how the label and mark back to original image.

Could you kindly help to guide me? thanks a lot

Image: https://us.v-cdn.net/5020878/uploads/editor/p5/gm13xcg9pm2v.jpg

Louis

YuriG · February 19

Hi Louis,

Can you record a macro that includes all the filters and threshold level, which you would use to segment pores on that image and send it to me? I will try to convert your macro to a recipe.

Did you know that you can record a recipe and then edit it in the scripting workbench? Here is the macro that was recorded from the recipe in my first comment:

    Public Function SmartSeg() As SimpleScript
        SmartSeg = New SimpleScript
        Dim doc1

        With Application.DocumentCommands.Active(SmartSeg)
            .Run(doc1)
        End With

        With New MediaCy.Addins.LearningSegmentation.SetRecipeCommand(SmartSeg)
            .Options.AccumulatedClass = False
            .Options.ObjectSize = 15
            .Options.AutoWeight = True
            .Options.UseAllChannels = False
            .Options.NumberOfTopChannels = 2
            .Options.Classes = New System.Collections.Generic.List(Of MMClassDescr)
                .Options.Classes.Add(New MMClassDescr("Class 1",System.Drawing.Color.Blue))
            .Options.BackgroundClass = New MMClassDescr("Background",System.Drawing.Color.FromArgb(255,255,128))
            .ObjectClasses = New Integer() {1,0}
            .UserChannels = New UserRecipeChannels
                .UserChannels.Add(New UserChannel(MediaCy.Addins.LearningSegmentation.rcpCh.Custom1,"My Close 3x3","Process.Filter.Morphological.Close3D","Gray","Width='3',Height='3',Passes='2'"))
                .UserChannels.Add(New UserChannel(MediaCy.Addins.LearningSegmentation.rcpCh.Custom2,"My High-Pass 15x15","Process.Filter.Large.HighPassLarge","Custom1","Width='15',Height='15',Strength='3',Passes='1'"))
                .UserChannels.Add(New UserChannel(MediaCy.Addins.LearningSegmentation.rcpCh.Custom3,"My Sobel Filter ","Process.Filter.Edge.Sobel","Custom2",""))
            .RecipeChannels = New RecipeChannels
                .RecipeChannels.Add(New RecipeChannel(MediaCy.Addins.LearningSegmentation.rcpCh.Custom3, 1))
                    .RecipeChannels.Last.ReferenceObjects.Add(New RefObjStats(1, 101, 0))
                    .RecipeChannels.Last.ReferenceObjects.Add(New RefObjStats(1, 99, 0))
            .FilterInput = True
            .Run(doc1)
        End With

    End Function

With this script you can set the final threshold directly in the macro, then you apply the recipe to the image and can save it to IQO file.
The threshold in this example is defined by intensities of reference objects (101+99)/2=100

Image: https://us.v-cdn.net/5020878/uploads/editor/td/5wu10u34407v.png

So, please send me the macro and I will try converting it to a recipe.

Regards,

Yuri

Louis · February 20

Hi Yuri,

i copy the macro as below. These are the filters i used. But without count or segment. because i did know how to proper combine these filters before smart segment.

could you kindly help to see how it works

Thanks a lot
Louis

Public Function pores() As SimpleScript
        pores = New SimpleScript
        Dim image1, doc1, window1, image2, doc2, window2

        With Application.DocumentCommands.ActiveImage(pores)
            .Run(image1)
        End With

        With Process.Filter.MorphologicalCommands.Close(pores)
            .Passes = 2
            .Shape = MediaCy.IQL.Filters.mcMorphoShape.mcms3x3Cross
            .Run(image1, image1)
        End With

        With Application.DocumentCommands.Define(pores)
            .Run(image1, doc1)
        End With

        With Process.Filter.MorphologicalCommands.Open(pores)
            .Passes = 5
            .Shape = MediaCy.IQL.Filters.mcMorphoShape.mcms2x2Square
            .Run(doc1, image1)
        End With

        With Process.Filter.LargeCommands.EdgeMinusLarge(pores)
            .Passes = 14
            .Strength = 5
            .Width = 9
            .Height = 9
            .Run(doc1, image1)
        End With

        With Application.WindowCommands.Define(pores)
            .Run(doc1, window1)
        End With

        With View.ImageViewCommands.Options(pores)
            .View.Pan = 0
            .View.Scroll = 3349
            .View.Magnification = 0.168779015006199R
            .Run(window1)
        End With

        With Application.RibbonCommands.SelectRibbonTab(pores)
            .TabName = "Adjust"
            .Run()
        End With

        With Adjust.ImageCommands.Convert(pores)
            .Destination = Image.ConvertTypes.MonoFloat
            .ConvertOption = Image.ConvertOptions.Direct
            .ApplyBestFit = False
            .Visible = True
            .UseTiles = False
            .Run(doc1, image2)
        End With

        With Application.DocumentCommands.Activate(pores)
            .Run(image2, doc2)
        End With

        With Application.WindowCommands.Define(pores)
            .Run(doc2, window2)
        End With

        With View.ImageViewCommands.Options(pores)
            .View.Magnification = 0.300044982803705R
            .View.Scroll = 0
            .View.Pan = 0
            .Run(window2)
        End With

YuriG · February 20

Hi Louis,

I checked your macro and created a recipe that replicates that. Here is the macro that can set the recipe to the active image:

    Public Sub ApplyPoresRecipeToActiveImage()
        Dim doc1

        With Application.DocumentCommands.Active(Nothing)
            .Run(doc1)
        End With

        Dim threshold As Double = 10' set segmentation threshold level

        With New MediaCy.Addins.LearningSegmentation.SetRecipeCommand(Nothing)
            .Options.AccumulatedClass = False
            .Options.ObjectSize = 15
            .Options.AutoWeight = True
            .Options.UseAllChannels = False
            .Options.NumberOfTopChannels = 2
            .Options.Classes = New System.Collections.Generic.List(Of MMClassDescr)
                .Options.Classes.Add(New MMClassDescr("Pores", System.Drawing.Color.Blue))
            .Options.BackgroundClass = New MMClassDescr("Background",System.Drawing.Color.FromArgb(255,255,128))
            .ObjectClasses = New Integer() {1,0}
            .UserChannels = New UserRecipeChannels
                .UserChannels.Add(New UserChannel(MediaCy.Addins.LearningSegmentation.rcpCh.Custom1,"My Close","Process.Filter.Morphological.Close3D","Gray","Width='3',Height='3',Passes='2'"))
                .UserChannels.Add(New UserChannel(MediaCy.Addins.LearningSegmentation.rcpCh.Custom2,"My Open","Process.Filter.Morphological.Open3D","Custom1","Width='3',Height='3',Passes='3'"))
                .UserChannels.Add(New UserChannel(MediaCy.Addins.LearningSegmentation.rcpCh.Custom3,"My Edge Minus","Process.Filter.Large.EdgeMinusLarge","Custom2","Width='9',Height='9',Strength='5',Passes='14'"))
            .RecipeChannels = New RecipeChannels
                .RecipeChannels.Add(New RecipeChannel(MediaCy.Addins.LearningSegmentation.rcpCh.Custom3, 1))
                    .RecipeChannels.Last.ReferenceObjects.Add(New RefObjStats(1, threshold + 1, 0))
                    .RecipeChannels.Last.ReferenceObjects.Add(New RefObjStats(1, threshold - 1, 0))
            .FilterInput = True
            .Run(doc1)
        End With
    End Sub

I created a separate variable "threshold", which can be modified if necessary, the default value is 10.
You can also modify filter parameters, if necessary.

I've attached Pores.iqo file that contains the recipe, you can just open an image, load IQO file and click Count.

The result should look like this:

Image: https://us.v-cdn.net/5020878/uploads/editor/q6/ez3l68xur4n9.png

Regards,

Yuri

Louis · February 21

Hi Yuri,

Thank you much for your help.

How to load the IQO file. It seems no where to load in version 11.1.... Is there any guidance or video to teach how to write the scrip to use filters before smart segment. not familiar with MediaCy.Addins.LearningSegmentation namespace.

and one more question, i noticed that in the end of code you creat an object of RecipeChannels then add filtered channel custom3 to the collection and refer objects for machine learning. my question is how to add more reference objects and background in the filtered channel before counting. in this case, you just simply set a threshold.

Thanks again, hopefully the software can add more GUI to guide use filters before smart segment or even deep learning.

Best,
Louis

Louis · February 21

one more question when you add the first channel how you make it not scale while use gray as input.

.UserChannels.Add(New UserChannel(MediaCy.Addins.LearningSegmentation.rcpCh.Custom1,"My Close","Process.Filter.Morphological.Close3D","Gray","Width='3',Height='3',Passes='2'"))

YuriG · February 21

Hi Louis,

An IQO file is the measurement options file, which also includes smart segmentation recipe. You can load IQO files from the Count/Size ribbon:

Image: https://us.v-cdn.net/5020878/uploads/editor/s3/pi6nmvocesak.png

Regarding the filters chain building:
The custom channel is defined with the following syntax: ("Name","Command","Input","Parameters").
Note, that you can use ANY image-Pro filter commands in custom filters! (e.g. Morpho, Large, ...)
When you build a sequence of filters it's important that the next filter uses the previous filter ID (e.g. Cusom1) as the "Input". And you use ONLY THE LAST filter in the chain as the recipe channel (Custom3):

Image: https://us.v-cdn.net/5020878/uploads/editor/2l/mbozc2475xcf.png

When you add new custom channels, they are not scaled (scaling is only applied when you use the Create Image button, but you should not worry about it when you execute the recipe) , so you can use the same threshold level on the recipe as you would do on the normal filtered image.

Let me know if you have more questions.

Yuri

Louis · February 26

Hi Yuri,

Thank you very much for your detailed explanation. Is there any general guide for how to make the macro which can set the recipe to active image. how to add ranges before count in macro. and then how to turn it into iqo.

I tried to modify in iqo files but these xml files are not easy to handle.

BR,
Louis

YuriG · February 26

Hi Louis,

You cannot modify IQO file. Here is what you can do to modify the recipe:

1. Modify the macro, which I sent, to change recipe parameters or add new channels.
2. Run the macro on a test image: it will apply that recipe to the image.
3. Save measurement options to IQO file - it will include the new recipe.

And, of cause, you can modify the recipe interactively, using the Smart segmentation panel.

Yuri

Louis · February 27

Thanks Yuri,

How to add range and count into the macro, so i can use it in batch process

YuriG · February 27

Hi Louis,

If you want to add a range and count, then just activate macro recording, set range in the Ranges dialog, close it, and click the Count button, stop recording. It will create a new macro with the commands you need, like this:

    Public Function NewMacro14() As SimpleScript
        NewMacro14 = New SimpleScript
        Dim doc1

        With Application.RibbonCommands.SelectRibbonTab(NewMacro14)
            .TabName = "CountSize"
            .Run()
        End With

        With Application.DocumentCommands.Active(NewMacro14)
            .Run(doc1)
        End With

        With Measure.MeasurementsCommands.Options(NewMacro14)
            .Segmentation.FilterRanges = New System.Collections.Generic.List(Of MeasFilterEntry)
                .Segmentation.FilterRanges.Add( New MeasFilterEntry(eMeasures.RgnArea,30R,1E+308R))
                .Segmentation.FilterRanges.Add( New MeasFilterEntry(eMeasures.RgnAreaUncalib,0R,1E+308R))
            .Run(doc1)
        End With

        With Measure.MeasurementsCommands.ApplyFilters(NewMacro14)
            .Run(doc1)
        End With

        With Measure.MeasurementsCommands.ExecuteCount(NewMacro14)
            .Run(doc1)
        End With

    End Function

Another solution, preferable, is to use IQO file to loads everything: recipe, ranges, class colors..., and then execute Count. You must create recipe, set ranges and other segmentation options, and then save IQO file under the name you want, e.g. CountPores.iqo. Also, if you want to process multiple images in the folder in one go, you must collect the results of every count to Data Collector.

Typically the Batch operations use 3 macros: Run Before (setup DC), Loop On (count and collect), Run After (Show DC):

Here is the example of these 3 macros:

    Public Function Pores_RunBefore() As SimpleScript
        Pores_RunBefore = New SimpleScript

        Measure.Data.CollectorCommands.Clear(Pores_RunBefore).Run()

        With Measure.Data.Collector.MeasurementsCommands.Open(Pores_RunBefore)
            .FileName = ThisApplication.Path(mcPathType.mcptConfigurationFiles) & "PoresDC.adoxsd"
            .Run()
        End With

    End Function


    Public Function Pores_LoopOn() As SimpleScript
        Pores_LoopOn = New SimpleScript
        Dim doc1

        With Application.DocumentCommands.Active(Pores_LoopOn)
            .Run(doc1)
        End With

        With Measure.Measurements.OptionsCommands.Open(Pores_LoopOn)
            .FileName = ThisApplication.Path(mcPathType.mcptConfigurationFiles) & "PoresCount.iqo"
            .FilterIndex = 1
            .Run(doc1)
        End With

        With Measure.MeasurementsCommands.ExecuteCount(Pores_LoopOn)
            .Run(doc1)
        End With

        With Measure.Data.CollectorCommands.Collect(Pores_LoopOn)
            .Run(New List({doc1}))
        End With

    End Function


    Public Function Pores_RunAfter() As SimpleScript
        Pores_RunAfter = New SimpleScript

        With Measure.Data.Collector.Gadgets.DataTable(Pores_RunAfter)
            .CheckState = MediaCy.IQL.Application.McCommand.mcCheckState.Checked
            .Run()
        End With

    End Function

Then you just set these macros to the Batch dialog, select folder with images and click the Start button. The data from all images will be shown in the Data Collector table:

Image: https://us.v-cdn.net/5020878/uploads/editor/aw/hj0ohdsdh6sy.png

I've also attached the macro project (PosesBatch.ipx) that includes all these macros. The ipx file also installs PoresCount.iqo and PoresDC.adoxsd, so you can just load the project, set macros as on my screenshot above and run it.

Regards,

Yuri

Louis · March 7

Hi Yuri,

How to run the macro you send me? i copied it into the workbench and click run. it seems nothing happened..

Could you kindly help to check where i'm wrong

Thanks a lot

Image: https://us.v-cdn.net/5020878/uploads/editor/sf/43aoe3jeo0gc.png

YuriG · March 7

Hi Louis,

The macro project I sent, supposed to be used from the Batch panel.
Here are the steps:

1. Load PoresBatch.ipx to Image-Pro (Apps tab - Open Project)
2. Open the Batch panel and set all the parameters (Run Before, Loop On,...) as on my screenshot in the last comment.
3. Select folder with images.
4. Click Start - the results will be shown in Data Collector.

Here is the help topic about Batch processing: https://help.mediacy.com/Image-Pro/11.2/en/Content/B_User's%20Guide/Automating%20Processes/Batch_Processing_Images_Using_Macros.htm?Highlight=Batch

Let me know if you have more questions,

Regards,

Yuri

Louis · March 7

Hi Yuri,

Thank you very much. i changed the sub into function in macro code and it works.

More questions need your help

 .Options.ObjectSize = 15         //how to figure proper size in machinelearning when there is not a process to 
                                  select objects and background

 Dim threshold As Double = 10' //set segmentation threshold level, what is this segmentation threshold mean? 
                                 in below code it is 10 -/+ 1 for objects


.UserChannels.Add(New UserChannel(MediaCy.Addins.LearningSegmentation.rcpCh.Custom1,"My Close",
"Process.Filter.Morphological.Close3D","Gray","Width='3',Height='3',Passes='2'")
.UserChannels.Add(New UserChannel(MediaCy.Addins.LearningSegmentation.rcpCh.Custom2,"My Open",
"Process.Filter.Morphological.Open3D","Custom1","Width='3',Height='3',Passes='3'"))
.UserChannels.Add(New UserChannel(MediaCy.Addins.LearningSegmentation.rcpCh.Custom3,
"My Edge Minus","Process.Filter.Large.EdgeMinusLarge","Custom2","Width='9',Height='9',Strength='5',Passes='14'"))

// how can i write those parameters if i need to use other filters. it do not pop out methods with explanation 
like other IDE. is there any where to look up other filters like dilate, erode..etc

Thank you as always

Best,
Louis

YuriG · March 7

Hi Louis,

ObjectSize is applied to the default Background and Filter channels and you should do some experimenting in the Smart panel to see, which size is the best. But in your recipe, you use only Custom channels, so ObjectSize is not used.
threshold is set based on your macro, you can adjust it, if needed.

You can find the list of filter commands in the Automation help, e.g. https://help.mediacy.com/Image-Pro/Automation/11/html/N_MediaCy_Commands_Filters_Morphological.htm

But, if you find it difficult to implement, I would recommend creating Smart segmentation recipes from the UI.

Regards,

Yuri

Louis · March 8

Thanks Yuri,

What's difference between close and close3D, i see almost every method has a 3D version

Louis · March 8

Hi Yuri,

I tried your code it is great. and how can i add certain ROI before the batch process. and sometime the ROI may need adjust by person, because position of each person to analysize will be slight different.

Best,
Louis

YuriG · March 10

Hi Louis,

The difference between Close and Close3D is in parameters, the 3D filters can take any kernel size, but 2D use pre-defined kernels. So, the Smart segmentation uses 3D filters to have for flexibility.

You can add any command in the Loop On macro, just activate recording and do what you want.

The macro may look like this:

    Public Function InteractiveROI() As SimpleScript
        InteractiveROI = New SimpleScript
        Dim image1

        With Application.RibbonCommands.SelectRibbonTab(InteractiveROI)
            .TabName = "Select"
            .Run()
        End With

        With Application.DocumentCommands.ActiveImage(InteractiveROI)
            .Run(image1)
        End With

        With [Select].RoiCommands.Add(InteractiveROI)
            .ROIType = Features.ROI.ROITypes.Polygon
            .Points = New System.Collections.Generic.List(Of System.Drawing.PointF)
            .Points.Add(New System.Drawing.PointF(109F,203F))
            .Points.Add(New System.Drawing.PointF(242F,19F))
            .Points.Add(New System.Drawing.PointF(374F,202F))
            .Prompt = "Draw a ROI around cells"
            .Interactive = True
            .Run(image1)
        End With

    End Function

Yuri

how add 2d filters before using smart segmentation?

Answers