Create a manual segmentation tool
CamiTK is a great tool that brings you the visualization power of VTK, image processing power of ITK and GUI handling capabilities of Qt letting you only concentrate on your work. There are many ways to go about implementing what you need in CamiTK. In this tutorial, you will be presented with on such way: development in increments.
For this tutorial, let’s create a segmentation tool that lets you manually segment an image by drawing along the contours of the objects of interest similar to the ITK-SNAP tool1 using the polygon mode.
First, let’s define the required functionality of this tool.
- Draw contours in a given orthogonal viewer (axial, coronal or sagittal) of CamiTK-imp (as different objects can be easily seen in different views)
- Copy an already drawn contour and paste it on another slice (as modifying a contour may be faster than drawing it from scratch)
- Visually see the segmentation in all four default viewers (axial, coronal, sagittal and 3D) of CamiTK-imp
- Save the segmentation as an image
- Make the tool less error prone
And let’s do the implementation in increments.
This tutorial was created and tested on CamiTK 4.0.4 using VTK 6.3 and QT 5.8. Depending on your configuration there may be slight changes or issues. |
Step 01: Use the wizard to create a CEP¶
We will use this tool to perform some operations on images. Hence, we only need to create a CamiTK Action that acts on CamiTK ImageComponents. But following the good practices of CamiTK let us create a CEP called Segmentor. The Segmentor CEP will have only one action also called Segmentor.
Let’s create this CEP using the CamiTK-Wizard. While creating the action let’s add two parameters to it.
- Contour Color: the color of the contour
- Contour Width: the line width of the contour
Build the CEP and add the Segmentor action to CamiTK-imp. Load an image to CamiTK-imp and you will find the Segmentor action under Tutorial action family. When you apply the action, it will create a copy of the current image by executing the automatically generated code.
Step 02: Remove the unnecessary code snippets and create the necessary GUI components¶
Usually, an action is executed when the user clicks the Apply button. This behavior is suitable when you perform only one operation on the data component. Since we are planning to use this tool to perform multiple operations on an image (draw contours, paste the last drawn contour, create segmented image, etc.), using only the Apply button is not adequate. Consequently, we should disable the default Apply button and provide our own GUI widgets to handle our needs.
We can disable the default Apply button in the following manner:
Let us determine the additional GUI widgets that we require for this tool.
- Radio buttons to specify on which viewer the contours are drawn
- Single slice related widgets
- Push button to complete the current contour
- Push button to reset the current contour
- Push button to accept the current drawn contour as the final contour
- Push button to paste the last drawn contour on the selected slice
- Whole image related widgets
- Reset everything
- Save the segmentation as an image
The above mentioned widgets will be created and added to a QFrame.
The default widget created by the Action will also be added to this
QFrame. Ultimately, this QFrame will be returned by the overloaded
getWidget( ) method of the action.
Finally, let’s remove the content of the
apply( ) method and delete the
process( ) method given by default.
When you load the action and click on the provided buttons nothing executes as we have not specified what to do when the user interacts with those widgets.
Step 03: Draw contours in a given orthogonal viewer (axial, coronal or sagittal) of CamiTK-imp¶
As the viewer on which the contours are drawn is going to be used throughout the implementation, we will create an ENum for the viewers. We will update the currently selected viewer depending on the radio button selected in the GUI. We could listen to the
Clicked( ) signal of the radio button and set the currently selected viewer accordingly.
In order to do this, we need to access the GUI widgets. Hence, we should set the objects names of the three radio buttons using the
setObjectName( ) method. Since we are going to use the signals and slots mechanism, we have to include the
Q_OBJECT macro in the class header file.
In order to draw contours on a viewer we can use the
vtkContourWidget2. We will create a member variable that will hold an instance of a
vtkContourWidget. The contour should be drawn on the selected slice of
the selected viewer. In order to calculate the plane where the contour
should be drawn, we require the information of the ImageComponent on
which the contours are drawn. We will use another member variable to
hold the currently selected ImageComponent call it
currentImageComp. We should
QWidget of the action is
requested. If the selected ImageComponent is different from the
previously selected one, then we have to handle the changes. We will see
how this is done at a later step. For the moment, let’s assume that the
actions will be carried out on the same ImageComponent.
We should draw on only one slice at a time. Hence, every time we want to
draw contours on a different viewer or a different slice of the same
viewer we have to reset the contours. Let’s define the
resetCurrentContourWidget( ) method
that handles the above mentioned requirements. And let’s setup this
method to be executed when the Reset button is clicked and also when
the selected viewer is changed.
Once the action is loaded, you will be able to draw contours on any of the slices of the image.
Step 04: Update the color and width of the contour and implement automatic contour completion¶
Try changing the color or width of the contour. You will observe that
nothing happens although we update the color and width of the contour
resetCurrentContourWidget( ) method. That is because the values of properties of an action
are updated only when the Apply button is clicked. But we have
disabled the Apply button of the action previously. In order to
correct this behavior all we need to do is to set the
setAutoUpdateProperty( ) to true of
ActionWidget when we generate
QWidget of the action.
vtkContourWidget comes with a
handy method to complete the currently drawn contour. In order to have a
closed loop we have to make sure at least two points are already drawn
when completing the contour. Let’s implement that in a method called
completeCurrentContour( ) and set
it up with the
clicked( ) signal of
the Complete button.
Now you will be able to complete the contour drawn.
Step 05: Build the segmentation image and show the region inside the contour in the orthogonal viewers¶
This is the heart of our tool and there are numerous ways of tackling this problem. Our solution is inspired by the approach used in ITK-SNAP.
First we are going to create an image that will hold information about
the development of the segmentation. Let’s define a member variable to
hold this image called
segmentedImage. The first time the
currentImageComp is assigned, we create
segmentedImage and set every voxel
of it to black. Every time a new region is added this image will be
updated to reflect the changes. In other words,
segmentedImage will work as a mask
to represent the segmentation.
Now that we have our
we can move onto showing the region inside the contour on the respective
orthogonal viewers. It is done in two steps.
- Find the voxels that correspond to the region inside the contour and
set those voxels of
segmentedImageas white (done by
- Find the intersection between the white voxels of
segmentedImageand the currently displayed slice of the
currentImageCompand draw an overlay on each orthogonal viewer (done by
First step will be implemented in the
updateSegmentedImage( ) method. The heart of this step is the
vtkPolyDataToImageStencil3 filter that converts the PolyData
into an image stencil. The PolyData from the
contourWidget will first be
triangulated and then extruded using the
vtkLinearExtrusionFilter filters respectively. With contours,
works if the image planes are aligned with the axial planes (see 4).
Consequently, if the contours are drawn on the sagittal or coronal
viewers we have to transform the relevant plane to be along the axial
plane. For the sagittal viewer the original x,y,z coordinates will
become y,z,x and for the coronal viewer the original x,y,z
coordinates will become z,x,y. Once the PolyData is transformed into
an image stencil, it will be cut using another temporary while image in
order to find the corresponding selected voxels from the original region
inside the contour. Finally, these voxels will be marked as white in the
segmentedImage. In order to achieve
this, we do not have to check the whole
segmentedImage but only the corresponding slice along the
In the second step we will implement the
drawOverlaysOn2DViewers( ) method. We will have to interact with
all 2D orthogonal viewers during this step. In order to do that let’s
create an actor and a mapper for each orthogonal viewer and add that to
RendererWidget or the
corresponding viewer. We do not have to do this every time the
drawOverlaysOn2DViewers( ) method
is called. Instead we will do this once when the
QWidget of the action is created
and make the actors and mappers available as members of the class. Since
vtkImageData is a structured point
dataset, we can use
vtkImageDataGeometryFilter to extract a single slice from the
segmentedImage. Then, we can use
vtkThresholdPoints to retrieve only the points that are white.
Finally, we can give these white points as the input for the mapper of
the corresponding viewer which in turn will take care of drawing the
These steps will be executed with the
clicked( ) signal of the Accept button. Let’s call the
Now when you click the Accept button the region inside the contour will be highlighted and the corresponding regions of other viewers will also be highlighted.
Step 06: Auto update the segmented regions with slice sliders¶
Draw contours on more than one slice and you will observe that the previously drawn contours become invisible when you move back to the previous slice.
That is normal as we have to update the 2D overlay when the slices are
changed. We can do that by listening to the
selectionChanged( ) signal of the 2D viewers. We only have to
update the corresponding 2D viewer. Let’s name the corresponding slot as
method. Since repeating the same lines of code is not a good coding
practice, we will modify the
drawOverlaysOn2DViewers( ) method to handle this situation as
Now when you change the viewer sliders the segmented regions will be correctly updated.
Step 07: Show the segmented regions in the 3D viewer¶
We will create another member variable to hold the segmented mesh and
segmentedMesh. As we already
have an updated
is very easy to get a surface mesh of the segmented regions using
vtkMarchingCubes filter5. This
behavior is implemented in the
updateSegmentedMesh( ) method.
Now when you add a new region to the segmentation, it will immediately be shown in the 3D viewer.
Step 08: Correcting for any arbitrary positioning in the world coordinates¶
After opening an image in
use the Edit Frame action under Frame to give an arbitrary
transform to the image in the world coordinates. And then start drawing
contours on the image. You will observe that the 3D mesh drawn by our
tool is not placed correctly.
In order to correct this mishap we need to transform all
PolyData belonging to the
segmentedMesh from the image to the
world coordinates. The
) method is modified accordingly.
Now even if you draw on an image which is subjected to an arbitrary transformation the segmentation mesh will be correctly shown in the 3D viewer.
Step 09: Paste the last contour on another slice¶
In order to paste the last contour on another slice, all we need to do
is to calculate the translation of the last drawn contour along the
corresponding axis and then transform the
PolyData of the contour to the new location. We also need to
PolyData as otherwise too
many points will be generated as the use of this function increases.
Then, we have to reset the
contourWidget at the new location but this time we should initialize it with
PolyData from the previous
contour. This functionality is implemented in the
pasteLastContour( ) method and it
will be triggered when the
clicked( ) signal is emitted from the Paste Last button.
Similar to step 06, we will refrain from copying the same lines of code
to reset the contour. Instead, lets modify the already implemented
resetCurrentContourWidget( ) method
and bring the common commands to a common method.
Now you can use Paste Last button to paste the last drawn contour on another image.
Step 10: Implement Reset All and Save As Image methods¶
In order to reset everything all we need to do is reset the
segmentedImage and then update 3D
viewer and 2D viewers. Let’s implement this behavior in a method called
resetEverything( ) and set this to
be executed when
clicked( ) signal
of Reset button is emitted.
In order to save the segmented image, we create a new
ImageComponent and save is using
the action Save. This behavior is implemented in the
saveSegmentedImage( ) method and is
clicked( ) signal of
Save As Image button is emitted.
Now the Reset and Save As Image button are functional.
Step 11: Make the tool less error prone¶
All the desired functionalities of the tool are implemented. However, since we can click any button without any restrictions, certain click combinations will lead the tool to crash. (e.g. Clicking Paste Last when you haven’t drawn anything leads the tool to crash.)
If we could restrict the access to Accept and Paste Last buttons we can make the tool more robust. Accept button should be enabled when:
- the contour is closed
- when the last contour is pasted on another slice
Paste Last button should be enabled only when Accept button is clicked. All other times both buttons should be disabled.
One way of completing the contour is by placing the last point of the
contour on the first point of it. We can capture this event if we listen
EndInteractionEvent of the
vtkContourWidget. The easiest way to
do this is by making the
class a subclass of
class. When we do that, we have to overload the
Execute( ) method of the
vtkCommand class in order to
Segmentor class. Let’s
encapsulate the changing button availability in a method called
Certains buttons are enabled/disabled appropriately in order to make the tool more robust.
Step 12: Handle different ImageComponents¶
currentImageComp on which
Segmentor action is applied
changes, we have to reset everything in order to make the tool robust.
segmentedMesh get the job done.
Hence, users are advised to finish segmenting one image before moving on
Now the user can change the
ImageComponent on which the contours are drawn.
It is our hope that you have gained some insight into making a CamiTK ready tool by increments. As with any piece of code, you may run into bugs with this tool. If that is the case, do not hesitate to let us know and we will try to fix them asap.