How to write a parallel macro part 2

A segmentation example

The goal of this worksheet is to show three versions of a very simple (and naive) segmentation pipeline.

Serial code

Consider the original Imagej macro here. The core of this macro

	setAutoThreshold("Default dark");
	setOption("BlackBackground", false);
	run("Convert to Mask");
	run("Invert LUT");
	run("Erode");
	run("Erode");
	run("Erode");
	run("Erode");
	run("Erode");

has been created by using the Macro recorder (Plugins -> Macros -> Record...) after a sample input image has been opened in Fiji. This piece of code thresholds the image, converts it into a true (segmentation) mask in which the segments are dilated (despite the name of the operations) a few times to extend the masks to roughly match the expected extent. In real life 😉, one wants to replace this snippet with something more appropriate to one's input data in question.

We wrapped this segmentation code into a separate function segmentHuhData().

In order to automate the whole segmentation process, the function has been extended with macro recorded load image and save image operations, which we have afterwards turned into general loaders and savers, and kept them in their own functions load(i) and save(i), respectively.

The complete and generic segmentHuhData(i) functions has been created in this way.

The automation, that is processing here some 30 images from a given input folder, is then achieved using the main loop:

for (i = 0; i < 30; ++i) {
	segmentHuhData(i);
}

Parallel code

Following up the manual put forth in this Short Guide, we "parallelized" the macro. In particular, we changed the main loop into

workers = parGetSize();
for (i = parGetRank(); i < 30; i += workers) {
	segmentHuhData(i);
}

which uses the parGetSize() and parGetRank() functions to enable this macro to process different images from the input folder when this one code is executed in multiple instances on multiple nodes inside some cluster.

Note that we had to copy the image data beforehand on the cluster, and adjust the paths to it at the beginning of the macro (the INFOLDER and OUTFOLDER variables).

This macro is already functional and can be well used in conjunction with the HPC Workflow Manager.

Parallel code with progress reporting

Finally, a couple of rather cosmetic changes were introduced to arrive to a parallel macro with progress reporting.

main hub:house: previous 👈 next 👉