Nuclear Segmentation in Microscope Cell Images

I decided to blog my old papers (from when I did not have a science blog), mostly because of Melissa Terra’s blog (although I cannot hope to have as much success as she had). In any case, expect the next few weeks to go back to the past.

I will start with this one:

NUCLEAR SEGMENTATION IN MICROSCOPE CELL IMAGES: A HAND-SEGMENTED DATASET AND COMPARISON OF ALGORITHMS by Luis Pedro Coelho, Aabid Shariff, and Robert F. Murphy in Biomedical Imaging: From Nano to Macro, 2009. ISBI ’09. IEEE International Symposium on, 2009. DOI: 10.1109/ISBI.2009.5193098 [Pubmed Central open access version]

It’s more of a solid paper than a one announcing a major breakthrough, so it is interesting that this is currently my most cited paper (according to Google Scholar).

The original question of this paper was very simple: is it worth it to code up and run a complex segmentation algorithm over a simple one on that we were working with?

I hand-segmented a bunch of images from our datasets. Frankly, if I knew how much work this would take; I’d not have done it. And I would not have written this paper. I believe that this is why it became widely cited: a lot of people understand the value of the dataset (and use it for their work).

At the centre of the paper, we presented images such as this one, which had been manually segmented (by me and a subset by Aabid Shariff, according to the label it twice principle):


We then implemented some automatic segmentation algorithms and measured which were best able to reproduce the human labeled data.

Major conclusions

1. The method which won was by Lin et al., which is a model-based method [1]. In the meanwhile, however, other groups have reported better results on our dataset (list of citations at Google Scholar).

This means that it is worth it to run a more complex method.

2. Neither the Rand nor the Jaccard indices do very well in method evaluation (the Dice index, also widely used, is equivalent to the Jaccard index).

These indices do not take the pixel location into account. We propose a new metric that does, what we call a spatially-aware evaluation method, the normalised sum of distances (NSD), which does.

3. The NSD metric does better than Rand or jaccard [2].

Another interesting result is that the mean pixel value is a very good threshold for fluorescent microscopy.

Here is the reproducible research archive for this paper.

[1] Yes, their model is in 3D, while our data was 2D. I just don’t want to get into that game of making a minor and obvious tweak to an existing algorithm and calling it new. We used their method with the obvious adaptations for our data.
[2] Nowadays, I might try to develop a metric based on random walks as well. The NSD has the advantage that it is very fast to compute.

10 thoughts on “Nuclear Segmentation in Microscope Cell Images

  1. “2. Neither the Rand nor the Jaccard indices do very well in method evaluation (the Dice index, also widely used, is equivalent to the Jaccard index).”

    I haven’t read the paper but here is my 2 cents
    Check the Adjusted Rand and Adjusted Wallace index for better comparisons. Those indexes correct for the chance agreement.

    • That’s one problem (the lack of correction), but more fundamentally; we think that using clustering metrics for a spatial problem (segmentation) loses too much information.

    • To elaborate: take your gold standard single-object segmentations and two proposed segmentations

      1) Like your gold standard, but dilated around the border so that you have 1 extra pixel all around. All in all, 500 pixel values (which is the perimeter) are different.

      2) Like your gold standard, but with a 500 pixel blob attached at one end.

      If you are just counting pixels, you will penalize both the same, but option (1) should actually get almost zero penalty because your gold standard is not pixel exact either. However, adding a blob is a pretty big mistake.

  2. Pingback: Why Pixel Counting is not Adequate for Evaluating Segmentation | Meta Rabbit

  3. Pingback: Is Cell Segmentation Needed for Cell Analysis? | Meta Rabbit

  4. Pingback: Old papers: Structured Literature Image Finder (SLIF) | Meta Rabbit

  5. @luispedro, thanks for a great blog post (as well as the subsequent ones) and paper! Question: where in the GitHub repo is your implementation of the algorithm from Lin et al ’03? I couldn’t find it very easily…


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s