Based Image Segmentation
This article makes a short comparison between several multidimensional image segmentation
methods using global image histogram multistage entropy functions and
2D entropy. Using two types of 3D image histogram entropy we can create automatic and optimal image
segmentation for different applications such as; micro
objects inspection and biomedical cell analysis systems, digital video for security applications
and multimedia motion based segmentation.
Image segmentation using 2D entropy function
Entropy function of 2D grey scale image histograms can be used for automatic
For more details see
Maximum of entropy function can
be used for automatic threshold pointer that defines the brightness threshold value – b* for the
image (Fig. 1.a and Fig. 1.b).
Fig.1.a, Grey scale image
(right) and its 2D PDF (left)
Fig.1.b, BW image (right)
calculated using threshold level defined by 2D entropy function (left)
2D multistage entropy
Single stage 2D entropy function
is fast and automatic approach for thresholding different images, preferably for
inspection needs. When we work with outdoor and complex images this is not
sufficient. Thus the multistage 2D entropy function can be used (Fig. 2)
compares multistage image thresholding using fixed values and entropy calculated
Fig.2.a, Multi threshold – 4
layers, using fixed values
Fig.2.b, Multi threshold – 4
layer image, using multistage 2D entropy (right) and its entropy (left)
This function can be used to create image object extraction mask
- this mask is used foe multilayer
object based segmentation. We can continue this local entropy segmentation using the
same procedure for any new entropy region inside image histogram. Best results
can be achieved using 4, 8 and 16
3D Entropy based image segmentation
2D entropy function is commonly
used for many image segmentation applications. 2D entropy lacks his robustness
when multiple images with large variety contrasts between different regions and background need to be segmented. Here
3D entropy function comes in sense.
Before this we need to define and calculate 3D image histogram (joint pixel
probability function) (Fig. 3.a).
Using this histogram we can calculate 3D entropy function (Fig. 3.b). Finding
its maximum we can threshold image much better than using 2D entropy. This type
of thresholding will provide us information regarding two types of joint pixels:
homogeneity zones (joint pixels having the same brightness) and non homogeneity
(object surrounding areas and shades).
Fig. 3, 3D image histogram (a),
3D entropy function (b)
3D entropy function is
calculated as a sum of all entropy functions calculated for 3D histogram columns
and rows. The maximum of this function can be found dividing 3D histogram in to
4 separate regions; A, B, C, D (Fig. 4). Zones A and B corresponds to object and
background, and zones C and D corresponds to object shades and surrounding areas
brightness gradient. In most of cases 3D entropy maximum lays on the main
diagonal passing through A and B quadrants.
Fig. 4, 3D entropy maximum
divide 3D image histogram in to 4 regions (A, B, C and D)
This diagonal plays an important
role in 3D histogram, because values laying on it represent the probability to
occur different homogeneity zones inside of image. If images are noisy or very
shaded, their histogram maximum can lie around main diagonal. In most cases well
filtered images have most of values grouped around main diagonal. Different
authors examine zone C and D separately, but we thing that for most of standard
applications we can treat B, C and D as background. Zones C and D take an
important role when multistage segmentation is taking place. If we have 3D
We calculate image 3D entropy
function using discrete histogram . 3D entropy is calculated for each column and
row inside using method described in section (2.1).
After this we create two 3D entropy functions, first is called gradient entropy
function and is presented by;
And second for rows;
After this we need to find
Some implementations may
consider that 3D entropy function is a simultaneously calculated for each one
point inside 3D histogram. We prefer to create 3D entropy function as a fraction
of 2D columns and rows. This kind of representation gives us the right to make
image segmentation based on entire 3D entropy (Fig. 5) and on 3D gradient
entropy, without having the need to recalculate 3D entropy each time we need to
make segmentation for homogeneity and gradient zones.
Fig. 5, Image thresholding using
2D entropy function (a), and using 3D entropy function (b).
Image segmentation using 3D
Because 3D entropy is based on
2D entropy calculation of single columns and rows we introduce a new 3D gradient
entropy function. This function examines only the columns of 3D histogram,
interpreting values on main homogeneity diagonal as zeros. This approach gives
us a powerful method to calculate entropy only of brightness transition zones.
This technique give use the power of local adaptive 2D entropy segmentation
method and histogram equalization using global 3D image histogram (Fig. 6).
Fig. 6, 3D image gradient
entropy (a), image segmented using 3D gradient entropy (b), and using
standard 3D entropy (c)
3D gradient entropy can help us
to find target object even if we work with noisy images having bad spatial
resolution and contrast (Fig. 7).
Fig. 7, Input image (a), 3D
gradient entropy (b), and 2D entropy (c) – object is lost.
Multistage 3D and gradient
In section 2.2 we explained how
a multistage segmentation mask can be calculated using entropy based histogram
division and local entropy definition. No considers we have 3D entropy function
calculated for . In section 3 we define how simple BW threshold value can be
calculated more precisely taking in account joint pixel probability
distribution. Nothing easier if we treat 3D histogram quadrant A and B, C, D as
we treated 2D histograms and . In this case we will define and histograms (Fig.
7). Remember to fill all non target histogram quadrants as filled with zeros.
Thus the multistage 3D entropy function is calculated.
Fig. 7, Multistage 3D entropy segmentation
3D gradient multistage entropy is calculated by calculating entropy functions
for all columns of 3D histogram, treating main diagonal as filled with zeros.
Difference between 3D multistage entropy and 3D gradient multistage entropy is
shown on Fig. 8. Because gradient entropy analyze and group only image shading
zones it can be used as a stage before image texture analysis.
Fig. 8, Multistage 2D entropy
image (a) and 3D histogram (c), 3D gradient entropy image (b) and his
histogram (d), 3D entropy function (e), 3D gradient entropy function (f).
Entire 3D entropy function
trends to separate 3D histogram mainly inside main homogeneity diagonal, making
easier to better extract image homogeneity zones and surrounding areas (Fig.
8,b). In opposite to this 3D gradient entropy separate 3D histogram in to
different pikes around homogeneity diagonal (Fig. 8,e).
Complex method for multidimensional entropy based image segmentation
Above we have explained two global histogram methods for 2D and 3D entropy based
multistage image segmentation. These methods can be combined in to
multidimensional entropy image. Results obtained from 2D and 3D multistage
entropy give us a good understanding about image object laying in large spatial
areas and their edges and surrounding areas and shades. 3D gradient entropy
multistage segmentation give us the information regarding image global shading,
thus giving a god background for future texture analysis and processing using
global histogram methods. Combining results achieved by 2D multistage entropy
and 3D gradient entropy provide valuable information regarding large and small
object statistical representation. Multistage entropy image 3D histograms have
less data to be compared, making image classification faster, than using entire
3D histograms. Combined images are more similar to original one (Fig. 9).
Fig. 9, Multidimensional entropy
image combined from 2D multistage entropy and 3D gradient entropy (a), and
original image (b)
The most important in new model
for complex multidimensional entropy based image segmentation is that we have
more information for spatial image regions – objects and background, including
information about the brightness gradient inside them. Because 3D image
histograms can be interpreted in several ways; main diagonal homogeneity
entropy, entire entropy and 3D histogram columns entropy, we can extract
corresponding data for each one selected area inside 3D histogram space and
finding corresponding objects inside grey scale image.
Fig. 10, Image segmented using
2D entropy function (a) and 3D gradient entropy (b).
3D image entropy is a global
histogram method producing homogenization effect, like statistical mode and
median filtration. This method is not comprehensive. It can be used in several
types of images: where we have constant background and many constant objects
having similar brightness and shape. Improvements can be significant in images
that have large amount of low pass and high pass noise, such as scanned
documents (Fig. 10). In such cases noise can be classified in 2-3 classes. Using
gradient 3D entropy function scanned documents pixels are simply divided in to 2
classes: background noise and object noise.
It is recommended to calculate
3D gradient entropy function for any grey level channel after 2D multistage
entropy mask calculation. In these case each one grey level channel can be
classified as a entropy defined maximum grey level + gradient entropy defined
pixel to pixel gradient (deviation). In most cases this method lack his
robustness in images having many different areas with different contrast,
shapes, size - natural photos having large homogeneity areas and large gradient
areas are not good for 3D gradient entropy analysis.
Video motion segmentation
using 3D multistage entropy frames
3D multidimensional image
entropy can help us to achieve better and faster digital video segmentation.
Such segmentation is important for content based video post processing. A good
example of statistical based motion detection is shown
here. Using 2D and 3D multistage entropy images we can faster the process of
motion detection simultaneously achieving object segmentation and statistical
classification, see section 7. Multistage 2D entropy + 3D gradient entropy
images can be used to improve results in algorithms that use statistical motion
based visual scene indexing using 3D image histograms. In such cases it improves
video segmentation (Fig. 11).
Fig. 11, Video motion
segmentation using 3D histograms of original 8bit grey level frame sequence
(a), corresponding 8 layer 3D entropy frame sequence (b), 8 layer 3D entropy
frame sequence after frame 3D histogram homogenization.
3D multistage entropy frame
sequence tend to blink
in small information changes between 2
video frames (Fig. 12). We can avoid this negative effect using frame sequence
3D histogram homogenization (equalization). 2D histogram equalization do not
provide such balancing effect.3D histogram equalization group joint pixels instead of single pixels like
this is done in 2D histogram equalization.
In some simple cases when
working with good filtered images we can use single stage 2D entropy function.
For better region segmentation in more shaded images we can use multistage 2D
entropy function. Even better spatial object segmentation can be achieved using
global 3D entropy function and its multistage variant.
Petrov, G.,Iliev, P. “Multidimensional dynamic scene analysis for video security
applications” – IEEE Computer Science’2006 Istanbul
Tzvetkov, P.; Petrov, G., Motion Detection Using 3D Image Histograms Sequences
Analysis IEEE Volume, Sept. 2005 Page(s):596 - 601
PUN. T. "A new
method for grey level thresholding using the entropy of the histogram", Signal
Processing, 1980, 2 pp. 223-237
G.,Althouse, "Target detection in multispectral images using the spectral
co-occurrence matrix and entropy thresholding" Optical Engineering 34(7),
2135—2148 (July 1995)