Introduce
A cluster refers to a collection of data points aggregated together because of certain similarities.
Fuzzy c means
Fuzzy c-means (FCM) is a method of clustering which allows one piece of data to belong to two or more clusters. This method (developed by Dunn in 1973 and improved by Bezdek in 1981) is frequently used in pattern recognition. It is based on minimization of the following objective function:
J m = ∑ i = 1 N ∑ j = 1 C u i j m ∥ x i − c j ∥ 2 , 1 ≤ m ≤ ∞ J_m = \sum_{i=1}^{N}\sum_{j=1}^{C}u_{ij}^m\|x_i-c_j\|^2,\,\,\,1\leq m\leq\infty Jm=i=1∑Nj=1∑Cuijm∥xi−cj∥2,1≤m≤∞
where m m m is any real number greater than 1, u i j u_{ij} uij is the degree of membership of x i x_i xi in the cluster j j j, x i x_i xi is the i i ith of d-dimensional measured data, c j c_j cj is the d d d-dimension center of the cluster, and ∥ ∗ ∥ \|*\| ∥∗∥ is any norm expressing the similarity between any measured data and the center.
Fuzzy partitioning is carried out through an iterative optimization of the objective function shown above, with the update of membership u i j u_{ij} uij and the cluster centers c j c_j cj by:
u i j = 1 ∑ k = 1 C ( ∥ x i − c j ∥ x i − c k ) 2 m − 1 c j = ∑ i = 1 N u i j m ⋅ x i ∑ i = 1 N u i j m \begin{array}{l} u_{ij} = \frac{1}{\sum_{k=1}^{C}\left(\frac{\|x_i-c_j\|}{x_i-c_k}\right)^{\frac{2}{m-1}}} \\ \\ c_j=\frac{\sum_{i=1}{N}u_{ij}^m\cdot x_i}{\sum_{i=1}^{N}u_{ij}^m} \end{array} uij=∑k=1C(xi−ck∥xi−cj∥)m−121cj=∑i=1Nuijm∑i=1Nuijm⋅xi
This iteration will stop when max i j { ∣ u i j ( k + 1 ) − u i j ( k ) ∣ } < ϵ \text{max}_{ij}\left\{|u_{ij}^{(k+1)}-u_{ij}^{(k)}|\right\}<\epsilon maxij{
∣uij(k+1)−uij(k)∣}<ϵ, where ϵ \epsilon ϵ is a termination criterion between 0 and 1, whereas k k k is the iteration steps. This procedure converges to a local minimum or a saddle point of J m J_m Jm.
The algorithm is composed of the following steps:
- Initialize U = [ u i j ] U=[u_{ij}] U=[uij] matrix, U ( 0 ) U^{(0)} U(0)
- At k − s t e p k-step k−step: calculate the centers vectors C ( k ) = [ c j ] C^(k)=[c_j] C(k)=[cj] with U ( k ) U^{(k)} U(k)
c j = ∑ i = 1 N u i j m ⋅ x i ∑ i = 1 N u i j m c_{j}=\frac{\sum_{i=1}^{N} u_{i j}^{m} \cdot x_{i}}{\sum_{i=1}^{N} u_{i j}^{m}} cj=∑i=1Nuijm∑i=1Nuijm⋅xi - Update U ( k ) U^{(k)} U(k), U ( k + 1 ) U^{(k+1)} U(k+1)
u i j = 1 ∑ k = 1 c ( ∥ x i − c j ∥ ∥ x i − c k ∥ ) 2 m − 1 u_{i j}=\frac{1}{\sum_{k=1}^{c}\left(\frac{\left\|x_{i}-c_{j}\right\|}{\left\|x_{i}-c_{k}\right\|}\right)^{\frac{2}{m-1}}} uij=∑k=1c(∥xi−ck∥∥xi−cj∥)m−121 - If ∥ U ( k + 1 ) − U ( k ) ∥ < ε \left\|U^{(k+1)}-U^{(k)}\right\|<\varepsilon ∥∥U(k+1)−U(k)∥∥<ε, then STOP; otherwise return to step 2.
import numpy as np
import cv2
import os
#import matplotlib.pyplot as plt
def fcm(data, cluster_n, options=None):
data_n = data.shape[0]
# Change the following to set default options
default_options = [2, # exponent for the partition matrix U
100, # max. number of iteration
1e-5, # min. amount of improvement
1] # info display during iteration
if options==None:
options = default_options
else:
# If "options" is not fully specified, pad it with default values.
if len(options) < 4:
tmp = default_options
tmp[0:len[options]] = options
options = tmp
expo = options[0] # Exponent for U
max_iter = options[1] # Max. iteration
min_impro = options[2] # Min. improvement
display = options[3] # Display info or not
obj_fcn = np.zeros([max_iter, 1]) # Array for objective function
U = initfcm(cluster_n, data_n) # Initial fuzzy partition
# Main loop
for i in np.arange(max_iter):
U, center, obj_fcn[i] = stepfcm(data, U, cluster_n, expo)
if display:
print('Iteration count = %d, obj. fcn = %f', i, obj_fcn[i])
# check termination condition
if i > 0:
if np.abs(obj_fcn[i] - obj_fcn[i-1]) < min_impro:
break
obj_fcn = obj_fcn[:i]
return center, U, obj_fcn
def stepfcm(data, U, cluster_n, expo):
"""
%STEPFCM One step in fuzzy c-mean clustering.
% [U_NEW, CENTER, ERR] = STEPFCM(DATA, U, CLUSTER_N, EXPO)
% performs one iteration of fuzzy c-mean clustering, where
%
% DATA: matrix of data to be clustered. (Each row is a data point.)
% U: partition matrix. (U(i,j) is the MF value of data j in cluster j.)
% CLUSTER_N: number of clusters.
% EXPO: exponent (> 1) for the partition matrix.
% U_NEW: new partition matrix.
% CENTER: center of clusters. (Each row is a center.)
% ERR: objective function for partition U.
%
% Note that the situation of "singularity" (one of the data points is
% exactly the same as one of the cluster centers) is not checked.
% However, it hardly occurs in practice.
%
"""
mf = np.power(U, expo) # MF matrix after exponential modification
center = mf.dot(data) / np.sum(mf,1,keepdims=True).dot(np.ones([1,data.shape[1]])) #new center
dist = distfcm(center, data) # fill the distance matrix
obj_fcn = np.sum(dist**2*mf) # objective function
tmp = dist**(-2/(expo-1)) # calculate new U, suppose expo != 1
U_new = tmp/(np.ones([cluster_n, 1])*np.sum(tmp,0,keepdims=True))
return U_new, center, obj_fcn
def distfcm(center, data):
"""
%DISTFCM Distance measure in fuzzy c-mean clustering.
% OUT = DISTFCM(CENTER, DATA) calculates the Euclidean distance
% between each row in CENTER and each row in DATA, and returns a
% distance matrix OUT of size M by N, where M and N are row
% dimensions of CENTER and DATA, respectively, and OUT(I, J) is
% the distance between CENTER(I,:) and DATA(J,:).
"""
out = np.zeros([center.shape[0], np.shape(data)[0]])
print(center.shape, data.shape)
# fill the output matrix
if center.shape[1] > 1:
for k in np.arange(0, center.shape[0]):
out[k, :] = np.sqrt(np.sum(((data-np.ones([data.shape[0], 1]).dot(center[k, :].reshape([1,-1])))**2), 1))
else: # 1-D data
for k in np.arange(0, center.shape[0]):
out[k, :] = np.transpose(np.abs(center[k]-data))
return out
def initfcm(cluster_n, data_n):
"""
%INITFCM Generate initial fuzzy partition matrix for fuzzy c-means clustering.
% U = INITFCM(CLUSTER_N, DATA_N) randomly generates a fuzzy partition
% matrix U that is CLUSTER_N by DATA_N, where CLUSTER_N is number of
% clusters and DATA_N is number of data points. The summation of each
% column of the generated U is equal to unity, as required by fuzzy
% c-means clustering.
"""
U = np.random.rand(cluster_n, data_n)
col_sum = np.sum(U, axis=0,keepdims=True)
U = U / np.repeat(col_sum, [cluster_n], axis=0)
return U
def main():
os.chdir(r'C:\Users\wgy\Desktop\literature\forestFire\postFire\c\Landsat5TMSardinia\camp\DATA\2017\fire_level_tar_files')
data1 = np.random.rand(50,2)+np.array([[7,8]])
data2 = np.random.rand(50,2)+np.array([[3,3]])
data = np.concatenate([data1,data2], axis=0)
#data = np.random.rand(100,2)
center, U, obj_fcn = fcm(data, 2)
#plt.plot(data[:,1], data[:,2],'o');
maxU = np.max(U, axis=0);
# Find the data points with highest grade of membership in cluster 1
index1 = U[0,:] == maxU
# Find the data points with highest grade of membership in cluster 2
index2 = U[1,:] == maxU
#fig = plt.figure()
#plt.plot(data[index1,0],data[index1,1], color='green', marker='*')
#plt.plot(data[index2,0],data[index2,1], color='red', marker='*')
# Plot the cluster centers
#plt.plot([center([0,1],1)],[center([0,1],2)],'*','color','k')
#fig.savefig('fcm.png', dpi=fig.dpi)
print(index1)
print(index2)
if __name__ == '__main__':
main()
K-Means
K-means clustering is one of the simplest and popular unsupervised machine learning algorithm.
problem discription
Given a set of observations ( x 1 x_1 x1, x 2 x_2 x2, …, x n x_n xn), where each observation is a d d d-dimensional real vector, k k k-means clustering aims to partition the n n n observations into k k k (≤ ‘‘n’’) sets S S S = {’‘S’'1, '‘S’'2, …, ‘‘Sk’’} so as to minimize the within-cluster sum of squares (WCSS) (i.e. variance). Formally, the objective is to find:
arg min S ∑ i = 1 k ∑ x ∈ S i ∥ x − μ i ∥ 2 = arg min S ∑ i = 1 k ∣ S i ∣ Var S i \underset{\mathbf{S}} {\operatorname{arg\,min}} \sum_{i=1}^{k} \sum_{\mathbf x \in S_i} \left\| \mathbf x - \boldsymbol\mu_i \right\|^2 = \underset{\mathbf{S}} {\operatorname{arg\,min}} \sum_{i=1}^k |S_i| \operatorname{Var} S_i Sargmini=1∑kx∈Si∑∥x−μi∥2=Sargmini=1∑k∣Si∣VarSi
where μ i \mu_i μi is the mean of points in S i S_i Si. This is equivalent to minimizing the pairwise squared deviations of points in the same cluster:
arg min S ∑ i = 1 k 1 2 ∣ S i ∣ ∑ x , y ∈ S i ∥ x − y ∥ 2 \underset{\mathbf{S}} {\operatorname{arg\,min}} \sum_{i=1}^{k} \, \frac{1}{2 |S_i|} \, \sum_{\mathbf{x}, \mathbf{y} \in S_i} \left\| \mathbf{x} - \mathbf{y} \right\|^2 Sargmini=1∑k2∣Si∣1x,y∈Si∑∥x−y∥2
The equivalence can be deduced from identity
∑ x ∈ S i ∥ x − μ i ∥ 2 = ∑ x ≠ y ∈ S i ( x − μ i ) ( μ i − y ) \sum_{\mathbf x \in S_i} \left\| \mathbf x - \boldsymbol\mu_i \right\|^2 =\sum_{\mathbf{x}\neq\mathbf{y} \in S_i}(\mathbf x - \boldsymbol\mu_i)(\boldsymbol\mu_i - \mathbf y) ∑x∈Si∥x−μi∥2=∑x=y∈Si(x−μi)(μi−y). Because the total variance is constant, this is equivalent to maximizing the sum of squared deviations between points in ‘‘different’’ clusters (between-cluster sum of squares, BCSS), which follows from the law of total variance.
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from sklearn.cluster import KMeans
X= -2 * np.random.rand(100,2)
X1 = 1 + 2 * np.random.rand(50,2)
X[50:100, :] = X1
plt.scatter(X[ : , 0], X[ :, 1], s = 50, c = 10*np.ones([X.shape[0]]))
plt.show()
from sklearn.cluster import KMeans
Kmean = KMeans(n_clusters=2)
Kmean.fit(X)
Kmean.cluster_centers_
plt.scatter(X[Kmean.labels_==0,0], X[Kmean.labels_==0,1], marker='s', s =50, color='green')
plt.scatter(X[Kmean.labels_==1,0], X[Kmean.labels_==1,1], marker='o', s =50, color='red')
plt.scatter(Kmean.cluster_centers_[0,0], Kmean.cluster_centers_[0,1], marker='s', s=200, color='black')
plt.scatter(Kmean.cluster_centers_[1,0], Kmean.cluster_centers_[1,1], marker='o', s=200, color='black')
plt.show()
Kmean.labels_
sample_test=np.array([-3.0,-3.0])
second_test=sample_test.reshape(1, -1)
Kmean.predict(second_test)
Rough K-Means
Illustrative Example
The following table shows example information system with real-valued conditional attributes. It consist of sis objects/genes, and two features/samples. k = 2 k=2 k=2, which is the number of clusters. Weight of the lower approximation W lower = 0.7 \mathrm{W}_{\text{lower}}=0.7 Wlower=0.7, Weight of the upper approximation W upper = 0.3 \mathrm{W}_{\text{upper}} = 0.3 Wupper=0.3 and Relative threshold = 2.
Example dataset for Rough K-Means U X Y 1 0 3 2 1 3 3 3 1 4 3 0.5 5 5 0 6 6 0 \text{Example dataset for Rough K-Means}\\ \begin{array}{|c|c|c|} \hline \mathbf{U} & \mathbf{X} & \mathbf{Y} \\ \hline 1 & 0 & 3 \\ \hline 2 & 1 & 3 \\ \hline 3 & 3 & 1 \\ \hline 4 & 3 & 0.5 \\ \hline 5 & 5 & 0 \\ \hline 6 & 6 & 0 \\ \hline \end{array} Example dataset for Rough K-MeansU123456X013356Y3310.500
Step1: Randomly assign each data objects to exactly one lower approximation
K 1 = { ( 0 , 3 ) , ( 1 , 3 ) , ( 3 , 1 ) } K 2 = { ( 3 , 0.5 ) , ( 5 , 0 ) , ( 6 , 0 ) } \begin{array}{l} \mathrm{K}_{1}=\{(0,3),(1,3),(3,1)\} \\ \mathrm{K}_{2}=\{(3,0.5),(5,0),(6,0)\} \end{array} K1={
(0,3),(1,3),(3,1)}K2={
(3,0.5),(5,0),(6,0)}
Step 2: In this case \underline{U}(K)\neq \emptyset$ and U ‾ ( K ) − U ‾ ( K ) = ∅ \overline{U}(K)-\underline{U}(K) = \emptyset U(K)−U(K)=∅, so we compute the centroid using C j = ∑ x ∈ U ‾ ( K ) x i ∣ U ‾ ( K ) ∣ \mathrm{C}_{\mathrm{j}}=\sum_{x \in \underline{U}(K)} \frac{x_{i}}{|\underline{U}(K)|} Cj=∑x∈U(K)∣U(K)∣xi,
C 1 = ( 0 + 1 + 3 3 , 3 + 3 + 1 3 ) = ( 1.33 , 2.33 ) C 2 = ( 3 + 5 + 6 3 , 0.5 + 0 + 0 3 ) = ( 4.67 , 0.17 ) \begin{array}{l} C_{1}=\left(\frac{0+1+3}{3}, \frac{3+3+1}{3}\right)=(1.33,2.33) \\ C_{2}=\left(\frac{3+5+6}{3}, \frac{0.5+0+0}{3}\right)=(4.67,0.17) \end{array} C1=(30+1+3,33+3+1)=(1.33,2.33)C2=(33+5+6,30.5+0+0)=(4.67,0.17)
Find the distance from centroid to each point using equlidean distance,
D i = ( x 2 − x 1 ) 2 + ( y 2 − y 1 ) 2 D_i=\sqrt{(x_2-x_1)^2+(y_2-y_1)^2} Di=(x2−x1)2+(y2−y1)2
d 1 ( X , C 1 ) d_{1}\left(X, C_{1}\right) d1(X,C1):
( 0 , 3 ) ( 1.33 , 2.33 ) ⇒ ( 1.33 − 0 ) 2 + ( 2.33 − 3 ) 2 = 1.49 ( 1 , 3 ) ( 1.33 , 2.33 ) ⇒ ( 1.33 − 1 ) 2 + ( 2.33 − 3 ) 2 = 0.75 ( 3 , 1 ) ( 1.33 , 2.33 ) ⇒ ( 1.33 − 3 ) 2 + ( 2.33 − 1 ) 2 = 2.13 ( 3 , 0.5 ) ( 1.33 , 2.33 ) ⇒ ( 1.33 − 3 ) 2 + ( 2.33 − 0.5 ) 2 = 2.48 ( 5 , 0 ) ( 1.33 , 2.33 ) ⇒ ( 1.33 − 5 ) 2 + ( 2.33 − 0 ) 2 = 4.45 ( 6 , 0 ) ( 1.33 , 2.33 ) ⇒ ( 1.33 − 6 ) 2 + ( 2.33 − 0 ) 2 = 5.22 \begin{array}{l} (0,3)(1.33,2.33) \Rightarrow \sqrt{(1.33-0)^{2}+(2.33-3)^{2}}=1.49 \\ (1,3)(1.33,2.33) \Rightarrow \sqrt{(1.33-1)^{2}+(2.33-3)^{2}}=0.75 \\ (3,1)(1.33,2.33) \Rightarrow \sqrt{(1.33-3)^{2}+(2.33-1)^{2}}=2.13 \\ (3,0.5)(1.33,2.33) \Rightarrow \sqrt{(1.33-3)^{2}+(2.33-0.5)^{2}}=2.48 \\ (5,0)(1.33,2.33) \Rightarrow \sqrt{(1.33-5)^{2}+(2.33-0)^{2}}=4.45 \\ (6,0)(1.33,2.33) \Rightarrow \sqrt{(1.33-6)^{2}+(2.33-0)^{2}}=5.22 \end{array} (0,3)(1.33,2.33)⇒(1.33−0)2+(2.33−3)2=1.49(1,3)(1.33,2.33)⇒(1.33−1)2+(2.33−3)2=0.75(3,1)(1.33,2.33)⇒(1.33−3)2+(2.33−1)2=2.13(3,0.5)(1.33,2.33)⇒(1.33−3)2+(2.33−0.5)2=2.48(5,0)(1.33,2.33)⇒(1.33−5)2+(2.33−0)2=4.45(6,0)(1.33,2.33)⇒(1.33−6)2+(2.33−0)2=5.22
d 2 ( X , C 2 ) d_{2}\left(X, C_{2}\right) d2(X,C2):
( 0 , 3 ) ( 4.67 , 0.17 ) ⇒ ( 4.67 − 0 ) 2 + ( 0.17 − 3 ) 2 = 5.46 ( 1 , 3 ) ( 4.67 , 0.17 ) ⇒ ( 4.67 − 1 ) 2 + ( 0.17 − 3 ) 2 = 4.63 ( 3 , 1 ) ( 4.67 , 0.17 ) ⇒ ( 4.67 − 3 ) 2 + ( 0.17 − 1 ) 2 = 1.86 ( 3 , 0.5 ) ( 4.67 , 0.17 ) ⇒ ( 4.67 − 3 ) 2 + ( 0.17 − 0.5 ) 2 = 1.70 ( 5 , 0 ) ( 4.67 , 0.17 ) ⇒ ( 4.67 − 5 ) 2 + ( 0.17 − 0 ) 2 = 0.37 ( 6 , 0 ) ( 4.67 , 0.17 ) ⇒ ( 4.67 − 6 ) 2 + ( 0.17 − 0 ) 2 = 1.34 \begin{array}{l} (0,3)(4.67,0.17) \Rightarrow \sqrt{(4.67-0)^{2}+(0.17-3)^{2}}=5.46 \\ (1,3)(4.67,0.17) \Rightarrow \sqrt{(4.67-1)^{2}+(0.17-3)^{2}}=4.63 \\ (3,1)(4.67,0.17) \Rightarrow \sqrt{(4.67-3)^{2}+(0.17-1)^{2}}=1.86 \\ (3,0.5)(4.67,0.17) \Rightarrow \sqrt{(4.67-3)^{2}+(0.17-0.5)^{2}}=1.70 \\ (5,0)(4.67,0.17) \Rightarrow \sqrt{(4.67-5)^{2}+(0.17-0)^{2}}=0.37 \\ (6,0)(4.67,0.17) \Rightarrow \sqrt{(4.67-6)^{2}+(0.17-0)^{2}}=1.34 \end{array} (0,3)(4.67,0.17)⇒(4.67−0)2+(0.17−3)2=5.46(1,3)(4.67,0.17)⇒(4.67−1)2+(0.17−3)2=4.63(3,1)(4.67,0.17)⇒(4.67−3)2+(0.17−1)2=1.86(3,0.5)(4.67,0.17)⇒(4.67−3)2+(0.17−0.5)2=1.70(5,0)(4.67,0.17)⇒(4.67−5)2+(0.17−0)2=0.37(6,0)(4.67,0.17)⇒(4.67−6)2+(0.17−0)2=1.34
Step 3: Assign each object to the lower approximation U ‾ ( K ) \underline{U}(K) U(K) or upper approximation U ‾ ( K ) \overline{U}(K) U(K) of cluster i i i respectively. Check if d ( X , C i ) / d ( X , C j ) ≤ epsilon d(X,C_i)/d(X,C_j)\leq \text{epsilon} d(X,Ci)/d(X,Cj)≤epsilon.
- ( 0 , 3 ) ⇒ d 2 / d 1 = 5.46 / 1.49 = 3.66443 ≰ 2 (0,3) \Rightarrow \mathrm{d}_{2} / \mathrm{d}_{1}=5.46 / 1.49=3.66443 \nleq 2 (0,3)⇒d2/d1=5.46/1.49=3.66443≰2. So, x 1 x_1 x1 will be part of K ‾ 1 \underline{K}_1 K1
- ( 1 , 3 ) ⇒ 4.63 / 0.75 = 6.173 ≰ 2 (1,3) \Rightarrow 4.63 / 0.75=6.173 \nleq 2 (1,3)⇒4.63/0.75=6.173≰2. So, x 2 x_2 x2 will be a part of K ‾ 1 \underline{K}_1 K1
- ( 1 , 3 ) ⇒ 2.13 / 1.86 = 1.145 < 2 (1,3) \Rightarrow 2.13/1.86=1.145 < 2 (1,3)⇒2.13/1.86=1.145<2. So, x 3 x_3 x3 will not be a part of K ‾ 1 & K ‾ 2 \underline{K}_1\&\underline{K}_2 K1&K2
- ( 3 , 1 ) ⇒ 2.48 / 1.70 = 1.458 < 2 (3,1) \Rightarrow 2.48/1.70=1.458 < 2 (3,1)⇒2.48/1.70=1.458<2. So, x 4 x_4 x4 will not be a part of K ‾ 1 & K ‾ 2 \underline{K}_1\&\underline{K}_2 K1&K2
- ( 5 , 0 ) ⇒ 4.35 / 0.37 = 11.756 ≰ 2 (5,0) \Rightarrow 4.35/0.37=11.756\nleq 2 (5,0)⇒4.35/0.37=11.756≰2. So, x 5 x_5 x5 will be a part of K ‾ 2 \underline{K}_2 K2
- ( 6 , 0 ) ⇒ 5.22 / 1.34 = 3.895 ≰ 2 (6,0) \Rightarrow 5.22/1.34=3.895 \nleq 2 (6,0)⇒5.22/1.34=3.895≰2. So, x 6 x_6 x6 will be a part of K ‾ 2 \underline{K}_2 K2
Now, we have cluster
K ‾ 1 = { ( 0 , 3 ) , ( 1 , 3 ) } K ‾ 1 = { ( 0 , 3 ) , ( 1 , 3 ) , ( 3 , 1 ) , ( 3 , 0.5 ) } \underline{K}_1=\{(0,3), (1,3)\}\,\,\,\,\overline{K}_1=\{ (0,3), (1,3), (3,1), (3,0.5)\} K1={
(0,3),(1,3)}K1={
(0,3),(1,3),(3,1),(3,0.5)}
K ‾ 2 = { ( 5 , 0 ) , ( 6 , 0 ) } K ‾ 2 = { ( 5 , 0 ) , ( 6 , 0 ) , ( 3 , 0 ) , ( 3 , 0.5 ) } \underline{K}_2=\{(5,0), (6,0)\}\,\,\,\,\overline{K}_2=\{ (5,0), (6,0), (3,0), (3,0.5)\} K2={
(5,0),(6,0)}K2={
(5,0),(6,0),(3,0),(3,0.5)}
Here, U ‾ ( K ) ≠ ∅ \underline{U}(K)\neq \emptyset U(K)=∅ and U ‾ ( K ) − U ‾ ( K ) ≠ ∅ \overline{U}(K)-\underline{U}(K)\neq\emptyset U(K)−U(K)=∅ then find out the new centroid by using below equation,
C j = W lower × ∑ x ∈ U ‾ ( K ) x j ∣ U ‾ ( K ) ∣ + W upper × ∑ x ∈ U ‾ ( K ) − U ‾ ( K ) x i ∣ U ‾ ( K ) − U ‾ ( K ) ∣ C_j=W_{\text{lower}}\times\sum_{x\in\underline{U}(K)} \frac{x_j}{|\underline{U}(K)|} + W_{\text{upper}} \times \sum_{x\in \overline{U}(K)-\underline{U}(K)}\frac{x_i}{|\overline{U}(K)-\underline{U}(K)|} Cj=Wlower×∑x∈U(K)∣U(K)∣xj+Wupper×∑x∈U(K)−U(K)∣U(K)−U(K)∣xi