Hierarchical Clustering & Dendrogram Analysis

Unlike K-Means, which requires pre-specifying the number of clusters K and partitions data into flat buckets, Hierarchical Clustering builds a multi-level nested tree of clusters called a Dendrogram. This allows researchers and data scientists to explore relationships at multiple granularities simultaneously.

Agglomerative (Bottom-Up) vs. Divisive (Top-Down)

• Agglomerative (Bottom-Up - Standard): Starts with each individual data point as its own solitary cluster (N clusters). At each iteration, the two most similar clusters are merged together, until all points coalesce into a single root cluster.
• Divisive (Top-Down): Starts with all data points in one giant cluster and recursively splits them into smaller sub-clusters using flat algorithms like K-Means.

Linkage Criteria: How Distance Between Clusters is Defined

When deciding which two multi-point clusters to merge, the chosen Linkage Criterion dramatically shapes the resulting tree geometry:

Interpreting Dendrograms & Cutting the Tree

A Dendrogram visualizes the hierarchical merger sequence. The vertical height of each horizontal cross-bar represents the exact distance threshold at which two clusters were merged. To choose the number of clusters, locate the longest vertical lines that are not intercepted by any horizontal bar, and draw a horizontal threshold cut line across the dendrogram.

from scipy.cluster.hierarchy import dendrogram, linkage
from sklearn.cluster import AgglomerativeClustering
import matplotlib.pyplot as plt

# Generate linkage matrix using Ward criterion
Z = linkage(X, method='ward')

# In Scikit-Learn: Fit Agglomerative model directly
agg = AgglomerativeClustering(n_clusters=3, linkage='ward')
cluster_labels = agg.fit_predict(X)