Self-Organizing Maps Tutorial – Algobeans

The time period ‘self-organizing map’ may conjure up a militaristic picture of information factors marching in direction of their contingents on a map, which is a slightly apt analogy of how the algorithm really works.

A self-organizing map (SOM) is a clustering approach that helps you uncover classes in massive datasets, resembling to search out buyer profiles based mostly on a listing of previous purchases. It’s a particular breed of unsupervised neural networks, the place neurons (additionally known as nodes or reference vectors) are organized in a single, 2-dimensional grid, which might take the form of both rectangles or hexagons.


SOM grids can either be rectangular or hexagonal.

Determine 1. SOM grids can both be rectangular or hexagonal. Every sq. or hexagon is a neuron.

By way of a number of iterations, neurons on the grid will step by step coalesce round areas with excessive density of information factors. Therefore, areas with many neurons may mirror underlying clusters within the knowledge. Because the neurons transfer, they inadvertently bend and twist the grid to extra intently mirror the general topological form of our knowledge.

Let’s see a visible instance of how SOM works. The accompanying SOM code in R is on our GitHub page.

A Visible Walkthrough

One instance of a knowledge sort with greater than two dimensions is colour. Colours have three dimensions, usually represented by RGB (purple, inexperienced, blue) values. On this instance, we are going to see how SOM can distinguish two colour clusters.

Knowledge Era

We picked two colours—yellow and inexperienced—round which to generate random samples to kind two clusters. We will visualize our colour clusters utilizing blue and inexperienced values, that are the size alongside which the clusters are most differentiated.

Two color clusters of yellow and green, in 3D and 2D spaces.

Determine 2. Two colour clusters of yellow and inexperienced, in 3D and 2D areas.

Knowledge Evaluation

It’s time to construct our SOM. We used an Eight x Eight rectangular grid, so there have been 64 neurons in whole. Initially, neurons within the SOM grid begin out in random positions, however they’re step by step massaged right into a mould outlining the form of our knowledge. That is an iterative course of, which we are able to watch from the animated GIF beneath:

Figure 3. Animated GIF showing how an SOM grid evolves to take the shape of our data. Note that the visualization is a top-down view of the data, and the neurons are actually moving in three dimensions

Determine 3. Animated GIF exhibiting how an SOM grid evolves to take the form of our knowledge. Be aware that the visualization is a top-down view of the info, and the neurons are literally transferring in three dimensions


We will see that the grid’s form stabilizes after a few hundred iterations. To test that the algorithm has converged, we are able to plot the evolution of the SOM’s vitality—initially, the SOM evolves quickly, however because it reaches the approximate form of the info, the speed of change slows down.


Determine 4. Evolution of the SOM’s vitality. It took about 150 iterations for neurons within the SOM grid to stabilize.

To get an outline of what number of knowledge factors every neuron corresponded to, we are able to plot a frequency map of the grid, proven beneath. Every neuron is represented by a sq., and the pink area throughout the sq. represents the relative variety of knowledge factors that neuron is positioned closest to—the bigger the pink space, the extra knowledge factors represented by that neuron.


Determine 5. Frequency map of neurons in an Eight x Eight SOM grid.

From the frequency map, we are able to see a transparent divide separating a high left neuron cluster from a smaller backside proper cluster. This divide is represented by the neurons in-between with small or no pink squares.

To confirm that there’s certainly a divide, we are able to plot what’s known as a U-matrix, which visualizes how a lot neurons differ from one another in 2-dimensional house. When two neurons correspond to vastly completely different units of information factors, they might be separated by a bigger distance, denoted by a pink colour. Alternatively, neurons representing comparable knowledge factors are separated by shorter distances, denoted by a blue colour.


Determine 6. U-matrix exhibiting similarity and dissimilarity between neurons. Helpful for figuring out clusters.

Evaluating the sizes of neuron clusters, we are able to infer that the larger cluster within the high left in all probability corresponds to the bigger group of yellow knowledge factors, leaving the smaller backside proper cluster to correspond to inexperienced knowledge factors.

To test if we labelled the clusters appropriately, we are able to infer a colour profile for every neuron, by averaging the RGB values of information factors related to that neuron:


Determine 7. Radar plots exhibiting the aggregated profile of every neuron, as decided by their constituent knowledge factors.

From the radar plots, we are able to observe that the neurons within the high left have knowledge factors of excessive purple and inexperienced values, and mixing these two colours in gentle provides us, as you’d have guessed, yellow. Conversely, the remaining neurons on the underside proper are characterised by knowledge factors with excessive values of blue and a slight tinge of inexperienced, which might give us a bluish inexperienced hue.

By representing knowledge with a number of variables in simply two dimensions, the SOM grid is well-suited for knowledge visualization (it really works equally to a different well-known approach known as t-SNE). Whereas the 2-dimesional house is well-liked as a result of its use for visualization, the SOM is a normal dimension discount approach that may simplify a dataset to any variety of variables, and is intently associated to what we learned previously on principal components analysis.

Now that you simply’ve seen how SOM successfully identifies clusters, we are going to clarify the way it works below the hood.

How does SOM Work?

In a nutshell, an SOM includes neurons within the grid, which step by step adapt to the intrinsic form of our knowledge. The ultimate outcome permits us to visualise knowledge factors and determine clusters in a decrease dimension.

So how does the SOM grid be taught the form of our knowledge? Properly, that is finished in an iterative course of, which is summarized within the following steps, and visualized within the animated GIF beneath:

Step 0: Randomly place the grid’s neurons within the knowledge house.

Step 1: Choose one knowledge level, both randomly or systematically biking by means of the dataset so as

Step 2: Discover the neuron that’s closest to the chosen knowledge level. This neuron is named the Finest Matching Unit (BMU).

Step 3: Transfer the BMU nearer to that knowledge level. The gap moved by the BMU is decided by a studying fee, which decreases after every iteration.

Step 4: Transfer the BMU’s neighbors nearer to that knowledge level as effectively, with farther away neighbors transferring much less. Neighbors are recognized utilizing a radius across the BMU, and the worth for this radius decreases after every iteration.

Step 5: Replace the educational fee and BMU radius, earlier than repeating Steps 1 to 4. Iterate these steps till positions of neurons have been stabilized.

Iterative process of an SOM.

Determine 8. Iterative strategy of an SOM.

The training fee and BMU radius ought to be tuned by way of validation. If values for each are too excessive, neurons might be shoved round continuously with out settling down. But when values are too low, the evaluation will take too lengthy as neurons inch in direction of their optimum positions. Therefore, it’s ultimate to begin with bigger studying fee and BMU radius first, earlier than lowering them over time.

One other characteristic that we have to validate is the optimum variety of neurons within the grid. Recall that as a result of every neuron has a number of knowledge factors related to it, it may be handled as a mini-cluster. We will thus validate every neuron to see if its related knowledge factors correspond to identified sub-clusters of, say, shopper profiles. To ensure that such clusters to be distilled nevertheless, there ought to be fewer neurons than knowledge factors, in order that comparable knowledge factors could be mapped to every neuron.

One factor to notice earlier than we apply SOM: Variables measured in several models might intervene with the velocity and accuracy of our evaluation. For instance, a variable measured in centimeters would have a worth 100 occasions bigger than the identical one measured in meters. To forestall any variable from overpowering the others, we have to standardize all variables. Standardization is analogous to expressing every variable by way of percentiles, that means to shift them onto a uniform normal scale, in order that they’re of the identical measurement unit.

Limitations of SOM

SOM simplifies datasets with many variables, which is beneficial for visualization and figuring out clusters. Nonetheless, it has a number of drawbacks:

Doesn’t deal with categorical variables effectively. To acquire a scatterplot with good knowledge unfold to determine clusters, SOM should assume that each one variables within the dataset are steady. Plotting categorical values will lead to knowledge factors lining up at discrete values as an alternative of spreading throughout the plot.

Computationally costly. A dataset with extra variables would require longer occasions to calculate distances and determine BMUs. To expedite computations, we might enhance our preliminary positions of neurons from a random state to a extra knowledgeable approximation with the assistance of a simpler dimension reduction technique, such as principal components analysis. By beginning the neurons off nearer to the info factors, much less time can be wanted to maneuver them to their optimum places.

Doubtlessly inconsistent options. As preliminary positions of neurons differ every time the SOM evaluation is run, the eventual SOM map generated may even differ. Generally, overly massive clusters could also be cut up up and represented by two separate clusters of neurons. Due to this fact, earlier than concluding on the variety of clusters, the SOM evaluation could be repeated to test for consistency, and ensuing clusters ought to be validated in opposition to precise instances.


  • A self-organizing map (SOM) is a grid of neurons which adapt to the topological form of a dataset, permitting us to visualise massive datasets and determine potential clusters.
  • An SOM learns the form of a dataset by repeatedly transferring its neurons nearer to the info factors. Distinct teams of neurons could thus mirror underlying clusters within the knowledge.
  • SOMs are finest for datasets with steady variables, and it ought to be repeated to test for consistency. Ensuing clusters must also be validated.


Did you be taught one thing helpful at this time? We’d be glad to tell you when now we have new tutorials, in order that your studying continues!

Enroll beneath to get bite-sized tutorials delivered to your inbox:

Free Data Science Tutorials

Copyright © 2015-Current All rights reserved. Be a cool bean.

Source link

Write a comment