Feature Scaling in Machine Learning: A Practical Guide

By /

Feature Scaling in Machine Learning: A Practical Guide
When training machine learning models, having features on vastly different scales can slow down learning or even lead to suboptimal performance. Feature scaling—often simply called normalization—is a technique to adjust the range of your features so that they become comparable. In this post, we’ll explain a few common methods, including simple scaling, mean normalization, and Z‑score normalization, and show you how to implement them in Python.

Why Scale Your Features?
Imagine one feature represents house sizes in square feet (ranging from 300 to 2,000) while another captures the number of bedrooms (ranging from 0 to 5). Without scaling, the larger values of the house size could dominate the learning process—even if both features are equally important. Scaling transforms each feature so that they contribute equally, which can also help gradient descent converge faster.

Methods for Feature Scaling
1. Simple Scaling by Maximum Value
One straightforward method is to divide each feature by its maximum value. For example, if your feature x1 ranges from 300 to 2,000, you can obtain a scaled version by calculating:

$$
x_{1,\text{scaled}} = \frac{x_1}{x_{1,\text{max}}}
$$

This operation transforms 𝑥1 so that its values now lie roughly between 0.15 and 1. Likewise, if another feature 𝑥2 ranges from 0 to 5, dividing by 5 will re-scale it to the
[0,1] interval.

Below is a simple Python example using scikit-learn:

import pandas as pd
from sklearn.preprocessing import MinMaxScaler

# Example dataset
data = pd.DataFrame({
    'HouseSize': [300, 600, 1200, 1800, 2000],
    'Bedrooms': [1, 2, 3, 4, 5]
})

# Initialize the scaler for each feature (using max values)
scaler = MinMaxScaler(feature_range=(0, 1))

# Fit and transform the dataset
data_scaled = scaler.fit_transform(data)

# Convert back to a DataFrame for readability
scaled_df = pd.DataFrame(data_scaled, columns=['HouseSize_scaled', 'Bedrooms_scaled'])
print(scaled_df)

In this code, each column is scaled so that its values fall between 0 and 1.

2. Mean Normalization
Mean normalization not only scales the values but also centers them around zero. For feature 𝑥1, if the mean 𝜇1 is 600, you calculate:
$$
x_{1,\text{norm}} = \frac{x_1 – \mu_1}{x_{1,\text{max}} – x_{1,\text{min}}}
$$

​

This results in values that might range from approximately –0.18 to 0.82.

Here’s a Python example to perform mean normalization manually:

import numpy as np

# Original feature values for House Size
house_sizes = np.array([300, 600, 1200, 1800, 2000])

# Calculate mean, min, and max
mu = np.mean(house_sizes)
min_val = np.min(house_sizes)
max_val = np.max(house_sizes)

# Perform mean normalization
house_sizes_norm = (house_sizes - mu) / (max_val - min_val)
print("Mean Normalized House Sizes:", house_sizes_norm)

You can apply a similar transformation to any feature to center it around zero.

3. Z‑Score Normalization
Z‑score normalization (or standardization) transforms data by subtracting the mean and dividing by the standard deviation. For a feature 𝑥1 with mean 𝜇1 and standard deviation 𝜎1, each value is transformed as:

$$
x_{1,\text{z}} = \frac{x_1 – \mu_1}{\sigma_1}
$$

This standardizes x1 so that it has a mean of 0 and a standard deviation of 1.

Below is a Python example using scikit-learn’s StandardScaler:

from sklearn.preprocessing import StandardScaler

# Example dataset (same house sizes)
house_sizes = np.array([[300], [600], [1200], [1800], [2000]])

# Initialize the StandardScaler
scaler = StandardScaler()

# Fit and transform the data
house_sizes_standardized = scaler.fit_transform(house_sizes)

print("Z-Score Normalized House Sizes:")
print(house_sizes_standardized)

Visualizing Scaled Data
When you plot your scaled features (whether using simple division, mean normalization, or Z‑score normalization), you’ll notice that the data points become more comparable. A typical rule of thumb is to aim for a feature range of roughly –1 to +1. However, small deviations—such as –0.3 to +0.3 or even –3 to +3—are perfectly acceptable depending on your application.

Final Thoughts
Feature scaling is an almost universal preprocessing step that can improve the performance of many machine learning algorithms—especially those sensitive to the magnitudes of input values like gradient descent and distance-based models. Whether you choose simple scaling, mean normalization, or Z‑score normalization depends on your data’s characteristics and the specific needs of your model.

By ensuring that each feature contributes fairly, you pave the way for faster convergence and a more robust model. In practice, when in doubt, scaling your features rarely hurts—and may even significantly enhance your model’s performance.

Happy scaling!

Scroll to Top