Mastering Time Series Analysis with Scikit-learn in Python

Introduction to Time Series Analysis

Time series analysis is a crucial skill for data scientists and analysts working with sequential data. Whether you're predicting stock prices, analyzing weather patterns, or forecasting sales, understanding how to handle time-dependent data is essential. In this blog post, we'll explore how to perform time series analysis using Scikit-learn in Python.

Setting Up Your Environment

Before we begin, make sure you have the necessary libraries installed:

import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error
import matplotlib.pyplot as plt

Loading and Preprocessing Time Series Data

Let's start by loading a sample time series dataset:


# Load data
data = pd.read_csv('time_series_data.csv')
data['Date'] = pd.to_datetime(data['Date'])
data.set_index('Date', inplace=True)

Next, we'll create features from the time series:

def create_features(df):
    df['dayofweek'] = df.index.dayofweek
    df['quarter'] = df.index.quarter
    df['month'] = df.index.month
    df['year'] = df.index.year
    df['dayofyear'] = df.index.dayofyear
    return df

data = create_features(data)

Splitting the Data

When working with time series, it's important to maintain the temporal order of the data:

train = data.loc[data.index < '2022-01-01']
test = data.loc[data.index >= '2022-01-01']

FEATURES = ['dayofweek', 'quarter', 'month', 'year', 'dayofyear']
TARGET = 'Sales'

X_train = train[FEATURES]
y_train = train[TARGET]

X_test = test[FEATURES]
y_test = test[TARGET]

Applying Linear Regression

Let's start with a simple linear regression model:

scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)

model = LinearRegression()
model.fit(X_train_scaled, y_train)

train_predictions = model.predict(X_train_scaled)
test_predictions = model.predict(X_test_scaled)

train_mse = mean_squared_error(y_train, train_predictions)
test_mse = mean_squared_error(y_test, test_predictions)

print(f"Train MSE: {train_mse}")
print(f"Test MSE: {test_mse}")

Visualizing the Results

Let's plot our predictions against the actual values:

plt.figure(figsize=(12, 6))
plt.plot(train.index, y_train, label='Train Actual')
plt.plot(train.index, train_predictions, label='Train Predicted')
plt.plot(test.index, y_test, label='Test Actual')
plt.plot(test.index, test_predictions, label='Test Predicted')
plt.legend()
plt.title('Time Series Forecast')
plt.show()

Advanced Techniques

While linear regression provides a good baseline, more advanced techniques can improve our predictions:

Random Forest Regressor

from sklearn.ensemble import RandomForestRegressor

rf_model = RandomForestRegressor(n_estimators=100, random_state=42)
rf_model.fit(X_train_scaled, y_train)

rf_train_predictions = rf_model.predict(X_train_scaled)
rf_test_predictions = rf_model.predict(X_test_scaled)

rf_train_mse = mean_squared_error(y_train, rf_train_predictions)
rf_test_mse = mean_squared_error(y_test, rf_test_predictions)

print(f"Random Forest Train MSE: {rf_train_mse}")
print(f"Random Forest Test MSE: {rf_test_mse}")

ARIMA with Scikit-learn

While Scikit-learn doesn't have built-in ARIMA models, we can combine it with statsmodels:

from statsmodels.tsa.arima.model import ARIMA
from sklearn.base import BaseEstimator, RegressorMixin

class ARIMAWrapper(BaseEstimator, RegressorMixin):
    def __init__(self, order=(1,1,1)):
        self.order = order
        
    def fit(self, X, y):
        self.model = ARIMA(y, order=self.order)
        self.result = self.model.fit()
        return self
    
    def predict(self, X):
        return self.result.forecast(steps=len(X))

arima_model = ARIMAWrapper(order=(1,1,1))
arima_model.fit(X_train, y_train)

arima_predictions = arima_model.predict(X_test)

arima_mse = mean_squared_error(y_test, arima_predictions)
print(f"ARIMA Test MSE: {arima_mse}")

Feature Importance

Understanding which features contribute most to our predictions can provide valuable insights:

feature_importance = pd.DataFrame({'feature': FEATURES, 'importance': rf_model.feature_importances_})
feature_importance = feature_importance.sort_values('importance', ascending=False)

plt.figure(figsize=(10, 6))
plt.bar(feature_importance['feature'], feature_importance['importance'])
plt.title('Feature Importance')
plt.xlabel('Features')
plt.ylabel('Importance')
plt.xticks(rotation=45)
plt.show()

Cross-Validation for Time Series

Traditional cross-validation doesn't work well for time series due to temporal dependencies. Instead, we can use time series cross-validation:

from sklearn.model_selection import TimeSeriesSplit

tscv = TimeSeriesSplit(n_splits=5)

cv_scores = []
for train_index, val_index in tscv.split(X_train):
    X_train_cv, X_val_cv = X_train.iloc[train_index], X_train.iloc[val_index]
    y_train_cv, y_val_cv = y_train.iloc[train_index], y_train.iloc[val_index]
    
    model = RandomForestRegressor(n_estimators=100, random_state=42)
    model.fit(X_train_cv, y_train_cv)
    
    predictions = model.predict(X_val_cv)
    mse = mean_squared_error(y_val_cv, predictions)
    cv_scores.append(mse)

print(f"Cross-validation MSE scores: {cv_scores}")
print(f"Average MSE: {np.mean(cv_scores)}")

Conclusion

In this blog post, we've explored various techniques for time series analysis using Scikit-learn in Python. We've covered data preprocessing, feature engineering, model building, and evaluation. By leveraging these tools and techniques, you can build powerful time series models for a wide range of applications.

Remember, time series analysis is a vast field, and there's always more to learn. Experiment with different models, feature engineering techniques, and preprocessing steps to find what works best for your specific time series problem.

Level Up Your Skills with Xperto-AI