In [ ]:
!pip install pyswarm
Collecting pyswarm Downloading pyswarm-0.6.tar.gz (4.3 kB) Preparing metadata (setup.py) ... done Requirement already satisfied: numpy in /usr/local/lib/python3.11/dist-packages (from pyswarm) (2.0.2) Building wheels for collected packages: pyswarm Building wheel for pyswarm (setup.py) ... done Created wheel for pyswarm: filename=pyswarm-0.6-py3-none-any.whl size=4463 sha256=20fd3a6038130753575b0deb21daccd1d63b09d26ad3e9d99817bbe3be28329a Stored in directory: /root/.cache/pip/wheels/bb/4f/ec/8970b83323e16aa95034da175454843947376614d6d5e9627f Successfully built pyswarm Installing collected packages: pyswarm Successfully installed pyswarm-0.6
In [ ]:
!pip install autokeras
Requirement already satisfied: autokeras in /usr/local/lib/python3.11/dist-packages (2.0.0) Requirement already satisfied: packaging in /usr/local/lib/python3.11/dist-packages (from autokeras) (24.2) Requirement already satisfied: keras-tuner>=1.4.0 in /usr/local/lib/python3.11/dist-packages (from autokeras) (1.4.7) Requirement already satisfied: keras-nlp>=0.8.0 in /usr/local/lib/python3.11/dist-packages (from autokeras) (0.18.1) Requirement already satisfied: keras>=3.0.0 in /usr/local/lib/python3.11/dist-packages (from autokeras) (3.8.0) Requirement already satisfied: dm-tree in /usr/local/lib/python3.11/dist-packages (from autokeras) (0.1.9) Requirement already satisfied: absl-py in /usr/local/lib/python3.11/dist-packages (from keras>=3.0.0->autokeras) (1.4.0) Requirement already satisfied: numpy in /usr/local/lib/python3.11/dist-packages (from keras>=3.0.0->autokeras) (2.0.2) Requirement already satisfied: rich in /usr/local/lib/python3.11/dist-packages (from keras>=3.0.0->autokeras) (13.9.4) Requirement already satisfied: namex in /usr/local/lib/python3.11/dist-packages (from keras>=3.0.0->autokeras) (0.0.9) Requirement already satisfied: h5py in /usr/local/lib/python3.11/dist-packages (from keras>=3.0.0->autokeras) (3.13.0) Requirement already satisfied: optree in /usr/local/lib/python3.11/dist-packages (from keras>=3.0.0->autokeras) (0.15.0) Requirement already satisfied: ml-dtypes in /usr/local/lib/python3.11/dist-packages (from keras>=3.0.0->autokeras) (0.5.1) Requirement already satisfied: keras-hub==0.18.1 in /usr/local/lib/python3.11/dist-packages (from keras-nlp>=0.8.0->autokeras) (0.18.1) Requirement already satisfied: regex in /usr/local/lib/python3.11/dist-packages (from keras-hub==0.18.1->keras-nlp>=0.8.0->autokeras) (2024.11.6) Requirement already satisfied: kagglehub in /usr/local/lib/python3.11/dist-packages (from keras-hub==0.18.1->keras-nlp>=0.8.0->autokeras) (0.3.12) Requirement already satisfied: tensorflow-text in /usr/local/lib/python3.11/dist-packages (from keras-hub==0.18.1->keras-nlp>=0.8.0->autokeras) (2.18.1) Requirement already satisfied: requests in /usr/local/lib/python3.11/dist-packages (from keras-tuner>=1.4.0->autokeras) (2.32.3) Requirement already satisfied: kt-legacy in /usr/local/lib/python3.11/dist-packages (from keras-tuner>=1.4.0->autokeras) (1.0.5) Requirement already satisfied: attrs>=18.2.0 in /usr/local/lib/python3.11/dist-packages (from dm-tree->autokeras) (25.3.0) Requirement already satisfied: wrapt>=1.11.2 in /usr/local/lib/python3.11/dist-packages (from dm-tree->autokeras) (1.17.2) Requirement already satisfied: typing-extensions>=4.5.0 in /usr/local/lib/python3.11/dist-packages (from optree->keras>=3.0.0->autokeras) (4.13.2) Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.11/dist-packages (from requests->keras-tuner>=1.4.0->autokeras) (3.4.1) Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.11/dist-packages (from requests->keras-tuner>=1.4.0->autokeras) (3.10) Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.11/dist-packages (from requests->keras-tuner>=1.4.0->autokeras) (2.4.0) Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.11/dist-packages (from requests->keras-tuner>=1.4.0->autokeras) (2025.4.26) Requirement already satisfied: markdown-it-py>=2.2.0 in /usr/local/lib/python3.11/dist-packages (from rich->keras>=3.0.0->autokeras) (3.0.0) Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /usr/local/lib/python3.11/dist-packages (from rich->keras>=3.0.0->autokeras) (2.19.1) Requirement already satisfied: mdurl~=0.1 in /usr/local/lib/python3.11/dist-packages (from markdown-it-py>=2.2.0->rich->keras>=3.0.0->autokeras) (0.1.2) Requirement already satisfied: pyyaml in /usr/local/lib/python3.11/dist-packages (from kagglehub->keras-hub==0.18.1->keras-nlp>=0.8.0->autokeras) (6.0.2) Requirement already satisfied: tqdm in /usr/local/lib/python3.11/dist-packages (from kagglehub->keras-hub==0.18.1->keras-nlp>=0.8.0->autokeras) (4.67.1) Requirement already satisfied: tensorflow<2.19,>=2.18.0 in /usr/local/lib/python3.11/dist-packages (from tensorflow-text->keras-hub==0.18.1->keras-nlp>=0.8.0->autokeras) (2.18.1) Requirement already satisfied: astunparse>=1.6.0 in /usr/local/lib/python3.11/dist-packages (from tensorflow<2.19,>=2.18.0->tensorflow-text->keras-hub==0.18.1->keras-nlp>=0.8.0->autokeras) (1.6.3) Requirement already satisfied: flatbuffers>=24.3.25 in /usr/local/lib/python3.11/dist-packages (from tensorflow<2.19,>=2.18.0->tensorflow-text->keras-hub==0.18.1->keras-nlp>=0.8.0->autokeras) (25.2.10) Requirement already satisfied: gast!=0.5.0,!=0.5.1,!=0.5.2,>=0.2.1 in /usr/local/lib/python3.11/dist-packages (from tensorflow<2.19,>=2.18.0->tensorflow-text->keras-hub==0.18.1->keras-nlp>=0.8.0->autokeras) (0.6.0) Requirement already satisfied: google-pasta>=0.1.1 in /usr/local/lib/python3.11/dist-packages (from tensorflow<2.19,>=2.18.0->tensorflow-text->keras-hub==0.18.1->keras-nlp>=0.8.0->autokeras) (0.2.0) Requirement already satisfied: libclang>=13.0.0 in /usr/local/lib/python3.11/dist-packages (from tensorflow<2.19,>=2.18.0->tensorflow-text->keras-hub==0.18.1->keras-nlp>=0.8.0->autokeras) (18.1.1) Requirement already satisfied: opt-einsum>=2.3.2 in /usr/local/lib/python3.11/dist-packages (from tensorflow<2.19,>=2.18.0->tensorflow-text->keras-hub==0.18.1->keras-nlp>=0.8.0->autokeras) (3.4.0) Requirement already satisfied: protobuf!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<6.0.0dev,>=3.20.3 in /usr/local/lib/python3.11/dist-packages (from tensorflow<2.19,>=2.18.0->tensorflow-text->keras-hub==0.18.1->keras-nlp>=0.8.0->autokeras) (5.29.4) Requirement already satisfied: setuptools in /usr/local/lib/python3.11/dist-packages (from tensorflow<2.19,>=2.18.0->tensorflow-text->keras-hub==0.18.1->keras-nlp>=0.8.0->autokeras) (75.2.0) Requirement already satisfied: six>=1.12.0 in /usr/local/lib/python3.11/dist-packages (from tensorflow<2.19,>=2.18.0->tensorflow-text->keras-hub==0.18.1->keras-nlp>=0.8.0->autokeras) (1.17.0) Requirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.11/dist-packages (from tensorflow<2.19,>=2.18.0->tensorflow-text->keras-hub==0.18.1->keras-nlp>=0.8.0->autokeras) (3.0.1) Requirement already satisfied: grpcio<2.0,>=1.24.3 in /usr/local/lib/python3.11/dist-packages (from tensorflow<2.19,>=2.18.0->tensorflow-text->keras-hub==0.18.1->keras-nlp>=0.8.0->autokeras) (1.71.0) Requirement already satisfied: tensorboard<2.19,>=2.18 in /usr/local/lib/python3.11/dist-packages (from tensorflow<2.19,>=2.18.0->tensorflow-text->keras-hub==0.18.1->keras-nlp>=0.8.0->autokeras) (2.18.0) Requirement already satisfied: tensorflow-io-gcs-filesystem>=0.23.1 in /usr/local/lib/python3.11/dist-packages (from tensorflow<2.19,>=2.18.0->tensorflow-text->keras-hub==0.18.1->keras-nlp>=0.8.0->autokeras) (0.37.1) Requirement already satisfied: wheel<1.0,>=0.23.0 in /usr/local/lib/python3.11/dist-packages (from astunparse>=1.6.0->tensorflow<2.19,>=2.18.0->tensorflow-text->keras-hub==0.18.1->keras-nlp>=0.8.0->autokeras) (0.45.1) Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.11/dist-packages (from tensorboard<2.19,>=2.18->tensorflow<2.19,>=2.18.0->tensorflow-text->keras-hub==0.18.1->keras-nlp>=0.8.0->autokeras) (3.8) Requirement already satisfied: tensorboard-data-server<0.8.0,>=0.7.0 in /usr/local/lib/python3.11/dist-packages (from tensorboard<2.19,>=2.18->tensorflow<2.19,>=2.18.0->tensorflow-text->keras-hub==0.18.1->keras-nlp>=0.8.0->autokeras) (0.7.2) Requirement already satisfied: werkzeug>=1.0.1 in /usr/local/lib/python3.11/dist-packages (from tensorboard<2.19,>=2.18->tensorflow<2.19,>=2.18.0->tensorflow-text->keras-hub==0.18.1->keras-nlp>=0.8.0->autokeras) (3.1.3) Requirement already satisfied: MarkupSafe>=2.1.1 in /usr/local/lib/python3.11/dist-packages (from werkzeug>=1.0.1->tensorboard<2.19,>=2.18->tensorflow<2.19,>=2.18.0->tensorflow-text->keras-hub==0.18.1->keras-nlp>=0.8.0->autokeras) (3.0.2)
In [ ]:
# Heart Disease Prediction Using PSO and AutoKeras
# 1. Introduction
# This project focuses on predicting the presence of heart disease.
# We utilize Particle Swarm Optimization (PSO) to tune a Random Forest Classifier
# and use Neural Architecture Search (NAS) via AutoKeras to discover optimal neural networks.
In [ ]:
# 2. Importing Libraries
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score, confusion_matrix, classification_report
from pyswarm import pso
import autokeras as ak
import tensorflow as tf
In [ ]:
# 3. Loading Dataset
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
# Load the dataset
df = pd.read_csv('heart.csv')
df.head()
Out[ ]:
| age | sex | cp | trestbps | chol | fbs | restecg | thalach | exang | oldpeak | slope | ca | thal | target | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 52 | 1 | 0 | 125 | 212 | 0 | 1 | 168 | 0 | 1.0 | 2 | 2 | 3 | 0 |
| 1 | 53 | 1 | 0 | 140 | 203 | 1 | 0 | 155 | 1 | 3.1 | 0 | 0 | 3 | 0 |
| 2 | 70 | 1 | 0 | 145 | 174 | 0 | 1 | 125 | 1 | 2.6 | 0 | 0 | 3 | 0 |
| 3 | 61 | 1 | 0 | 148 | 203 | 0 | 1 | 161 | 0 | 0.0 | 2 | 1 | 3 | 0 |
| 4 | 62 | 0 | 0 | 138 | 294 | 1 | 1 | 106 | 0 | 1.9 | 1 | 3 | 2 | 0 |
In [ ]:
# 4. Data Preprocessing
# Checking for missing values
df.isnull().sum()
# Feature and Target split
X = df.drop(['target'], axis=1).values
y = df['target'].values
print(X.shape, y.shape)
# Normalize features
from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)
# Prepare training and testing sets
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(
X_scaled, y, test_size=0.2, random_state=42, stratify=y
)
print(X_train.shape, X_test.shape)
(1025, 13) (1025,) (820, 13) (205, 13)
In [ ]:
# Install NiaPy package for metaheuristic optimization
!pip install niapy --quiet
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.0/61.0 kB 2.4 MB/s eta 0:00:00 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 188.0/188.0 kB 7.2 MB/s eta 0:00:00 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 18.3/18.3 MB 87.1 MB/s eta 0:00:00 ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. google-colab 1.0.0 requires pandas==2.2.2, but you have pandas 2.2.3 which is incompatible. tensorflow-decision-forests 1.11.0 requires tensorflow==2.18.0, but you have tensorflow 2.18.1 which is incompatible. thinc 8.3.6 requires numpy<3.0.0,>=2.0.0, but you have numpy 1.26.4 which is incompatible.
In [ ]:
# Define Search Space and Optimization Task
import numpy as np
import tensorflow as tf
from niapy.task import Task
from niapy.problems import Problem
from niapy.algorithms.basic import GeneticAlgorithm
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import cross_val_score
# Initialize fitness tracking
fitness_history = []
# Custom Problem Class for Random Forest Optimization
class RandomForestOptimization(Problem):
def __init__(self):
super().__init__(dimension=3, lower=[50, 2, 2], upper=[200, 50, 10])
def _evaluate(self, solution):
solution = [int(round(x)) for x in solution]
n_estimators, max_depth, min_samples_split = solution
max_depth = max_depth if max_depth > 0 else None
model = RandomForestClassifier(
n_estimators=n_estimators,
max_depth=max_depth,
min_samples_split=min_samples_split,
random_state=42,
n_jobs=-1
)
score = cross_val_score(model, X_train, y_train, cv=5, scoring="accuracy").mean()
neg_score = -score
fitness_history.append(neg_score)
return neg_score
# Define optimization task
task = Task(problem=RandomForestOptimization(), max_iters=10)
# Define and run Genetic Algorithm
print("Running Genetic Algorithm...")
ga_algorithm = GeneticAlgorithm(population_size=10, crossover_probability=0.8, mutation_probability=0.1)
ga_solution, ga_fitness = ga_algorithm.run(task)
# Extract best hyperparameters
best_n_estimators = int(ga_solution[0])
best_max_depth = int(ga_solution[1]) if ga_solution[1] > 0 else None
best_min_samples_split = int(ga_solution[2])
print("\nBest Hyperparameters Found:")
print(f"n_estimators: {best_n_estimators}")
print(f"max_depth: {best_max_depth}")
print(f"min_samples_split: {best_min_samples_split}")
print(f"Best Accuracy: {-ga_fitness:.4f}")
Running Genetic Algorithm... Best Hyperparameters Found: n_estimators: 176 max_depth: 16 min_samples_split: 2 Best Accuracy: 0.9817
In [ ]:
# Optimization Curve
plt.figure(figsize=(8, 5))
plt.plot(range(1, len(fitness_history) + 1), fitness_history, marker='o', linestyle='-', color='blue')
plt.xlabel("Iteration")
plt.ylabel("Fitness Value (Negative Accuracy)")
plt.title("Genetic Algorithm Optimization Curve")
plt.grid(True)
plt.show()
In [ ]:
# Build the ML Model with Optimized Parameters
from sklearn.metrics import accuracy_score, confusion_matrix, classification_report
import seaborn as sns
# Train final Random Forest
final_rf = RandomForestClassifier(
n_estimators=best_n_estimators,
max_depth=best_max_depth,
min_samples_split=best_min_samples_split,
random_state=42,
n_jobs=-1
)
final_rf.fit(X_train, y_train)
# Make predictions
y_pred = final_rf.predict(X_test)
# Evaluate Accuracy
accuracy = accuracy_score(y_test, y_pred)
print(f"\nFinal Random Forest Accuracy: {accuracy:.4f}")
# Compute Confusion Matrix
cm = confusion_matrix(y_test, y_pred)
# Plot Confusion Matrix
plt.figure(figsize=(8,6))
sns.heatmap(cm, annot=True, fmt="d", cmap="Blues")
plt.xlabel("Predicted Label")
plt.ylabel("True Label")
plt.title("Confusion Matrix - Optimized Random Forest (Heart Disease)")
plt.show()
# Print Classification Report
print("\nClassification Report:")
print(classification_report(y_test, y_pred, target_names=["No Disease", "Disease"]))
Final Random Forest Accuracy: 1.0000
Classification Report:
precision recall f1-score support
No Disease 1.00 1.00 1.00 100
Disease 1.00 1.00 1.00 105
accuracy 1.00 205
macro avg 1.00 1.00 1.00 205
weighted avg 1.00 1.00 1.00 205
In [ ]:
# Install AutoKeras for NAS
!pip install --upgrade autokeras tensorflow pandas scikit-learn --quiet
import autokeras as ak
In [ ]:
# Neural Architecture Search (NAS) Using Auto-Keras
# Define Auto-Keras Model
clf = ak.AutoModel(
inputs=ak.Input(),
outputs=ak.ClassificationHead(num_classes=2),
max_trials=10,
overwrite=True
)
# Train Auto-Keras Model
clf.fit(X_train, y_train, epochs=10)
# Evaluate Auto-Keras Model
test_accuracy = clf.evaluate(X_test, y_test)
print("\nAuto-Keras Test Accuracy:", test_accuracy)
# Export Best Model
best_model = clf.export_model()
# Model Summary
print("\nBest Model Structure:")
best_model.summary()
Trial 10 Complete [00h 00m 07s] val_loss: 0.12217776477336884 Best val_loss So Far: 0.08254747092723846 Total elapsed time: 00h 00m 57s Epoch 1/10 26/26 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.7736 - loss: 0.5334 Epoch 2/10 26/26 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step - accuracy: 0.8873 - loss: 0.2981 Epoch 3/10 26/26 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.9149 - loss: 0.2502 Epoch 4/10 26/26 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.9105 - loss: 0.2095 Epoch 5/10 26/26 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.9321 - loss: 0.1672 Epoch 6/10 26/26 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.9427 - loss: 0.1357 Epoch 7/10 26/26 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.9688 - loss: 0.0965 Epoch 8/10 26/26 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.9862 - loss: 0.0671 Epoch 9/10 26/26 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.9875 - loss: 0.0445 Epoch 10/10 26/26 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.9917 - loss: 0.0326
/usr/local/lib/python3.11/dist-packages/keras/src/saving/saving_lib.py:757: UserWarning: Skipping variable loading for optimizer 'adam', because it has 2 variables whereas the saved optimizer has 18 variables. saveable.load_own_variables(weights_store.get(inner_path))
7/7 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.9801 - loss: 0.0790 Auto-Keras Test Accuracy: [0.09587649255990982, 0.9707317352294922] Best Model Structure:
Model: "functional"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓ ┃ Layer (type) ┃ Output Shape ┃ Param # ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩ │ input_layer (InputLayer) │ (None, 13) │ 0 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ cast (Cast) │ (None, 13) │ 0 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ cast_to_float32 (CastToFloat32) │ (None, 13) │ 0 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ dense (Dense) │ (None, 128) │ 1,792 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ re_lu (ReLU) │ (None, 128) │ 0 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ dense_1 (Dense) │ (None, 512) │ 66,048 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ re_lu_1 (ReLU) │ (None, 512) │ 0 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ dense_2 (Dense) │ (None, 32) │ 16,416 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ re_lu_2 (ReLU) │ (None, 32) │ 0 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ dropout (Dropout) │ (None, 32) │ 0 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ dense_3 (Dense) │ (None, 1) │ 33 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ classification_head_1 │ (None, 1) │ 0 │ │ (Activation) │ │ │ └─────────────────────────────────┴────────────────────────┴───────────────┘
Total params: 84,289 (329.25 KB)
Trainable params: 84,289 (329.25 KB)
Non-trainable params: 0 (0.00 B)
In [ ]: