# Implementing LSTM RNN using Keras and TensorFlow

In this Article we will implement LTSM RNN model using Keras and TensorFlow. LSTM model was discussed in my previous article , but we will make this implementation easier by using Keras framework.

We will be using Jena Climate dataset recorded by the [Max Planck Institute for Biogeochemistry. The dataset consists of 14 features su](https://www.bgc-jena.mpg.de/wetter/)ch as temperature, pressure, humidity etc, recorded once per 10 minutes.

**Lets Load the libraries**

```python
from urllib.request import urlretrieve
import numpy as np
import os
from tensorflow import keras
```

**Load the Data**

```python
url=("https://s3.amazonaws.com/keras-datasets/jena_climate_2009_2016.csv.zip")
filename = "jena_climate_2009_2016.csv.zip"
path, headers = urlretrieve(url, filename)
```

**Lets inspect the data**

```python

fname = os.path.join("jena_climate_2009_2016.csv")
with open(fname) as f:
    data = f.read()
lines = data.split("\n")
header = lines[0].split(",")
lines = lines[1:]
print(header)
print(len(lines))
```

```python
['"Date Time"', '"p (mbar)"', '"T (degC)"', '"Tpot (K)"', '"Tdew (degC)"', '"rh (%)"', '"VPmax (mbar)"', '"VPact (mbar)"', '"VPdef (mbar)"', '"sh (g/kg)"', '"H2OC (mmol/mol)"', '"rho (g/m**3)"', '"wv (m/s)"', '"max. wv (m/s)"', '"wd (deg)"']
420451
```

We will Split the data into two numpy arrays as temperature and raw\_data

```python
temperature = np.zeros((len(lines),))
raw_data = np.zeros((len(lines), len(header) - 1))
for i, line in enumerate(lines):
    values = [float(x) for x in line.split(",")[1:]]
    temperature[i] = values[1]
    raw_data[i, :] = values[:]
```

Shape of the raw\_data is

```python
raw_data.shape
(420451, 14)
```

So we have 420451 samples containing 14 features of data

**Plotting the temperature timeseries**

```python
from matplotlib import pyplot as plt
plt.plot(range(len(temperature)), temperature)
```

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1741830966033/d0902862-96fc-4ca3-9d06-7ad91d01619f.png align="center")

**Computing the number of samples we'll use for each data split**

We will split data into 50 % for training data, 25 % validation and test data each.

```python
num_train_samples = int(0.5 * len(raw_data))
num_val_samples = int(0.25 * len(raw_data))
num_test_samples = len(raw_data) - num_train_samples - num_val_samples
print("num_train_samples:", num_train_samples)
print("num_val_samples:", num_val_samples)
print("num_test_samples:", num_test_samples)
print("raw_data Shape:", raw_data.shape)
```

```python
num_train_samples: 210225
num_val_samples: 105112
num_test_samples: 105114
raw_data Shape: (420451, 14)
```

**Normalizing the data**

We will normalize the data using mean and standard deviation

```python
mean = raw_data[:num_train_samples].mean(axis=0)
raw_data -= mean
std = raw_data[:num_train_samples].std(axis=0)
raw_data /= std
```

We will create Timeseries dataser by using keras API keras.utils.timeseries\_dataset\_from\_array

We will keep the sequence length of 120 hr and use batch size of 256 form the samples. We will forecast temperature for the next 24 hr.

**Instantiating datasets for training, validation, and testing**

```python
sampling_rate = 6
sequence_length = 120
delay = sampling_rate * (sequence_length + 24 - 1)
batch_size = 256

train_dataset = keras.utils.timeseries_dataset_from_array(
    raw_data[:-delay],
    targets=temperature[delay:],
    sampling_rate=sampling_rate,
    sequence_length=sequence_length,
    shuffle=True,
    batch_size=batch_size,
    start_index=0,
    end_index=num_train_samples)
val_dataset = keras.utils.timeseries_dataset_from_array(
    raw_data[:-delay],
    targets=temperature[delay:],
    sampling_rate=sampling_rate,
    sequence_length=sequence_length,
    shuffle=True,
    batch_size=batch_size,
    start_index=num_train_samples,
    end_index=num_train_samples + num_val_samples)
test_dataset = keras.utils.timeseries_dataset_from_array(
    raw_data[:-delay],
    targets=temperature[delay:],
    sampling_rate=sampling_rate,
    sequence_length=sequence_length,
    shuffle=True,
    batch_size=batch_size,
    start_index=num_train_samples + num_val_samples)
```

**Inspecting the output of one of our datasets**

```python
for samples, targets in train_dataset:
    print("samples shape:", samples.shape)
    print("targets shape:", targets.shape)
    break
```

```python
samples shape: (256, 120, 14)
targets shape: (256,)
```

### Lets create A recurrent layer in Keras

**Training and evaluating a dropout-regularized LSTM**

```python
inputs = keras.Input(shape=(sequence_length, raw_data.shape[-1]))
x = layers.LSTM(32, recurrent_dropout=0.25)(inputs)
x = layers.Dropout(0.5)(x)
outputs = layers.Dense(1)(x)
model = keras.Model(inputs, outputs)

callbacks = [
    keras.callbacks.ModelCheckpoint("jena_lstm_dropout.sample",
                                    save_best_only=True)
]
model.compile(optimizer="rmsprop", loss="mse", metrics=["mae"])
history = model.fit(train_dataset,
                    epochs=50,
                    validation_data=val_dataset,
                    callbacks=callbacks)
```

```python
Epoch 48/50
819/819 [==============================] - 66s 80ms/step - loss: 8.9161 - mae: 2.3114 - val_loss: 11.4386 - val_mae: 2.6123
Epoch 49/50
819/819 [==============================] - 68s 83ms/step - loss: 8.8472 - mae: 2.3058 - val_loss: 11.4213 - val_mae: 2.6145
Epoch 50/50
819/819 [==============================] - 65s 79ms/step - loss: 8.8340 - mae: 2.3063 - val_loss: 11.3844 - val_mae: 2.6109
```

### Load the the best saved model

```python
model = keras.models.load_model("jena_lstm_dropout.sample")
print(f"Test MAE: {model.evaluate(test_dataset)[1]:.2f}")
```

```python
405/405 [==============================] - 6s 14ms/step - loss: 10.3629 - mae: 2.5492
Test MAE: 2.55
```

**Lets plot**

```python
import matplotlib.pyplot as plt
loss = history.history["mae"]
val_loss = history.history["val_mae"]
epochs = range(1, len(loss) + 1)
plt.figure()
plt.plot(epochs, loss, "bo", label="Training MAE")
plt.plot(epochs, val_loss, "b", label="Validation MAE")
plt.title("Training and validation MAE")
plt.legend()
plt.show()
```

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1741831585789/c020411c-d03d-4819-82af-2b914972d2bd.png align="center")