In rstudio/keras: R Interface to 'Keras'

Introduction

This example shows how to do timeseries classification from scratch, starting from raw CSV timeseries files on disk. We demonstrate the workflow on the FordA dataset from the UCR/UEA archive.

Setup

library(keras3)
use_backend("jax")

Load the data: the FordA dataset

Dataset description

The dataset we are using here is called FordA. The data comes from the UCR archive. The dataset contains 3601 training instances and another 1320 testing instances. Each timeseries corresponds to a measurement of engine noise captured by a motor sensor. For this task, the goal is to automatically detect the presence of a specific issue with the engine. The problem is a balanced binary classification task. The full description of this dataset can be found here.

Read the TSV data

We will use the FordA_TRAIN file for training and the FordA_TEST file for testing. The simplicity of this dataset allows us to demonstrate effectively how to use ConvNets for timeseries classification. In this file, the first column corresponds to the label.

get_data <- function(path) {
  if(path |> startsWith("https://"))
    path <- get_file(origin = path)  # cache file locally

  data <- readr::read_tsv(
    path, col_names = FALSE,
    # Each row is: one integer (the label),
    # followed by 500 doubles (the timeseries)
    col_types = paste0("i", strrep("d", 500))
  )

  y <- as.matrix(data[[1]])
  x <- as.matrix(data[,-1])
  dimnames(x) <- dimnames(y) <- NULL

  list(x, y)
}

root_url <- "https://raw.githubusercontent.com/hfawaz/cd-diagram/master/FordA/"
c(x_train, y_train) %<-% get_data(paste0(root_url, "FordA_TRAIN.tsv"))
c(x_test, y_test) %<-% get_data(paste0(root_url, "FordA_TEST.tsv"))

str(keras3:::named_list(
  x_train, y_train,
  x_test, y_test
))

## List of 4
##  $ x_train: num [1:3601, 1:500] -0.797 0.805 0.728 -0.234 -0.171 ...
##  $ y_train: int [1:3601, 1] -1 1 -1 -1 -1 1 1 1 1 1 ...
##  $ x_test : num [1:1320, 1:500] -0.14 0.334 0.717 1.24 -1.159 ...
##  $ y_test : int [1:1320, 1] -1 -1 -1 1 -1 1 -1 -1 1 1 ...

Visualize the data

Here we visualize one timeseries example for each class in the dataset.

plot(NULL, main = "Timeseries Data",
     xlab = "Timepoints",  ylab = "Values",
     xlim = c(1, ncol(x_test)),
     ylim = range(x_test))
grid()
lines(x_test[match(-1, y_test), ], col = "blue")
lines(x_test[match( 1, y_test), ], col = "red")
legend("topright", legend=c("label -1", "label 1"), col=c("blue", "red"), lty=1)

Plot of Example Timeseries Data

Standardize the data

Our timeseries are already in a single length (500). However, their values are usually in various ranges. This is not ideal for a neural network; in general we should seek to make the input values normalized. For this specific dataset, the data is already z-normalized: each timeseries sample has a mean equal to zero and a standard deviation equal to one. This type of normalization is very common for timeseries classification problems, see Bagnall et al. (2016).

Note that the timeseries data used here are univariate, meaning we only have one channel per timeseries example. We will therefore transform the timeseries into a multivariate one with one channel using a simple reshaping via numpy. This will allow us to construct a model that is easily applicable to multivariate time series.

dim(x_train) <- c(dim(x_train), 1)
dim(x_test) <- c(dim(x_test), 1)

Finally, in order to use sparse_categorical_crossentropy, we will have to count the number of classes beforehand.

num_classes <- length(unique(y_train))

Now we shuffle the training set because we will be using the validation_split option later when training.

c(x_train, y_train) %<-% listarrays::shuffle_rows(x_train, y_train)
# idx <- sample.int(nrow(x_train))
# x_train %<>% .[idx,, ,drop = FALSE]
# y_train %<>% .[idx,  ,drop = FALSE]

Standardize the labels to positive integers. The expected labels will then be 0 and 1.

y_train[y_train == -1L] <- 0L
y_test[y_test == -1L] <- 0L

Build a model

We build a Fully Convolutional Neural Network originally proposed in this paper. The implementation is based on the TF 2 version provided here. The following hyperparameters (kernel_size, filters, the usage of BatchNorm) were found via random search using KerasTuner.

make_model <- function(input_shape) {
  inputs <- keras_input(input_shape)

  outputs <- inputs |>
    # conv1
    layer_conv_1d(filters = 64, kernel_size = 3, padding = "same") |>
    layer_batch_normalization() |>
    layer_activation_relu() |>
    # conv2
    layer_conv_1d(filters = 64, kernel_size = 3, padding = "same") |>
    layer_batch_normalization() |>
    layer_activation_relu() |>
    # conv3
    layer_conv_1d(filters = 64, kernel_size = 3, padding = "same") |>
    layer_batch_normalization() |>
    layer_activation_relu() |>
    # pooling
    layer_global_average_pooling_1d() |>
    # final output
    layer_dense(num_classes, activation = "softmax")

  keras_model(inputs, outputs)
}

model <- make_model(input_shape = dim(x_train)[-1])

model

## [1mModel: "functional_1"[0m
## ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━┓
## ┃[1m [0m[1mLayer (type)               [0m[1m [0m┃[1m [0m[1mOutput Shape         [0m[1m [0m┃[1m [0m[1m   Param #[0m[1m [0m┃[1m [0m[1mTrai…[0m[1m [0m┃
## ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━┩
## │ input_layer ([38;5;33mInputLayer[0m)    │ ([38;5;45mNone[0m, [38;5;34m500[0m, [38;5;34m1[0m)        │          [38;5;34m0[0m │   [1m-[0m   │
## ├─────────────────────────────┼───────────────────────┼────────────┼───────┤
## │ conv1d ([38;5;33mConv1D[0m)             │ ([38;5;45mNone[0m, [38;5;34m500[0m, [38;5;34m64[0m)       │        [38;5;34m256[0m │   [1;38;5;34mY[0m   │
## ├─────────────────────────────┼───────────────────────┼────────────┼───────┤
## │ batch_normalization         │ ([38;5;45mNone[0m, [38;5;34m500[0m, [38;5;34m64[0m)       │        [38;5;34m256[0m │   [1;38;5;34mY[0m   │
## │ ([38;5;33mBatchNormalization[0m)        │                       │            │       │
## ├─────────────────────────────┼───────────────────────┼────────────┼───────┤
## │ re_lu ([38;5;33mReLU[0m)                │ ([38;5;45mNone[0m, [38;5;34m500[0m, [38;5;34m64[0m)       │          [38;5;34m0[0m │   [1m-[0m   │
## ├─────────────────────────────┼───────────────────────┼────────────┼───────┤
## │ conv1d_1 ([38;5;33mConv1D[0m)           │ ([38;5;45mNone[0m, [38;5;34m500[0m, [38;5;34m64[0m)       │     [38;5;34m12,352[0m │   [1;38;5;34mY[0m   │
## ├─────────────────────────────┼───────────────────────┼────────────┼───────┤
## │ batch_normalization_1       │ ([38;5;45mNone[0m, [38;5;34m500[0m, [38;5;34m64[0m)       │        [38;5;34m256[0m │   [1;38;5;34mY[0m   │
## │ ([38;5;33mBatchNormalization[0m)        │                       │            │       │
## ├─────────────────────────────┼───────────────────────┼────────────┼───────┤
## │ re_lu_1 ([38;5;33mReLU[0m)              │ ([38;5;45mNone[0m, [38;5;34m500[0m, [38;5;34m64[0m)       │          [38;5;34m0[0m │   [1m-[0m   │
## ├─────────────────────────────┼───────────────────────┼────────────┼───────┤
## │ conv1d_2 ([38;5;33mConv1D[0m)           │ ([38;5;45mNone[0m, [38;5;34m500[0m, [38;5;34m64[0m)       │     [38;5;34m12,352[0m │   [1;38;5;34mY[0m   │
## ├─────────────────────────────┼───────────────────────┼────────────┼───────┤
## │ batch_normalization_2       │ ([38;5;45mNone[0m, [38;5;34m500[0m, [38;5;34m64[0m)       │        [38;5;34m256[0m │   [1;38;5;34mY[0m   │
## │ ([38;5;33mBatchNormalization[0m)        │                       │            │       │
## ├─────────────────────────────┼───────────────────────┼────────────┼───────┤
## │ re_lu_2 ([38;5;33mReLU[0m)              │ ([38;5;45mNone[0m, [38;5;34m500[0m, [38;5;34m64[0m)       │          [38;5;34m0[0m │   [1m-[0m   │
## ├─────────────────────────────┼───────────────────────┼────────────┼───────┤
## │ global_average_pooling1d    │ ([38;5;45mNone[0m, [38;5;34m64[0m)            │          [38;5;34m0[0m │   [1m-[0m   │
## │ ([38;5;33mGlobalAveragePooling1D[0m)    │                       │            │       │
## ├─────────────────────────────┼───────────────────────┼────────────┼───────┤
## │ dense ([38;5;33mDense[0m)               │ ([38;5;45mNone[0m, [38;5;34m2[0m)             │        [38;5;34m130[0m │   [1;38;5;34mY[0m   │
## └─────────────────────────────┴───────────────────────┴────────────┴───────┘
## [1m Total params: [0m[38;5;34m25,858[0m (101.01 KB)
## [1m Trainable params: [0m[38;5;34m25,474[0m (99.51 KB)
## [1m Non-trainable params: [0m[38;5;34m384[0m (1.50 KB)

plot(model, show_shapes = TRUE)

plot of chunk unnamed-chunk-9

Train the model

epochs <- 500
batch_size <- 32

callbacks <- c(
  callback_model_checkpoint(
    "best_model.keras", save_best_only = TRUE,
    monitor = "val_loss"
  ),
  callback_reduce_lr_on_plateau(
    monitor = "val_loss", factor = 0.5,
    patience = 20, min_lr = 0.0001
  ),
  callback_early_stopping(
    monitor = "val_loss", patience = 50,
    verbose = 1
  )
)


model |> compile(
  optimizer = "adam",
  loss = "sparse_categorical_crossentropy",
  metrics = "sparse_categorical_accuracy"
)

history <- model |> fit(
  x_train, y_train,
  batch_size = batch_size,
  epochs = epochs,
  callbacks = callbacks,
  validation_split = 0.2
)

## Epoch 1/500
## 90/90 - 2s - 24ms/step - loss: 0.5557 - sparse_categorical_accuracy: 0.7073 - val_loss: 0.8517 - val_sparse_categorical_accuracy: 0.4896 - learning_rate: 0.0010
## Epoch 2/500
## 90/90 - 1s - 6ms/step - loss: 0.4850 - sparse_categorical_accuracy: 0.7611 - val_loss: 0.9464 - val_sparse_categorical_accuracy: 0.4896 - learning_rate: 0.0010
## Epoch 3/500
## 90/90 - 0s - 2ms/step - loss: 0.4713 - sparse_categorical_accuracy: 0.7705 - val_loss: 0.7593 - val_sparse_categorical_accuracy: 0.4896 - learning_rate: 0.0010
## Epoch 4/500
## 90/90 - 0s - 2ms/step - loss: 0.4239 - sparse_categorical_accuracy: 0.7885 - val_loss: 0.6721 - val_sparse_categorical_accuracy: 0.5021 - learning_rate: 0.0010
## Epoch 5/500
## 90/90 - 0s - 2ms/step - loss: 0.4218 - sparse_categorical_accuracy: 0.7885 - val_loss: 0.5647 - val_sparse_categorical_accuracy: 0.7198 - learning_rate: 0.0010
## Epoch 6/500
## 90/90 - 0s - 2ms/step - loss: 0.4060 - sparse_categorical_accuracy: 0.8028 - val_loss: 0.4563 - val_sparse_categorical_accuracy: 0.8141 - learning_rate: 0.0010
## Epoch 7/500
## 90/90 - 0s - 2ms/step - loss: 0.4032 - sparse_categorical_accuracy: 0.8017 - val_loss: 0.4209 - val_sparse_categorical_accuracy: 0.7892 - learning_rate: 0.0010
## Epoch 8/500
## 90/90 - 0s - 2ms/step - loss: 0.3920 - sparse_categorical_accuracy: 0.8003 - val_loss: 0.3938 - val_sparse_categorical_accuracy: 0.8239 - learning_rate: 0.0010
## Epoch 9/500
## 90/90 - 0s - 2ms/step - loss: 0.3892 - sparse_categorical_accuracy: 0.8135 - val_loss: 0.3849 - val_sparse_categorical_accuracy: 0.8225 - learning_rate: 0.0010
## Epoch 10/500
## 90/90 - 0s - 2ms/step - loss: 0.3874 - sparse_categorical_accuracy: 0.8021 - val_loss: 0.4934 - val_sparse_categorical_accuracy: 0.7503 - learning_rate: 0.0010
## Epoch 11/500
## 90/90 - 0s - 1ms/step - loss: 0.3798 - sparse_categorical_accuracy: 0.8149 - val_loss: 0.5693 - val_sparse_categorical_accuracy: 0.6990 - learning_rate: 0.0010
## Epoch 12/500
## 90/90 - 0s - 1ms/step - loss: 0.3671 - sparse_categorical_accuracy: 0.8215 - val_loss: 0.3998 - val_sparse_categorical_accuracy: 0.8086 - learning_rate: 0.0010
## Epoch 13/500
## 90/90 - 0s - 1ms/step - loss: 0.3555 - sparse_categorical_accuracy: 0.8389 - val_loss: 0.4145 - val_sparse_categorical_accuracy: 0.8031 - learning_rate: 0.0010
## Epoch 14/500
## 90/90 - 0s - 1ms/step - loss: 0.3628 - sparse_categorical_accuracy: 0.8253 - val_loss: 0.3886 - val_sparse_categorical_accuracy: 0.8211 - learning_rate: 0.0010
## Epoch 15/500
## 90/90 - 0s - 1ms/step - loss: 0.3433 - sparse_categorical_accuracy: 0.8406 - val_loss: 0.7763 - val_sparse_categorical_accuracy: 0.6019 - learning_rate: 0.0010
## Epoch 16/500
## 90/90 - 0s - 2ms/step - loss: 0.3455 - sparse_categorical_accuracy: 0.8434 - val_loss: 0.3537 - val_sparse_categorical_accuracy: 0.8419 - learning_rate: 0.0010
## Epoch 17/500
## 90/90 - 0s - 1ms/step - loss: 0.3336 - sparse_categorical_accuracy: 0.8462 - val_loss: 0.5375 - val_sparse_categorical_accuracy: 0.7074 - learning_rate: 0.0010
## Epoch 18/500
## 90/90 - 0s - 1ms/step - loss: 0.3263 - sparse_categorical_accuracy: 0.8503 - val_loss: 0.4134 - val_sparse_categorical_accuracy: 0.7892 - learning_rate: 0.0010
## Epoch 19/500
## 90/90 - 0s - 2ms/step - loss: 0.3195 - sparse_categorical_accuracy: 0.8601 - val_loss: 0.3432 - val_sparse_categorical_accuracy: 0.8433 - learning_rate: 0.0010
## Epoch 20/500
## 90/90 - 0s - 1ms/step - loss: 0.3136 - sparse_categorical_accuracy: 0.8674 - val_loss: 0.9287 - val_sparse_categorical_accuracy: 0.5950 - learning_rate: 0.0010
## Epoch 21/500
## 90/90 - 0s - 1ms/step - loss: 0.3001 - sparse_categorical_accuracy: 0.8691 - val_loss: 0.3926 - val_sparse_categorical_accuracy: 0.7795 - learning_rate: 0.0010
## Epoch 22/500
## 90/90 - 0s - 2ms/step - loss: 0.2984 - sparse_categorical_accuracy: 0.8719 - val_loss: 0.3023 - val_sparse_categorical_accuracy: 0.8655 - learning_rate: 0.0010
## Epoch 23/500
## 90/90 - 0s - 1ms/step - loss: 0.2930 - sparse_categorical_accuracy: 0.8767 - val_loss: 0.3501 - val_sparse_categorical_accuracy: 0.8086 - learning_rate: 0.0010
## Epoch 24/500
## 90/90 - 0s - 1ms/step - loss: 0.2861 - sparse_categorical_accuracy: 0.8799 - val_loss: 0.9187 - val_sparse_categorical_accuracy: 0.5992 - learning_rate: 0.0010
## Epoch 25/500
## 90/90 - 0s - 1ms/step - loss: 0.2843 - sparse_categorical_accuracy: 0.8816 - val_loss: 0.3731 - val_sparse_categorical_accuracy: 0.8363 - learning_rate: 0.0010
## Epoch 26/500
## 90/90 - 0s - 1ms/step - loss: 0.3182 - sparse_categorical_accuracy: 0.8569 - val_loss: 0.5553 - val_sparse_categorical_accuracy: 0.7115 - learning_rate: 0.0010
## Epoch 27/500
## 90/90 - 0s - 1ms/step - loss: 0.2732 - sparse_categorical_accuracy: 0.8875 - val_loss: 0.3950 - val_sparse_categorical_accuracy: 0.7628 - learning_rate: 0.0010
## Epoch 28/500
## 90/90 - 0s - 1ms/step - loss: 0.2943 - sparse_categorical_accuracy: 0.8705 - val_loss: 0.8138 - val_sparse_categorical_accuracy: 0.7254 - learning_rate: 0.0010
## Epoch 29/500
## 90/90 - 0s - 1ms/step - loss: 0.2772 - sparse_categorical_accuracy: 0.8781 - val_loss: 0.3712 - val_sparse_categorical_accuracy: 0.8405 - learning_rate: 0.0010
## Epoch 30/500
## 90/90 - 0s - 1ms/step - loss: 0.2714 - sparse_categorical_accuracy: 0.8833 - val_loss: 0.4209 - val_sparse_categorical_accuracy: 0.7975 - learning_rate: 0.0010
## Epoch 31/500
## 90/90 - 0s - 2ms/step - loss: 0.2720 - sparse_categorical_accuracy: 0.8854 - val_loss: 1.9298 - val_sparse_categorical_accuracy: 0.5104 - learning_rate: 0.0010
## Epoch 32/500
## 90/90 - 0s - 2ms/step - loss: 0.2699 - sparse_categorical_accuracy: 0.8833 - val_loss: 0.3155 - val_sparse_categorical_accuracy: 0.8516 - learning_rate: 0.0010
## Epoch 33/500
## 90/90 - 0s - 2ms/step - loss: 0.2586 - sparse_categorical_accuracy: 0.8951 - val_loss: 0.3628 - val_sparse_categorical_accuracy: 0.7947 - learning_rate: 0.0010
## Epoch 34/500
## 90/90 - 0s - 1ms/step - loss: 0.2490 - sparse_categorical_accuracy: 0.9003 - val_loss: 0.5844 - val_sparse_categorical_accuracy: 0.7309 - learning_rate: 0.0010
## Epoch 35/500
## 90/90 - 0s - 1ms/step - loss: 0.2627 - sparse_categorical_accuracy: 0.8938 - val_loss: 0.3480 - val_sparse_categorical_accuracy: 0.8419 - learning_rate: 0.0010
## Epoch 36/500
## 90/90 - 0s - 1ms/step - loss: 0.2558 - sparse_categorical_accuracy: 0.8948 - val_loss: 0.3793 - val_sparse_categorical_accuracy: 0.8128 - learning_rate: 0.0010
## Epoch 37/500
## 90/90 - 0s - 1ms/step - loss: 0.2515 - sparse_categorical_accuracy: 0.8958 - val_loss: 0.4828 - val_sparse_categorical_accuracy: 0.7503 - learning_rate: 0.0010
## Epoch 38/500
## 90/90 - 0s - 2ms/step - loss: 0.2541 - sparse_categorical_accuracy: 0.8986 - val_loss: 0.2950 - val_sparse_categorical_accuracy: 0.8669 - learning_rate: 0.0010
## Epoch 39/500
## 90/90 - 0s - 1ms/step - loss: 0.2505 - sparse_categorical_accuracy: 0.8976 - val_loss: 0.7151 - val_sparse_categorical_accuracy: 0.7087 - learning_rate: 0.0010
## Epoch 40/500
## 90/90 - 0s - 1ms/step - loss: 0.2490 - sparse_categorical_accuracy: 0.8979 - val_loss: 0.3170 - val_sparse_categorical_accuracy: 0.8627 - learning_rate: 0.0010
## Epoch 41/500
## 90/90 - 0s - 2ms/step - loss: 0.2325 - sparse_categorical_accuracy: 0.9062 - val_loss: 0.2560 - val_sparse_categorical_accuracy: 0.8904 - learning_rate: 0.0010
## Epoch 42/500
## 90/90 - 0s - 1ms/step - loss: 0.2312 - sparse_categorical_accuracy: 0.9076 - val_loss: 0.3396 - val_sparse_categorical_accuracy: 0.8405 - learning_rate: 0.0010
## Epoch 43/500
## 90/90 - 0s - 1ms/step - loss: 0.2402 - sparse_categorical_accuracy: 0.8976 - val_loss: 0.3650 - val_sparse_categorical_accuracy: 0.8294 - learning_rate: 0.0010
## Epoch 44/500
## 90/90 - 0s - 1ms/step - loss: 0.2342 - sparse_categorical_accuracy: 0.9038 - val_loss: 1.0563 - val_sparse_categorical_accuracy: 0.6172 - learning_rate: 0.0010
## Epoch 45/500
## 90/90 - 0s - 1ms/step - loss: 0.2320 - sparse_categorical_accuracy: 0.9080 - val_loss: 0.2506 - val_sparse_categorical_accuracy: 0.8932 - learning_rate: 0.0010
## Epoch 46/500
## 90/90 - 0s - 1ms/step - loss: 0.2317 - sparse_categorical_accuracy: 0.9035 - val_loss: 0.4473 - val_sparse_categorical_accuracy: 0.7920 - learning_rate: 0.0010
## Epoch 47/500
## 90/90 - 0s - 1ms/step - loss: 0.2164 - sparse_categorical_accuracy: 0.9125 - val_loss: 0.6324 - val_sparse_categorical_accuracy: 0.7240 - learning_rate: 0.0010
## Epoch 48/500
## 90/90 - 0s - 2ms/step - loss: 0.2186 - sparse_categorical_accuracy: 0.9122 - val_loss: 0.2525 - val_sparse_categorical_accuracy: 0.9001 - learning_rate: 0.0010
## Epoch 49/500
## 90/90 - 0s - 2ms/step - loss: 0.2119 - sparse_categorical_accuracy: 0.9118 - val_loss: 0.2401 - val_sparse_categorical_accuracy: 0.9015 - learning_rate: 0.0010
## Epoch 50/500
## 90/90 - 0s - 1ms/step - loss: 0.2134 - sparse_categorical_accuracy: 0.9191 - val_loss: 0.2410 - val_sparse_categorical_accuracy: 0.8988 - learning_rate: 0.0010
## Epoch 51/500
## 90/90 - 0s - 2ms/step - loss: 0.2120 - sparse_categorical_accuracy: 0.9174 - val_loss: 0.6803 - val_sparse_categorical_accuracy: 0.6935 - learning_rate: 0.0010
## Epoch 52/500
## 90/90 - 0s - 2ms/step - loss: 0.2023 - sparse_categorical_accuracy: 0.9205 - val_loss: 0.2938 - val_sparse_categorical_accuracy: 0.8779 - learning_rate: 0.0010
## Epoch 53/500
## 90/90 - 0s - 2ms/step - loss: 0.1980 - sparse_categorical_accuracy: 0.9243 - val_loss: 0.5380 - val_sparse_categorical_accuracy: 0.7531 - learning_rate: 0.0010
## Epoch 54/500
## 90/90 - 0s - 2ms/step - loss: 0.1883 - sparse_categorical_accuracy: 0.9271 - val_loss: 0.2272 - val_sparse_categorical_accuracy: 0.9071 - learning_rate: 0.0010
## Epoch 55/500
## 90/90 - 0s - 2ms/step - loss: 0.1761 - sparse_categorical_accuracy: 0.9385 - val_loss: 0.2707 - val_sparse_categorical_accuracy: 0.8821 - learning_rate: 0.0010
## Epoch 56/500
## 90/90 - 0s - 1ms/step - loss: 0.1683 - sparse_categorical_accuracy: 0.9417 - val_loss: 0.2584 - val_sparse_categorical_accuracy: 0.9015 - learning_rate: 0.0010
## Epoch 57/500
## 90/90 - 0s - 1ms/step - loss: 0.1581 - sparse_categorical_accuracy: 0.9490 - val_loss: 1.4305 - val_sparse_categorical_accuracy: 0.7184 - learning_rate: 0.0010
## Epoch 58/500
## 90/90 - 0s - 1ms/step - loss: 0.1518 - sparse_categorical_accuracy: 0.9497 - val_loss: 0.7991 - val_sparse_categorical_accuracy: 0.7531 - learning_rate: 0.0010
## Epoch 59/500
## 90/90 - 0s - 1ms/step - loss: 0.1529 - sparse_categorical_accuracy: 0.9479 - val_loss: 0.3389 - val_sparse_categorical_accuracy: 0.8724 - learning_rate: 0.0010
## Epoch 60/500
## 90/90 - 0s - 1ms/step - loss: 0.1784 - sparse_categorical_accuracy: 0.9368 - val_loss: 0.2534 - val_sparse_categorical_accuracy: 0.8974 - learning_rate: 0.0010
## Epoch 61/500
## 90/90 - 0s - 1ms/step - loss: 0.1335 - sparse_categorical_accuracy: 0.9566 - val_loss: 0.2323 - val_sparse_categorical_accuracy: 0.9098 - learning_rate: 0.0010
## Epoch 62/500
## 90/90 - 0s - 1ms/step - loss: 0.1401 - sparse_categorical_accuracy: 0.9531 - val_loss: 0.3259 - val_sparse_categorical_accuracy: 0.8696 - learning_rate: 0.0010
## Epoch 63/500
## 90/90 - 0s - 1ms/step - loss: 0.1404 - sparse_categorical_accuracy: 0.9521 - val_loss: 0.2970 - val_sparse_categorical_accuracy: 0.8627 - learning_rate: 0.0010
## Epoch 64/500
## 90/90 - 0s - 1ms/step - loss: 0.1375 - sparse_categorical_accuracy: 0.9559 - val_loss: 0.3416 - val_sparse_categorical_accuracy: 0.8183 - learning_rate: 0.0010
## Epoch 65/500
## 90/90 - 0s - 1ms/step - loss: 0.1303 - sparse_categorical_accuracy: 0.9583 - val_loss: 0.2586 - val_sparse_categorical_accuracy: 0.8766 - learning_rate: 0.0010
## Epoch 66/500
## 90/90 - 0s - 1ms/step - loss: 0.1199 - sparse_categorical_accuracy: 0.9608 - val_loss: 1.2218 - val_sparse_categorical_accuracy: 0.7337 - learning_rate: 0.0010
## Epoch 67/500
## 90/90 - 0s - 2ms/step - loss: 0.1186 - sparse_categorical_accuracy: 0.9615 - val_loss: 0.1464 - val_sparse_categorical_accuracy: 0.9431 - learning_rate: 0.0010
## Epoch 68/500
## 90/90 - 0s - 2ms/step - loss: 0.1176 - sparse_categorical_accuracy: 0.9632 - val_loss: 0.1451 - val_sparse_categorical_accuracy: 0.9431 - learning_rate: 0.0010
## Epoch 69/500
## 90/90 - 0s - 2ms/step - loss: 0.1295 - sparse_categorical_accuracy: 0.9583 - val_loss: 1.7723 - val_sparse_categorical_accuracy: 0.5395 - learning_rate: 0.0010
## Epoch 70/500
## 90/90 - 0s - 2ms/step - loss: 0.1232 - sparse_categorical_accuracy: 0.9587 - val_loss: 0.1802 - val_sparse_categorical_accuracy: 0.9237 - learning_rate: 0.0010
## Epoch 71/500
## 90/90 - 0s - 1ms/step - loss: 0.1202 - sparse_categorical_accuracy: 0.9573 - val_loss: 0.1932 - val_sparse_categorical_accuracy: 0.9085 - learning_rate: 0.0010
## Epoch 72/500
## 90/90 - 0s - 1ms/step - loss: 0.1124 - sparse_categorical_accuracy: 0.9660 - val_loss: 0.4178 - val_sparse_categorical_accuracy: 0.8003 - learning_rate: 0.0010
## Epoch 73/500
## 90/90 - 0s - 1ms/step - loss: 0.1068 - sparse_categorical_accuracy: 0.9663 - val_loss: 0.1517 - val_sparse_categorical_accuracy: 0.9473 - learning_rate: 0.0010
## Epoch 74/500
## 90/90 - 0s - 1ms/step - loss: 0.1085 - sparse_categorical_accuracy: 0.9639 - val_loss: 0.2126 - val_sparse_categorical_accuracy: 0.9223 - learning_rate: 0.0010
## Epoch 75/500
## 90/90 - 0s - 1ms/step - loss: 0.1263 - sparse_categorical_accuracy: 0.9580 - val_loss: 0.1689 - val_sparse_categorical_accuracy: 0.9390 - learning_rate: 0.0010
## Epoch 76/500
## 90/90 - 0s - 1ms/step - loss: 0.1169 - sparse_categorical_accuracy: 0.9597 - val_loss: 0.2955 - val_sparse_categorical_accuracy: 0.8460 - learning_rate: 0.0010
## Epoch 77/500
## 90/90 - 0s - 1ms/step - loss: 0.1089 - sparse_categorical_accuracy: 0.9622 - val_loss: 0.6333 - val_sparse_categorical_accuracy: 0.6241 - learning_rate: 0.0010
## Epoch 78/500
## 90/90 - 0s - 2ms/step - loss: 0.1017 - sparse_categorical_accuracy: 0.9688 - val_loss: 0.1416 - val_sparse_categorical_accuracy: 0.9459 - learning_rate: 0.0010
## Epoch 79/500
## 90/90 - 0s - 2ms/step - loss: 0.1099 - sparse_categorical_accuracy: 0.9642 - val_loss: 0.3434 - val_sparse_categorical_accuracy: 0.8544 - learning_rate: 0.0010
## Epoch 80/500
## 90/90 - 0s - 2ms/step - loss: 0.1046 - sparse_categorical_accuracy: 0.9653 - val_loss: 0.4430 - val_sparse_categorical_accuracy: 0.8280 - learning_rate: 0.0010
## Epoch 81/500
## 90/90 - 0s - 1ms/step - loss: 0.1098 - sparse_categorical_accuracy: 0.9622 - val_loss: 0.1632 - val_sparse_categorical_accuracy: 0.9362 - learning_rate: 0.0010
## Epoch 82/500
## 90/90 - 0s - 1ms/step - loss: 0.1021 - sparse_categorical_accuracy: 0.9670 - val_loss: 0.3984 - val_sparse_categorical_accuracy: 0.7906 - learning_rate: 0.0010
## Epoch 83/500
## 90/90 - 0s - 1ms/step - loss: 0.1086 - sparse_categorical_accuracy: 0.9642 - val_loss: 0.1976 - val_sparse_categorical_accuracy: 0.9085 - learning_rate: 0.0010
## Epoch 84/500
## 90/90 - 0s - 1ms/step - loss: 0.1026 - sparse_categorical_accuracy: 0.9642 - val_loss: 0.2123 - val_sparse_categorical_accuracy: 0.9251 - learning_rate: 0.0010
## Epoch 85/500
## 90/90 - 0s - 1ms/step - loss: 0.1032 - sparse_categorical_accuracy: 0.9632 - val_loss: 0.3491 - val_sparse_categorical_accuracy: 0.8613 - learning_rate: 0.0010
## Epoch 86/500
## 90/90 - 0s - 1ms/step - loss: 0.1136 - sparse_categorical_accuracy: 0.9576 - val_loss: 0.1498 - val_sparse_categorical_accuracy: 0.9362 - learning_rate: 0.0010
## Epoch 87/500
## 90/90 - 0s - 2ms/step - loss: 0.1107 - sparse_categorical_accuracy: 0.9608 - val_loss: 0.1406 - val_sparse_categorical_accuracy: 0.9501 - learning_rate: 0.0010
## Epoch 88/500
## 90/90 - 0s - 1ms/step - loss: 0.0995 - sparse_categorical_accuracy: 0.9677 - val_loss: 0.6864 - val_sparse_categorical_accuracy: 0.7656 - learning_rate: 0.0010
## Epoch 89/500
## 90/90 - 0s - 1ms/step - loss: 0.1013 - sparse_categorical_accuracy: 0.9656 - val_loss: 0.8218 - val_sparse_categorical_accuracy: 0.7476 - learning_rate: 0.0010
## Epoch 90/500
## 90/90 - 0s - 1ms/step - loss: 0.1002 - sparse_categorical_accuracy: 0.9670 - val_loss: 0.2009 - val_sparse_categorical_accuracy: 0.9209 - learning_rate: 0.0010
## Epoch 91/500
## 90/90 - 0s - 1ms/step - loss: 0.1010 - sparse_categorical_accuracy: 0.9674 - val_loss: 0.2869 - val_sparse_categorical_accuracy: 0.8849 - learning_rate: 0.0010
## Epoch 92/500
## 90/90 - 0s - 1ms/step - loss: 0.1012 - sparse_categorical_accuracy: 0.9688 - val_loss: 0.5128 - val_sparse_categorical_accuracy: 0.7434 - learning_rate: 0.0010
## Epoch 93/500
## 90/90 - 0s - 1ms/step - loss: 0.1009 - sparse_categorical_accuracy: 0.9660 - val_loss: 0.5212 - val_sparse_categorical_accuracy: 0.7753 - learning_rate: 0.0010
## Epoch 94/500
## 90/90 - 0s - 1ms/step - loss: 0.1016 - sparse_categorical_accuracy: 0.9663 - val_loss: 0.3146 - val_sparse_categorical_accuracy: 0.8682 - learning_rate: 0.0010
## Epoch 95/500
## 90/90 - 0s - 1ms/step - loss: 0.0975 - sparse_categorical_accuracy: 0.9663 - val_loss: 0.1750 - val_sparse_categorical_accuracy: 0.9376 - learning_rate: 0.0010
## Epoch 96/500
## 90/90 - 0s - 1ms/step - loss: 0.1121 - sparse_categorical_accuracy: 0.9583 - val_loss: 0.2399 - val_sparse_categorical_accuracy: 0.9112 - learning_rate: 0.0010
## Epoch 97/500
## 90/90 - 0s - 1ms/step - loss: 0.1000 - sparse_categorical_accuracy: 0.9670 - val_loss: 0.5611 - val_sparse_categorical_accuracy: 0.7101 - learning_rate: 0.0010
## Epoch 98/500
## 90/90 - 0s - 1ms/step - loss: 0.0949 - sparse_categorical_accuracy: 0.9663 - val_loss: 0.1864 - val_sparse_categorical_accuracy: 0.9279 - learning_rate: 0.0010
## Epoch 99/500
## 90/90 - 0s - 2ms/step - loss: 0.0925 - sparse_categorical_accuracy: 0.9667 - val_loss: 0.1228 - val_sparse_categorical_accuracy: 0.9431 - learning_rate: 0.0010
## Epoch 100/500
## 90/90 - 0s - 1ms/step - loss: 0.0922 - sparse_categorical_accuracy: 0.9691 - val_loss: 1.9448 - val_sparse_categorical_accuracy: 0.7115 - learning_rate: 0.0010
## Epoch 101/500
## 90/90 - 0s - 1ms/step - loss: 0.0887 - sparse_categorical_accuracy: 0.9733 - val_loss: 0.2127 - val_sparse_categorical_accuracy: 0.9071 - learning_rate: 0.0010
## Epoch 102/500
## 90/90 - 0s - 1ms/step - loss: 0.0999 - sparse_categorical_accuracy: 0.9660 - val_loss: 0.1903 - val_sparse_categorical_accuracy: 0.9154 - learning_rate: 0.0010
## Epoch 103/500
## 90/90 - 0s - 1ms/step - loss: 0.0929 - sparse_categorical_accuracy: 0.9698 - val_loss: 0.2647 - val_sparse_categorical_accuracy: 0.8988 - learning_rate: 0.0010
## Epoch 104/500
## 90/90 - 0s - 1ms/step - loss: 0.1008 - sparse_categorical_accuracy: 0.9642 - val_loss: 1.1019 - val_sparse_categorical_accuracy: 0.6019 - learning_rate: 0.0010
## Epoch 105/500
## 90/90 - 0s - 1ms/step - loss: 0.0969 - sparse_categorical_accuracy: 0.9681 - val_loss: 1.1464 - val_sparse_categorical_accuracy: 0.5645 - learning_rate: 0.0010
## Epoch 106/500
## 90/90 - 0s - 2ms/step - loss: 0.0924 - sparse_categorical_accuracy: 0.9670 - val_loss: 0.2535 - val_sparse_categorical_accuracy: 0.9085 - learning_rate: 0.0010
## Epoch 107/500
## 90/90 - 0s - 2ms/step - loss: 0.0930 - sparse_categorical_accuracy: 0.9677 - val_loss: 1.5288 - val_sparse_categorical_accuracy: 0.5118 - learning_rate: 0.0010
## Epoch 108/500
## 90/90 - 0s - 1ms/step - loss: 0.0893 - sparse_categorical_accuracy: 0.9701 - val_loss: 0.1496 - val_sparse_categorical_accuracy: 0.9404 - learning_rate: 0.0010
## Epoch 109/500
## 90/90 - 0s - 1ms/step - loss: 0.0891 - sparse_categorical_accuracy: 0.9712 - val_loss: 0.2799 - val_sparse_categorical_accuracy: 0.8835 - learning_rate: 0.0010
## Epoch 110/500
## 90/90 - 0s - 1ms/step - loss: 0.0963 - sparse_categorical_accuracy: 0.9677 - val_loss: 0.4827 - val_sparse_categorical_accuracy: 0.8155 - learning_rate: 0.0010
## Epoch 111/500
## 90/90 - 0s - 2ms/step - loss: 0.0841 - sparse_categorical_accuracy: 0.9701 - val_loss: 0.1203 - val_sparse_categorical_accuracy: 0.9501 - learning_rate: 0.0010
## Epoch 112/500
## 90/90 - 0s - 1ms/step - loss: 0.0861 - sparse_categorical_accuracy: 0.9719 - val_loss: 0.5081 - val_sparse_categorical_accuracy: 0.7559 - learning_rate: 0.0010
## Epoch 113/500
## 90/90 - 0s - 1ms/step - loss: 0.0831 - sparse_categorical_accuracy: 0.9729 - val_loss: 1.2423 - val_sparse_categorical_accuracy: 0.5687 - learning_rate: 0.0010
## Epoch 114/500
## 90/90 - 0s - 1ms/step - loss: 0.1094 - sparse_categorical_accuracy: 0.9653 - val_loss: 0.3459 - val_sparse_categorical_accuracy: 0.8252 - learning_rate: 0.0010
## Epoch 115/500
## 90/90 - 0s - 1ms/step - loss: 0.0886 - sparse_categorical_accuracy: 0.9691 - val_loss: 0.7117 - val_sparse_categorical_accuracy: 0.7226 - learning_rate: 0.0010
## Epoch 116/500
## 90/90 - 0s - 1ms/step - loss: 0.0989 - sparse_categorical_accuracy: 0.9698 - val_loss: 0.1669 - val_sparse_categorical_accuracy: 0.9334 - learning_rate: 0.0010
## Epoch 117/500
## 90/90 - 0s - 1ms/step - loss: 0.0891 - sparse_categorical_accuracy: 0.9705 - val_loss: 0.2413 - val_sparse_categorical_accuracy: 0.9154 - learning_rate: 0.0010
## Epoch 118/500
## 90/90 - 0s - 1ms/step - loss: 0.0902 - sparse_categorical_accuracy: 0.9691 - val_loss: 0.7796 - val_sparse_categorical_accuracy: 0.6741 - learning_rate: 0.0010
## Epoch 119/500
## 90/90 - 0s - 1ms/step - loss: 0.0877 - sparse_categorical_accuracy: 0.9691 - val_loss: 0.1806 - val_sparse_categorical_accuracy: 0.9168 - learning_rate: 0.0010
## Epoch 120/500
## 90/90 - 0s - 2ms/step - loss: 0.0880 - sparse_categorical_accuracy: 0.9688 - val_loss: 0.1146 - val_sparse_categorical_accuracy: 0.9598 - learning_rate: 0.0010
## Epoch 121/500
## 90/90 - 0s - 1ms/step - loss: 0.0896 - sparse_categorical_accuracy: 0.9726 - val_loss: 0.1279 - val_sparse_categorical_accuracy: 0.9528 - learning_rate: 0.0010
## Epoch 122/500
## 90/90 - 0s - 1ms/step - loss: 0.0883 - sparse_categorical_accuracy: 0.9701 - val_loss: 0.3067 - val_sparse_categorical_accuracy: 0.8918 - learning_rate: 0.0010
## Epoch 123/500
## 90/90 - 0s - 2ms/step - loss: 0.0855 - sparse_categorical_accuracy: 0.9722 - val_loss: 0.1060 - val_sparse_categorical_accuracy: 0.9667 - learning_rate: 0.0010
## Epoch 124/500
## 90/90 - 0s - 1ms/step - loss: 0.0917 - sparse_categorical_accuracy: 0.9663 - val_loss: 0.2169 - val_sparse_categorical_accuracy: 0.9112 - learning_rate: 0.0010
## Epoch 125/500
## 90/90 - 0s - 1ms/step - loss: 0.0976 - sparse_categorical_accuracy: 0.9642 - val_loss: 0.1407 - val_sparse_categorical_accuracy: 0.9459 - learning_rate: 0.0010
## Epoch 126/500
## 90/90 - 0s - 1ms/step - loss: 0.0866 - sparse_categorical_accuracy: 0.9722 - val_loss: 1.4019 - val_sparse_categorical_accuracy: 0.7295 - learning_rate: 0.0010
## Epoch 127/500
## 90/90 - 0s - 1ms/step - loss: 0.0877 - sparse_categorical_accuracy: 0.9747 - val_loss: 0.2518 - val_sparse_categorical_accuracy: 0.8918 - learning_rate: 0.0010
## Epoch 128/500
## 90/90 - 0s - 1ms/step - loss: 0.0942 - sparse_categorical_accuracy: 0.9677 - val_loss: 0.1465 - val_sparse_categorical_accuracy: 0.9445 - learning_rate: 0.0010
## Epoch 129/500
## 90/90 - 0s - 1ms/step - loss: 0.0904 - sparse_categorical_accuracy: 0.9712 - val_loss: 0.1546 - val_sparse_categorical_accuracy: 0.9445 - learning_rate: 0.0010
## Epoch 130/500
## 90/90 - 0s - 1ms/step - loss: 0.0953 - sparse_categorical_accuracy: 0.9705 - val_loss: 0.1201 - val_sparse_categorical_accuracy: 0.9584 - learning_rate: 0.0010
## Epoch 131/500
## 90/90 - 0s - 1ms/step - loss: 0.0846 - sparse_categorical_accuracy: 0.9712 - val_loss: 0.1980 - val_sparse_categorical_accuracy: 0.9293 - learning_rate: 0.0010
## Epoch 132/500
## 90/90 - 0s - 1ms/step - loss: 0.0812 - sparse_categorical_accuracy: 0.9760 - val_loss: 0.4080 - val_sparse_categorical_accuracy: 0.8599 - learning_rate: 0.0010
## Epoch 133/500
## 90/90 - 0s - 1ms/step - loss: 0.0785 - sparse_categorical_accuracy: 0.9733 - val_loss: 0.3284 - val_sparse_categorical_accuracy: 0.8835 - learning_rate: 0.0010
## Epoch 134/500
## 90/90 - 0s - 1ms/step - loss: 0.0885 - sparse_categorical_accuracy: 0.9719 - val_loss: 0.5050 - val_sparse_categorical_accuracy: 0.8419 - learning_rate: 0.0010
## Epoch 135/500
## 90/90 - 0s - 1ms/step - loss: 0.0859 - sparse_categorical_accuracy: 0.9726 - val_loss: 0.4296 - val_sparse_categorical_accuracy: 0.8141 - learning_rate: 0.0010
## Epoch 136/500
## 90/90 - 0s - 1ms/step - loss: 0.0885 - sparse_categorical_accuracy: 0.9670 - val_loss: 0.1295 - val_sparse_categorical_accuracy: 0.9515 - learning_rate: 0.0010
## Epoch 137/500
## 90/90 - 0s - 1ms/step - loss: 0.0892 - sparse_categorical_accuracy: 0.9712 - val_loss: 0.4052 - val_sparse_categorical_accuracy: 0.8031 - learning_rate: 0.0010
## Epoch 138/500
## 90/90 - 0s - 1ms/step - loss: 0.0835 - sparse_categorical_accuracy: 0.9719 - val_loss: 0.1160 - val_sparse_categorical_accuracy: 0.9653 - learning_rate: 0.0010
## Epoch 139/500
## 90/90 - 0s - 1ms/step - loss: 0.0877 - sparse_categorical_accuracy: 0.9708 - val_loss: 3.0631 - val_sparse_categorical_accuracy: 0.7032 - learning_rate: 0.0010
## Epoch 140/500
## 90/90 - 0s - 1ms/step - loss: 0.0867 - sparse_categorical_accuracy: 0.9719 - val_loss: 1.3638 - val_sparse_categorical_accuracy: 0.7143 - learning_rate: 0.0010
## Epoch 141/500
## 90/90 - 0s - 1ms/step - loss: 0.0857 - sparse_categorical_accuracy: 0.9715 - val_loss: 0.2599 - val_sparse_categorical_accuracy: 0.9015 - learning_rate: 0.0010
## Epoch 142/500
## 90/90 - 0s - 1ms/step - loss: 0.0771 - sparse_categorical_accuracy: 0.9764 - val_loss: 0.2860 - val_sparse_categorical_accuracy: 0.8988 - learning_rate: 0.0010
## Epoch 143/500
## 90/90 - 0s - 1ms/step - loss: 0.0835 - sparse_categorical_accuracy: 0.9694 - val_loss: 0.1252 - val_sparse_categorical_accuracy: 0.9639 - learning_rate: 0.0010
## Epoch 144/500
## 90/90 - 1s - 7ms/step - loss: 0.0674 - sparse_categorical_accuracy: 0.9809 - val_loss: 0.1234 - val_sparse_categorical_accuracy: 0.9528 - learning_rate: 5.0000e-04
## Epoch 145/500
## 90/90 - 0s - 1ms/step - loss: 0.0680 - sparse_categorical_accuracy: 0.9771 - val_loss: 0.1147 - val_sparse_categorical_accuracy: 0.9570 - learning_rate: 5.0000e-04
## Epoch 146/500
## 90/90 - 0s - 2ms/step - loss: 0.0702 - sparse_categorical_accuracy: 0.9781 - val_loss: 0.1042 - val_sparse_categorical_accuracy: 0.9626 - learning_rate: 5.0000e-04
## Epoch 147/500
## 90/90 - 0s - 1ms/step - loss: 0.0742 - sparse_categorical_accuracy: 0.9757 - val_loss: 0.1511 - val_sparse_categorical_accuracy: 0.9501 - learning_rate: 5.0000e-04
## Epoch 148/500
## 90/90 - 0s - 1ms/step - loss: 0.0801 - sparse_categorical_accuracy: 0.9708 - val_loss: 0.1219 - val_sparse_categorical_accuracy: 0.9626 - learning_rate: 5.0000e-04
## Epoch 149/500
## 90/90 - 0s - 2ms/step - loss: 0.0725 - sparse_categorical_accuracy: 0.9767 - val_loss: 0.1028 - val_sparse_categorical_accuracy: 0.9639 - learning_rate: 5.0000e-04
## Epoch 150/500
## 90/90 - 0s - 1ms/step - loss: 0.0738 - sparse_categorical_accuracy: 0.9733 - val_loss: 0.2021 - val_sparse_categorical_accuracy: 0.9334 - learning_rate: 5.0000e-04
## Epoch 151/500
## 90/90 - 0s - 1ms/step - loss: 0.0707 - sparse_categorical_accuracy: 0.9750 - val_loss: 0.1358 - val_sparse_categorical_accuracy: 0.9376 - learning_rate: 5.0000e-04
## Epoch 152/500
## 90/90 - 0s - 1ms/step - loss: 0.0727 - sparse_categorical_accuracy: 0.9740 - val_loss: 0.1989 - val_sparse_categorical_accuracy: 0.9098 - learning_rate: 5.0000e-04
## Epoch 153/500
## 90/90 - 0s - 1ms/step - loss: 0.0671 - sparse_categorical_accuracy: 0.9778 - val_loss: 0.1250 - val_sparse_categorical_accuracy: 0.9639 - learning_rate: 5.0000e-04
## Epoch 154/500
## 90/90 - 0s - 1ms/step - loss: 0.0687 - sparse_categorical_accuracy: 0.9781 - val_loss: 0.1973 - val_sparse_categorical_accuracy: 0.9237 - learning_rate: 5.0000e-04
## Epoch 155/500
## 90/90 - 0s - 1ms/step - loss: 0.0697 - sparse_categorical_accuracy: 0.9774 - val_loss: 0.1190 - val_sparse_categorical_accuracy: 0.9515 - learning_rate: 5.0000e-04
## Epoch 156/500
## 90/90 - 0s - 1ms/step - loss: 0.0731 - sparse_categorical_accuracy: 0.9750 - val_loss: 0.1409 - val_sparse_categorical_accuracy: 0.9556 - learning_rate: 5.0000e-04
## Epoch 157/500
## 90/90 - 0s - 1ms/step - loss: 0.0719 - sparse_categorical_accuracy: 0.9740 - val_loss: 0.4024 - val_sparse_categorical_accuracy: 0.8655 - learning_rate: 5.0000e-04
## Epoch 158/500
## 90/90 - 0s - 1ms/step - loss: 0.0722 - sparse_categorical_accuracy: 0.9750 - val_loss: 0.3254 - val_sparse_categorical_accuracy: 0.8821 - learning_rate: 5.0000e-04
## Epoch 159/500
## 90/90 - 0s - 1ms/step - loss: 0.0706 - sparse_categorical_accuracy: 0.9760 - val_loss: 0.2014 - val_sparse_categorical_accuracy: 0.9265 - learning_rate: 5.0000e-04
## Epoch 160/500
## 90/90 - 0s - 1ms/step - loss: 0.0707 - sparse_categorical_accuracy: 0.9753 - val_loss: 0.1128 - val_sparse_categorical_accuracy: 0.9612 - learning_rate: 5.0000e-04
## Epoch 161/500
## 90/90 - 0s - 1ms/step - loss: 0.0727 - sparse_categorical_accuracy: 0.9767 - val_loss: 0.1118 - val_sparse_categorical_accuracy: 0.9584 - learning_rate: 5.0000e-04
## Epoch 162/500
## 90/90 - 0s - 1ms/step - loss: 0.0660 - sparse_categorical_accuracy: 0.9767 - val_loss: 0.2972 - val_sparse_categorical_accuracy: 0.8960 - learning_rate: 5.0000e-04
## Epoch 163/500
## 90/90 - 0s - 1ms/step - loss: 0.0619 - sparse_categorical_accuracy: 0.9795 - val_loss: 0.1079 - val_sparse_categorical_accuracy: 0.9598 - learning_rate: 5.0000e-04
## Epoch 164/500
## 90/90 - 0s - 1ms/step - loss: 0.0670 - sparse_categorical_accuracy: 0.9774 - val_loss: 0.2169 - val_sparse_categorical_accuracy: 0.9168 - learning_rate: 5.0000e-04
## Epoch 165/500
## 90/90 - 0s - 1ms/step - loss: 0.0690 - sparse_categorical_accuracy: 0.9757 - val_loss: 0.2980 - val_sparse_categorical_accuracy: 0.9015 - learning_rate: 5.0000e-04
## Epoch 166/500
## 90/90 - 0s - 1ms/step - loss: 0.0650 - sparse_categorical_accuracy: 0.9785 - val_loss: 0.1257 - val_sparse_categorical_accuracy: 0.9473 - learning_rate: 5.0000e-04
## Epoch 167/500
## 90/90 - 0s - 1ms/step - loss: 0.0656 - sparse_categorical_accuracy: 0.9788 - val_loss: 0.1451 - val_sparse_categorical_accuracy: 0.9445 - learning_rate: 5.0000e-04
## Epoch 168/500
## 90/90 - 0s - 1ms/step - loss: 0.0750 - sparse_categorical_accuracy: 0.9736 - val_loss: 0.3402 - val_sparse_categorical_accuracy: 0.8350 - learning_rate: 5.0000e-04
## Epoch 169/500
## 90/90 - 0s - 1ms/step - loss: 0.0664 - sparse_categorical_accuracy: 0.9785 - val_loss: 0.1295 - val_sparse_categorical_accuracy: 0.9459 - learning_rate: 5.0000e-04
## Epoch 170/500
## 90/90 - 0s - 2ms/step - loss: 0.0635 - sparse_categorical_accuracy: 0.9816 - val_loss: 0.0992 - val_sparse_categorical_accuracy: 0.9612 - learning_rate: 2.5000e-04
## Epoch 171/500
## 90/90 - 0s - 1ms/step - loss: 0.0593 - sparse_categorical_accuracy: 0.9816 - val_loss: 0.1230 - val_sparse_categorical_accuracy: 0.9626 - learning_rate: 2.5000e-04
## Epoch 172/500
## 90/90 - 0s - 1ms/step - loss: 0.0591 - sparse_categorical_accuracy: 0.9799 - val_loss: 0.1079 - val_sparse_categorical_accuracy: 0.9695 - learning_rate: 2.5000e-04
## Epoch 173/500
## 90/90 - 0s - 1ms/step - loss: 0.0593 - sparse_categorical_accuracy: 0.9816 - val_loss: 0.1073 - val_sparse_categorical_accuracy: 0.9584 - learning_rate: 2.5000e-04
## Epoch 174/500
## 90/90 - 0s - 2ms/step - loss: 0.0605 - sparse_categorical_accuracy: 0.9819 - val_loss: 0.0979 - val_sparse_categorical_accuracy: 0.9639 - learning_rate: 2.5000e-04
## Epoch 175/500
## 90/90 - 0s - 1ms/step - loss: 0.0616 - sparse_categorical_accuracy: 0.9802 - val_loss: 0.2084 - val_sparse_categorical_accuracy: 0.9223 - learning_rate: 2.5000e-04
## Epoch 176/500
## 90/90 - 0s - 1ms/step - loss: 0.0584 - sparse_categorical_accuracy: 0.9795 - val_loss: 0.0993 - val_sparse_categorical_accuracy: 0.9667 - learning_rate: 2.5000e-04
## Epoch 177/500
## 90/90 - 0s - 1ms/step - loss: 0.0582 - sparse_categorical_accuracy: 0.9823 - val_loss: 0.1498 - val_sparse_categorical_accuracy: 0.9487 - learning_rate: 2.5000e-04
## Epoch 178/500
## 90/90 - 0s - 1ms/step - loss: 0.0566 - sparse_categorical_accuracy: 0.9806 - val_loss: 0.1165 - val_sparse_categorical_accuracy: 0.9528 - learning_rate: 2.5000e-04
## Epoch 179/500
## 90/90 - 0s - 2ms/step - loss: 0.0566 - sparse_categorical_accuracy: 0.9806 - val_loss: 0.1714 - val_sparse_categorical_accuracy: 0.9293 - learning_rate: 2.5000e-04
## Epoch 180/500
## 90/90 - 0s - 2ms/step - loss: 0.0563 - sparse_categorical_accuracy: 0.9833 - val_loss: 0.1042 - val_sparse_categorical_accuracy: 0.9598 - learning_rate: 2.5000e-04
## Epoch 181/500
## 90/90 - 0s - 2ms/step - loss: 0.0609 - sparse_categorical_accuracy: 0.9806 - val_loss: 0.2017 - val_sparse_categorical_accuracy: 0.9209 - learning_rate: 2.5000e-04
## Epoch 182/500
## 90/90 - 0s - 1ms/step - loss: 0.0559 - sparse_categorical_accuracy: 0.9833 - val_loss: 0.1059 - val_sparse_categorical_accuracy: 0.9653 - learning_rate: 2.5000e-04
## Epoch 183/500
## 90/90 - 0s - 1ms/step - loss: 0.0597 - sparse_categorical_accuracy: 0.9802 - val_loss: 0.1279 - val_sparse_categorical_accuracy: 0.9445 - learning_rate: 2.5000e-04
## Epoch 184/500
## 90/90 - 0s - 1ms/step - loss: 0.0553 - sparse_categorical_accuracy: 0.9816 - val_loss: 0.1036 - val_sparse_categorical_accuracy: 0.9667 - learning_rate: 2.5000e-04
## Epoch 185/500
## 90/90 - 0s - 1ms/step - loss: 0.0570 - sparse_categorical_accuracy: 0.9809 - val_loss: 0.1022 - val_sparse_categorical_accuracy: 0.9681 - learning_rate: 2.5000e-04
## Epoch 186/500
## 90/90 - 0s - 1ms/step - loss: 0.0633 - sparse_categorical_accuracy: 0.9781 - val_loss: 0.1086 - val_sparse_categorical_accuracy: 0.9584 - learning_rate: 2.5000e-04
## Epoch 187/500
## 90/90 - 0s - 2ms/step - loss: 0.0576 - sparse_categorical_accuracy: 0.9833 - val_loss: 0.0969 - val_sparse_categorical_accuracy: 0.9681 - learning_rate: 2.5000e-04
## Epoch 188/500
## 90/90 - 0s - 1ms/step - loss: 0.0567 - sparse_categorical_accuracy: 0.9840 - val_loss: 0.1043 - val_sparse_categorical_accuracy: 0.9612 - learning_rate: 2.5000e-04
## Epoch 189/500
## 90/90 - 0s - 1ms/step - loss: 0.0546 - sparse_categorical_accuracy: 0.9854 - val_loss: 0.1734 - val_sparse_categorical_accuracy: 0.9362 - learning_rate: 2.5000e-04
## Epoch 190/500
## 90/90 - 0s - 1ms/step - loss: 0.0639 - sparse_categorical_accuracy: 0.9799 - val_loss: 0.2847 - val_sparse_categorical_accuracy: 0.9015 - learning_rate: 2.5000e-04
## Epoch 191/500
## 90/90 - 0s - 1ms/step - loss: 0.0551 - sparse_categorical_accuracy: 0.9799 - val_loss: 0.4059 - val_sparse_categorical_accuracy: 0.8682 - learning_rate: 2.5000e-04
## Epoch 192/500
## 90/90 - 0s - 1ms/step - loss: 0.0613 - sparse_categorical_accuracy: 0.9802 - val_loss: 0.3821 - val_sparse_categorical_accuracy: 0.8724 - learning_rate: 2.5000e-04
## Epoch 193/500
## 90/90 - 0s - 1ms/step - loss: 0.0559 - sparse_categorical_accuracy: 0.9833 - val_loss: 0.1028 - val_sparse_categorical_accuracy: 0.9667 - learning_rate: 2.5000e-04
## Epoch 194/500
## 90/90 - 0s - 1ms/step - loss: 0.0553 - sparse_categorical_accuracy: 0.9806 - val_loss: 0.1008 - val_sparse_categorical_accuracy: 0.9612 - learning_rate: 2.5000e-04
## Epoch 195/500
## 90/90 - 0s - 1ms/step - loss: 0.0557 - sparse_categorical_accuracy: 0.9830 - val_loss: 0.0996 - val_sparse_categorical_accuracy: 0.9653 - learning_rate: 2.5000e-04
## Epoch 196/500
## 90/90 - 0s - 1ms/step - loss: 0.0581 - sparse_categorical_accuracy: 0.9778 - val_loss: 0.1143 - val_sparse_categorical_accuracy: 0.9542 - learning_rate: 2.5000e-04
## Epoch 197/500
## 90/90 - 0s - 1ms/step - loss: 0.0544 - sparse_categorical_accuracy: 0.9830 - val_loss: 0.1027 - val_sparse_categorical_accuracy: 0.9709 - learning_rate: 2.5000e-04
## Epoch 198/500
## 90/90 - 0s - 1ms/step - loss: 0.0533 - sparse_categorical_accuracy: 0.9861 - val_loss: 0.1310 - val_sparse_categorical_accuracy: 0.9598 - learning_rate: 2.5000e-04
## Epoch 199/500
## 90/90 - 0s - 1ms/step - loss: 0.0578 - sparse_categorical_accuracy: 0.9795 - val_loss: 0.1236 - val_sparse_categorical_accuracy: 0.9515 - learning_rate: 2.5000e-04
## Epoch 200/500
## 90/90 - 0s - 1ms/step - loss: 0.0582 - sparse_categorical_accuracy: 0.9813 - val_loss: 0.0987 - val_sparse_categorical_accuracy: 0.9626 - learning_rate: 2.5000e-04
## Epoch 201/500
## 90/90 - 0s - 1ms/step - loss: 0.0551 - sparse_categorical_accuracy: 0.9830 - val_loss: 0.1327 - val_sparse_categorical_accuracy: 0.9612 - learning_rate: 2.5000e-04
## Epoch 202/500
## 90/90 - 0s - 1ms/step - loss: 0.0543 - sparse_categorical_accuracy: 0.9823 - val_loss: 0.2115 - val_sparse_categorical_accuracy: 0.9140 - learning_rate: 2.5000e-04
## Epoch 203/500
## 90/90 - 0s - 1ms/step - loss: 0.0553 - sparse_categorical_accuracy: 0.9830 - val_loss: 0.2015 - val_sparse_categorical_accuracy: 0.9223 - learning_rate: 2.5000e-04
## Epoch 204/500
## 90/90 - 0s - 1ms/step - loss: 0.0508 - sparse_categorical_accuracy: 0.9840 - val_loss: 0.1078 - val_sparse_categorical_accuracy: 0.9695 - learning_rate: 2.5000e-04
## Epoch 205/500
## 90/90 - 0s - 1ms/step - loss: 0.0542 - sparse_categorical_accuracy: 0.9830 - val_loss: 0.0996 - val_sparse_categorical_accuracy: 0.9612 - learning_rate: 2.5000e-04
## Epoch 206/500
## 90/90 - 0s - 1ms/step - loss: 0.0559 - sparse_categorical_accuracy: 0.9806 - val_loss: 0.1172 - val_sparse_categorical_accuracy: 0.9528 - learning_rate: 2.5000e-04
## Epoch 207/500
## 90/90 - 0s - 2ms/step - loss: 0.0570 - sparse_categorical_accuracy: 0.9819 - val_loss: 0.0987 - val_sparse_categorical_accuracy: 0.9542 - learning_rate: 2.5000e-04
## Epoch 208/500
## 90/90 - 0s - 2ms/step - loss: 0.0521 - sparse_categorical_accuracy: 0.9840 - val_loss: 0.1000 - val_sparse_categorical_accuracy: 0.9653 - learning_rate: 1.2500e-04
## Epoch 209/500
## 90/90 - 0s - 1ms/step - loss: 0.0497 - sparse_categorical_accuracy: 0.9823 - val_loss: 0.1286 - val_sparse_categorical_accuracy: 0.9584 - learning_rate: 1.2500e-04
## Epoch 210/500
## 90/90 - 0s - 1ms/step - loss: 0.0541 - sparse_categorical_accuracy: 0.9816 - val_loss: 0.1028 - val_sparse_categorical_accuracy: 0.9695 - learning_rate: 1.2500e-04
## Epoch 211/500
## 90/90 - 0s - 1ms/step - loss: 0.0509 - sparse_categorical_accuracy: 0.9844 - val_loss: 0.0993 - val_sparse_categorical_accuracy: 0.9695 - learning_rate: 1.2500e-04
## Epoch 212/500
## 90/90 - 0s - 1ms/step - loss: 0.0503 - sparse_categorical_accuracy: 0.9854 - val_loss: 0.0987 - val_sparse_categorical_accuracy: 0.9709 - learning_rate: 1.2500e-04
## Epoch 213/500
## 90/90 - 0s - 2ms/step - loss: 0.0534 - sparse_categorical_accuracy: 0.9837 - val_loss: 0.0968 - val_sparse_categorical_accuracy: 0.9612 - learning_rate: 1.2500e-04
## Epoch 214/500
## 90/90 - 0s - 1ms/step - loss: 0.0532 - sparse_categorical_accuracy: 0.9844 - val_loss: 0.1078 - val_sparse_categorical_accuracy: 0.9681 - learning_rate: 1.2500e-04
## Epoch 215/500
## 90/90 - 0s - 1ms/step - loss: 0.0488 - sparse_categorical_accuracy: 0.9851 - val_loss: 0.1013 - val_sparse_categorical_accuracy: 0.9709 - learning_rate: 1.2500e-04
## Epoch 216/500
## 90/90 - 0s - 1ms/step - loss: 0.0497 - sparse_categorical_accuracy: 0.9851 - val_loss: 0.1244 - val_sparse_categorical_accuracy: 0.9570 - learning_rate: 1.2500e-04
## Epoch 217/500
## 90/90 - 0s - 2ms/step - loss: 0.0495 - sparse_categorical_accuracy: 0.9847 - val_loss: 0.1141 - val_sparse_categorical_accuracy: 0.9639 - learning_rate: 1.2500e-04
## Epoch 218/500
## 90/90 - 0s - 2ms/step - loss: 0.0547 - sparse_categorical_accuracy: 0.9819 - val_loss: 0.0976 - val_sparse_categorical_accuracy: 0.9626 - learning_rate: 1.2500e-04
## Epoch 219/500
## 90/90 - 0s - 2ms/step - loss: 0.0480 - sparse_categorical_accuracy: 0.9868 - val_loss: 0.1387 - val_sparse_categorical_accuracy: 0.9556 - learning_rate: 1.2500e-04
## Epoch 220/500
## 90/90 - 0s - 1ms/step - loss: 0.0497 - sparse_categorical_accuracy: 0.9854 - val_loss: 0.1161 - val_sparse_categorical_accuracy: 0.9612 - learning_rate: 1.2500e-04
## Epoch 221/500
## 90/90 - 0s - 1ms/step - loss: 0.0516 - sparse_categorical_accuracy: 0.9813 - val_loss: 0.1338 - val_sparse_categorical_accuracy: 0.9542 - learning_rate: 1.2500e-04
## Epoch 222/500
## 90/90 - 0s - 1ms/step - loss: 0.0525 - sparse_categorical_accuracy: 0.9806 - val_loss: 0.0995 - val_sparse_categorical_accuracy: 0.9695 - learning_rate: 1.2500e-04
## Epoch 223/500
## 90/90 - 0s - 1ms/step - loss: 0.0504 - sparse_categorical_accuracy: 0.9823 - val_loss: 0.0979 - val_sparse_categorical_accuracy: 0.9570 - learning_rate: 1.2500e-04
## Epoch 224/500
## 90/90 - 0s - 1ms/step - loss: 0.0471 - sparse_categorical_accuracy: 0.9851 - val_loss: 0.0994 - val_sparse_categorical_accuracy: 0.9584 - learning_rate: 1.2500e-04
## Epoch 225/500
## 90/90 - 0s - 1ms/step - loss: 0.0536 - sparse_categorical_accuracy: 0.9837 - val_loss: 0.1145 - val_sparse_categorical_accuracy: 0.9667 - learning_rate: 1.2500e-04
## Epoch 226/500
## 90/90 - 0s - 1ms/step - loss: 0.0487 - sparse_categorical_accuracy: 0.9840 - val_loss: 0.1047 - val_sparse_categorical_accuracy: 0.9736 - learning_rate: 1.2500e-04
## Epoch 227/500
## 90/90 - 0s - 1ms/step - loss: 0.0529 - sparse_categorical_accuracy: 0.9813 - val_loss: 0.1087 - val_sparse_categorical_accuracy: 0.9542 - learning_rate: 1.2500e-04
## Epoch 228/500
## 90/90 - 0s - 1ms/step - loss: 0.0477 - sparse_categorical_accuracy: 0.9882 - val_loss: 0.0992 - val_sparse_categorical_accuracy: 0.9612 - learning_rate: 1.2500e-04
## Epoch 229/500
## 90/90 - 0s - 1ms/step - loss: 0.0499 - sparse_categorical_accuracy: 0.9854 - val_loss: 0.1064 - val_sparse_categorical_accuracy: 0.9681 - learning_rate: 1.2500e-04
## Epoch 230/500
## 90/90 - 0s - 2ms/step - loss: 0.0476 - sparse_categorical_accuracy: 0.9844 - val_loss: 0.0967 - val_sparse_categorical_accuracy: 0.9626 - learning_rate: 1.2500e-04
## Epoch 231/500
## 90/90 - 0s - 1ms/step - loss: 0.0527 - sparse_categorical_accuracy: 0.9816 - val_loss: 0.1730 - val_sparse_categorical_accuracy: 0.9390 - learning_rate: 1.2500e-04
## Epoch 232/500
## 90/90 - 0s - 1ms/step - loss: 0.0496 - sparse_categorical_accuracy: 0.9844 - val_loss: 0.1109 - val_sparse_categorical_accuracy: 0.9639 - learning_rate: 1.2500e-04
## Epoch 233/500
## 90/90 - 0s - 2ms/step - loss: 0.0483 - sparse_categorical_accuracy: 0.9840 - val_loss: 0.0963 - val_sparse_categorical_accuracy: 0.9612 - learning_rate: 1.2500e-04
## Epoch 234/500
## 90/90 - 0s - 1ms/step - loss: 0.0493 - sparse_categorical_accuracy: 0.9844 - val_loss: 0.1158 - val_sparse_categorical_accuracy: 0.9653 - learning_rate: 1.2500e-04
## Epoch 235/500
## 90/90 - 0s - 1ms/step - loss: 0.0493 - sparse_categorical_accuracy: 0.9837 - val_loss: 0.1057 - val_sparse_categorical_accuracy: 0.9667 - learning_rate: 1.2500e-04
## Epoch 236/500
## 90/90 - 0s - 1ms/step - loss: 0.0502 - sparse_categorical_accuracy: 0.9837 - val_loss: 0.0982 - val_sparse_categorical_accuracy: 0.9570 - learning_rate: 1.2500e-04
## Epoch 237/500
## 90/90 - 0s - 1ms/step - loss: 0.0501 - sparse_categorical_accuracy: 0.9819 - val_loss: 0.0992 - val_sparse_categorical_accuracy: 0.9695 - learning_rate: 1.2500e-04
## Epoch 238/500
## 90/90 - 0s - 1ms/step - loss: 0.0487 - sparse_categorical_accuracy: 0.9872 - val_loss: 0.1031 - val_sparse_categorical_accuracy: 0.9556 - learning_rate: 1.2500e-04
## Epoch 239/500
## 90/90 - 0s - 1ms/step - loss: 0.0491 - sparse_categorical_accuracy: 0.9844 - val_loss: 0.1139 - val_sparse_categorical_accuracy: 0.9515 - learning_rate: 1.2500e-04
## Epoch 240/500
## 90/90 - 0s - 2ms/step - loss: 0.0478 - sparse_categorical_accuracy: 0.9844 - val_loss: 0.0990 - val_sparse_categorical_accuracy: 0.9626 - learning_rate: 1.2500e-04
## Epoch 241/500
## 90/90 - 0s - 2ms/step - loss: 0.0498 - sparse_categorical_accuracy: 0.9830 - val_loss: 0.0994 - val_sparse_categorical_accuracy: 0.9570 - learning_rate: 1.2500e-04
## Epoch 242/500
## 90/90 - 0s - 2ms/step - loss: 0.0490 - sparse_categorical_accuracy: 0.9851 - val_loss: 0.1118 - val_sparse_categorical_accuracy: 0.9681 - learning_rate: 1.2500e-04
## Epoch 243/500
## 90/90 - 0s - 2ms/step - loss: 0.0523 - sparse_categorical_accuracy: 0.9823 - val_loss: 0.1052 - val_sparse_categorical_accuracy: 0.9653 - learning_rate: 1.2500e-04
## Epoch 244/500
## 90/90 - 0s - 2ms/step - loss: 0.0527 - sparse_categorical_accuracy: 0.9806 - val_loss: 0.1055 - val_sparse_categorical_accuracy: 0.9695 - learning_rate: 1.2500e-04
## Epoch 245/500
## 90/90 - 0s - 2ms/step - loss: 0.0476 - sparse_categorical_accuracy: 0.9865 - val_loss: 0.1045 - val_sparse_categorical_accuracy: 0.9542 - learning_rate: 1.2500e-04
## Epoch 246/500
## 90/90 - 0s - 1ms/step - loss: 0.0518 - sparse_categorical_accuracy: 0.9823 - val_loss: 0.1214 - val_sparse_categorical_accuracy: 0.9501 - learning_rate: 1.2500e-04
## Epoch 247/500
## 90/90 - 0s - 1ms/step - loss: 0.0457 - sparse_categorical_accuracy: 0.9875 - val_loss: 0.0964 - val_sparse_categorical_accuracy: 0.9653 - learning_rate: 1.2500e-04
## Epoch 248/500
## 90/90 - 0s - 1ms/step - loss: 0.0450 - sparse_categorical_accuracy: 0.9885 - val_loss: 0.1161 - val_sparse_categorical_accuracy: 0.9584 - learning_rate: 1.2500e-04
## Epoch 249/500
## 90/90 - 0s - 1ms/step - loss: 0.0492 - sparse_categorical_accuracy: 0.9875 - val_loss: 0.1282 - val_sparse_categorical_accuracy: 0.9570 - learning_rate: 1.2500e-04
## Epoch 250/500
## 90/90 - 0s - 2ms/step - loss: 0.0510 - sparse_categorical_accuracy: 0.9826 - val_loss: 0.0972 - val_sparse_categorical_accuracy: 0.9612 - learning_rate: 1.2500e-04
## Epoch 251/500
## 90/90 - 0s - 1ms/step - loss: 0.0506 - sparse_categorical_accuracy: 0.9833 - val_loss: 0.0986 - val_sparse_categorical_accuracy: 0.9653 - learning_rate: 1.2500e-04
## Epoch 252/500
## 90/90 - 0s - 2ms/step - loss: 0.0503 - sparse_categorical_accuracy: 0.9847 - val_loss: 0.1075 - val_sparse_categorical_accuracy: 0.9653 - learning_rate: 1.2500e-04
## Epoch 253/500
## 90/90 - 0s - 2ms/step - loss: 0.0507 - sparse_categorical_accuracy: 0.9837 - val_loss: 0.0999 - val_sparse_categorical_accuracy: 0.9612 - learning_rate: 1.2500e-04
## Epoch 254/500
## 90/90 - 0s - 2ms/step - loss: 0.0478 - sparse_categorical_accuracy: 0.9858 - val_loss: 0.0971 - val_sparse_categorical_accuracy: 0.9736 - learning_rate: 1.0000e-04
## Epoch 255/500
## 90/90 - 0s - 2ms/step - loss: 0.0464 - sparse_categorical_accuracy: 0.9844 - val_loss: 0.0954 - val_sparse_categorical_accuracy: 0.9653 - learning_rate: 1.0000e-04
## Epoch 256/500
## 90/90 - 0s - 2ms/step - loss: 0.0476 - sparse_categorical_accuracy: 0.9865 - val_loss: 0.0954 - val_sparse_categorical_accuracy: 0.9653 - learning_rate: 1.0000e-04
## Epoch 257/500
## 90/90 - 0s - 1ms/step - loss: 0.0472 - sparse_categorical_accuracy: 0.9875 - val_loss: 0.1047 - val_sparse_categorical_accuracy: 0.9736 - learning_rate: 1.0000e-04
## Epoch 258/500
## 90/90 - 0s - 1ms/step - loss: 0.0448 - sparse_categorical_accuracy: 0.9872 - val_loss: 0.1099 - val_sparse_categorical_accuracy: 0.9695 - learning_rate: 1.0000e-04
## Epoch 259/500
## 90/90 - 0s - 1ms/step - loss: 0.0447 - sparse_categorical_accuracy: 0.9868 - val_loss: 0.1245 - val_sparse_categorical_accuracy: 0.9570 - learning_rate: 1.0000e-04
## Epoch 260/500
## 90/90 - 0s - 1ms/step - loss: 0.0454 - sparse_categorical_accuracy: 0.9865 - val_loss: 0.1113 - val_sparse_categorical_accuracy: 0.9584 - learning_rate: 1.0000e-04
## Epoch 261/500
## 90/90 - 0s - 1ms/step - loss: 0.0488 - sparse_categorical_accuracy: 0.9840 - val_loss: 0.1697 - val_sparse_categorical_accuracy: 0.9404 - learning_rate: 1.0000e-04
## Epoch 262/500
## 90/90 - 0s - 1ms/step - loss: 0.0450 - sparse_categorical_accuracy: 0.9878 - val_loss: 0.0987 - val_sparse_categorical_accuracy: 0.9653 - learning_rate: 1.0000e-04
## Epoch 263/500
## 90/90 - 0s - 1ms/step - loss: 0.0439 - sparse_categorical_accuracy: 0.9875 - val_loss: 0.0975 - val_sparse_categorical_accuracy: 0.9695 - learning_rate: 1.0000e-04
## Epoch 264/500
## 90/90 - 0s - 1ms/step - loss: 0.0498 - sparse_categorical_accuracy: 0.9840 - val_loss: 0.1111 - val_sparse_categorical_accuracy: 0.9653 - learning_rate: 1.0000e-04
## Epoch 265/500
## 90/90 - 0s - 1ms/step - loss: 0.0523 - sparse_categorical_accuracy: 0.9809 - val_loss: 0.0983 - val_sparse_categorical_accuracy: 0.9695 - learning_rate: 1.0000e-04
## Epoch 266/500
## 90/90 - 0s - 1ms/step - loss: 0.0459 - sparse_categorical_accuracy: 0.9865 - val_loss: 0.0997 - val_sparse_categorical_accuracy: 0.9709 - learning_rate: 1.0000e-04
## Epoch 267/500
## 90/90 - 0s - 1ms/step - loss: 0.0462 - sparse_categorical_accuracy: 0.9868 - val_loss: 0.1002 - val_sparse_categorical_accuracy: 0.9709 - learning_rate: 1.0000e-04
## Epoch 268/500
## 90/90 - 0s - 1ms/step - loss: 0.0438 - sparse_categorical_accuracy: 0.9882 - val_loss: 0.1058 - val_sparse_categorical_accuracy: 0.9681 - learning_rate: 1.0000e-04
## Epoch 269/500
## 90/90 - 0s - 2ms/step - loss: 0.0491 - sparse_categorical_accuracy: 0.9872 - val_loss: 0.1289 - val_sparse_categorical_accuracy: 0.9556 - learning_rate: 1.0000e-04
## Epoch 270/500
## 90/90 - 0s - 2ms/step - loss: 0.0430 - sparse_categorical_accuracy: 0.9889 - val_loss: 0.1078 - val_sparse_categorical_accuracy: 0.9556 - learning_rate: 1.0000e-04
## Epoch 271/500
## 90/90 - 0s - 1ms/step - loss: 0.0454 - sparse_categorical_accuracy: 0.9875 - val_loss: 0.1096 - val_sparse_categorical_accuracy: 0.9681 - learning_rate: 1.0000e-04
## Epoch 272/500
## 90/90 - 0s - 1ms/step - loss: 0.0455 - sparse_categorical_accuracy: 0.9854 - val_loss: 0.1262 - val_sparse_categorical_accuracy: 0.9570 - learning_rate: 1.0000e-04
## Epoch 273/500
## 90/90 - 0s - 1ms/step - loss: 0.0469 - sparse_categorical_accuracy: 0.9844 - val_loss: 0.1129 - val_sparse_categorical_accuracy: 0.9626 - learning_rate: 1.0000e-04
## Epoch 274/500
## 90/90 - 0s - 1ms/step - loss: 0.0429 - sparse_categorical_accuracy: 0.9896 - val_loss: 0.1119 - val_sparse_categorical_accuracy: 0.9653 - learning_rate: 1.0000e-04
## Epoch 275/500
## 90/90 - 0s - 1ms/step - loss: 0.0430 - sparse_categorical_accuracy: 0.9865 - val_loss: 0.1392 - val_sparse_categorical_accuracy: 0.9542 - learning_rate: 1.0000e-04
## Epoch 276/500
## 90/90 - 0s - 1ms/step - loss: 0.0463 - sparse_categorical_accuracy: 0.9837 - val_loss: 0.0976 - val_sparse_categorical_accuracy: 0.9709 - learning_rate: 1.0000e-04
## Epoch 277/500
## 90/90 - 0s - 1ms/step - loss: 0.0457 - sparse_categorical_accuracy: 0.9878 - val_loss: 0.1036 - val_sparse_categorical_accuracy: 0.9695 - learning_rate: 1.0000e-04
## Epoch 278/500
## 90/90 - 0s - 1ms/step - loss: 0.0465 - sparse_categorical_accuracy: 0.9851 - val_loss: 0.1205 - val_sparse_categorical_accuracy: 0.9612 - learning_rate: 1.0000e-04
## Epoch 279/500
## 90/90 - 0s - 1ms/step - loss: 0.0473 - sparse_categorical_accuracy: 0.9858 - val_loss: 0.0958 - val_sparse_categorical_accuracy: 0.9695 - learning_rate: 1.0000e-04
## Epoch 280/500
## 90/90 - 0s - 2ms/step - loss: 0.0459 - sparse_categorical_accuracy: 0.9844 - val_loss: 0.0947 - val_sparse_categorical_accuracy: 0.9653 - learning_rate: 1.0000e-04
## Epoch 281/500
## 90/90 - 0s - 1ms/step - loss: 0.0438 - sparse_categorical_accuracy: 0.9861 - val_loss: 0.1216 - val_sparse_categorical_accuracy: 0.9598 - learning_rate: 1.0000e-04
## Epoch 282/500
## 90/90 - 0s - 1ms/step - loss: 0.0419 - sparse_categorical_accuracy: 0.9878 - val_loss: 0.1008 - val_sparse_categorical_accuracy: 0.9695 - learning_rate: 1.0000e-04
## Epoch 283/500
## 90/90 - 0s - 1ms/step - loss: 0.0481 - sparse_categorical_accuracy: 0.9854 - val_loss: 0.1129 - val_sparse_categorical_accuracy: 0.9612 - learning_rate: 1.0000e-04
## Epoch 284/500
## 90/90 - 0s - 1ms/step - loss: 0.0480 - sparse_categorical_accuracy: 0.9847 - val_loss: 0.1418 - val_sparse_categorical_accuracy: 0.9542 - learning_rate: 1.0000e-04
## Epoch 285/500
## 90/90 - 0s - 1ms/step - loss: 0.0467 - sparse_categorical_accuracy: 0.9844 - val_loss: 0.1058 - val_sparse_categorical_accuracy: 0.9709 - learning_rate: 1.0000e-04
## Epoch 286/500
## 90/90 - 0s - 1ms/step - loss: 0.0449 - sparse_categorical_accuracy: 0.9837 - val_loss: 0.1039 - val_sparse_categorical_accuracy: 0.9723 - learning_rate: 1.0000e-04
## Epoch 287/500
## 90/90 - 0s - 1ms/step - loss: 0.0450 - sparse_categorical_accuracy: 0.9872 - val_loss: 0.1015 - val_sparse_categorical_accuracy: 0.9695 - learning_rate: 1.0000e-04
## Epoch 288/500
## 90/90 - 0s - 2ms/step - loss: 0.0447 - sparse_categorical_accuracy: 0.9865 - val_loss: 0.1156 - val_sparse_categorical_accuracy: 0.9612 - learning_rate: 1.0000e-04
## Epoch 289/500
## 90/90 - 0s - 1ms/step - loss: 0.0460 - sparse_categorical_accuracy: 0.9844 - val_loss: 0.1000 - val_sparse_categorical_accuracy: 0.9681 - learning_rate: 1.0000e-04
## Epoch 290/500
## 90/90 - 0s - 1ms/step - loss: 0.0482 - sparse_categorical_accuracy: 0.9865 - val_loss: 0.0967 - val_sparse_categorical_accuracy: 0.9653 - learning_rate: 1.0000e-04
## Epoch 291/500
## 90/90 - 0s - 1ms/step - loss: 0.0433 - sparse_categorical_accuracy: 0.9882 - val_loss: 0.1002 - val_sparse_categorical_accuracy: 0.9570 - learning_rate: 1.0000e-04
## Epoch 292/500
## 90/90 - 0s - 1ms/step - loss: 0.0506 - sparse_categorical_accuracy: 0.9830 - val_loss: 0.1102 - val_sparse_categorical_accuracy: 0.9542 - learning_rate: 1.0000e-04
## Epoch 293/500
## 90/90 - 0s - 1ms/step - loss: 0.0473 - sparse_categorical_accuracy: 0.9868 - val_loss: 0.1139 - val_sparse_categorical_accuracy: 0.9542 - learning_rate: 1.0000e-04
## Epoch 294/500
## 90/90 - 0s - 1ms/step - loss: 0.0459 - sparse_categorical_accuracy: 0.9861 - val_loss: 0.1005 - val_sparse_categorical_accuracy: 0.9584 - learning_rate: 1.0000e-04
## Epoch 295/500
## 90/90 - 0s - 1ms/step - loss: 0.0502 - sparse_categorical_accuracy: 0.9854 - val_loss: 0.0978 - val_sparse_categorical_accuracy: 0.9695 - learning_rate: 1.0000e-04
## Epoch 296/500
## 90/90 - 0s - 1ms/step - loss: 0.0477 - sparse_categorical_accuracy: 0.9851 - val_loss: 0.1006 - val_sparse_categorical_accuracy: 0.9667 - learning_rate: 1.0000e-04
## Epoch 297/500
## 90/90 - 0s - 1ms/step - loss: 0.0474 - sparse_categorical_accuracy: 0.9868 - val_loss: 0.0978 - val_sparse_categorical_accuracy: 0.9612 - learning_rate: 1.0000e-04
## Epoch 298/500
## 90/90 - 0s - 1ms/step - loss: 0.0465 - sparse_categorical_accuracy: 0.9865 - val_loss: 0.0953 - val_sparse_categorical_accuracy: 0.9667 - learning_rate: 1.0000e-04
## Epoch 299/500
## 90/90 - 0s - 1ms/step - loss: 0.0410 - sparse_categorical_accuracy: 0.9896 - val_loss: 0.1107 - val_sparse_categorical_accuracy: 0.9653 - learning_rate: 1.0000e-04
## Epoch 300/500
## 90/90 - 0s - 1ms/step - loss: 0.0423 - sparse_categorical_accuracy: 0.9875 - val_loss: 0.1099 - val_sparse_categorical_accuracy: 0.9639 - learning_rate: 1.0000e-04
## Epoch 301/500
## 90/90 - 0s - 1ms/step - loss: 0.0457 - sparse_categorical_accuracy: 0.9851 - val_loss: 0.0983 - val_sparse_categorical_accuracy: 0.9598 - learning_rate: 1.0000e-04
## Epoch 302/500
## 90/90 - 0s - 1ms/step - loss: 0.0422 - sparse_categorical_accuracy: 0.9889 - val_loss: 0.1233 - val_sparse_categorical_accuracy: 0.9598 - learning_rate: 1.0000e-04
## Epoch 303/500
## 90/90 - 0s - 2ms/step - loss: 0.0417 - sparse_categorical_accuracy: 0.9896 - val_loss: 0.1175 - val_sparse_categorical_accuracy: 0.9626 - learning_rate: 1.0000e-04
## Epoch 304/500
## 90/90 - 0s - 2ms/step - loss: 0.0467 - sparse_categorical_accuracy: 0.9861 - val_loss: 0.1007 - val_sparse_categorical_accuracy: 0.9723 - learning_rate: 1.0000e-04
## Epoch 305/500
## 90/90 - 0s - 2ms/step - loss: 0.0466 - sparse_categorical_accuracy: 0.9851 - val_loss: 0.1058 - val_sparse_categorical_accuracy: 0.9695 - learning_rate: 1.0000e-04
## Epoch 306/500
## 90/90 - 0s - 1ms/step - loss: 0.0402 - sparse_categorical_accuracy: 0.9899 - val_loss: 0.0999 - val_sparse_categorical_accuracy: 0.9667 - learning_rate: 1.0000e-04
## Epoch 307/500
## 90/90 - 0s - 2ms/step - loss: 0.0465 - sparse_categorical_accuracy: 0.9844 - val_loss: 0.1130 - val_sparse_categorical_accuracy: 0.9528 - learning_rate: 1.0000e-04
## Epoch 308/500
## 90/90 - 0s - 1ms/step - loss: 0.0463 - sparse_categorical_accuracy: 0.9858 - val_loss: 0.0987 - val_sparse_categorical_accuracy: 0.9626 - learning_rate: 1.0000e-04
## Epoch 309/500
## 90/90 - 0s - 1ms/step - loss: 0.0472 - sparse_categorical_accuracy: 0.9851 - val_loss: 0.1179 - val_sparse_categorical_accuracy: 0.9612 - learning_rate: 1.0000e-04
## Epoch 310/500
## 90/90 - 0s - 1ms/step - loss: 0.0447 - sparse_categorical_accuracy: 0.9854 - val_loss: 0.1036 - val_sparse_categorical_accuracy: 0.9709 - learning_rate: 1.0000e-04
## Epoch 311/500
## 90/90 - 0s - 1ms/step - loss: 0.0448 - sparse_categorical_accuracy: 0.9854 - val_loss: 0.1012 - val_sparse_categorical_accuracy: 0.9681 - learning_rate: 1.0000e-04
## Epoch 312/500
## 90/90 - 0s - 1ms/step - loss: 0.0442 - sparse_categorical_accuracy: 0.9851 - val_loss: 0.1172 - val_sparse_categorical_accuracy: 0.9598 - learning_rate: 1.0000e-04
## Epoch 313/500
## 90/90 - 0s - 2ms/step - loss: 0.0438 - sparse_categorical_accuracy: 0.9889 - val_loss: 0.0990 - val_sparse_categorical_accuracy: 0.9695 - learning_rate: 1.0000e-04
## Epoch 314/500
## 90/90 - 0s - 2ms/step - loss: 0.0432 - sparse_categorical_accuracy: 0.9875 - val_loss: 0.0979 - val_sparse_categorical_accuracy: 0.9695 - learning_rate: 1.0000e-04
## Epoch 315/500
## 90/90 - 0s - 1ms/step - loss: 0.0474 - sparse_categorical_accuracy: 0.9858 - val_loss: 0.0980 - val_sparse_categorical_accuracy: 0.9584 - learning_rate: 1.0000e-04
## Epoch 316/500
## 90/90 - 0s - 1ms/step - loss: 0.0429 - sparse_categorical_accuracy: 0.9889 - val_loss: 0.1030 - val_sparse_categorical_accuracy: 0.9570 - learning_rate: 1.0000e-04
## Epoch 317/500
## 90/90 - 0s - 3ms/step - loss: 0.0447 - sparse_categorical_accuracy: 0.9840 - val_loss: 0.1018 - val_sparse_categorical_accuracy: 0.9584 - learning_rate: 1.0000e-04
## Epoch 318/500
## 90/90 - 0s - 2ms/step - loss: 0.0423 - sparse_categorical_accuracy: 0.9868 - val_loss: 0.1024 - val_sparse_categorical_accuracy: 0.9723 - learning_rate: 1.0000e-04
## Epoch 319/500
## 90/90 - 0s - 2ms/step - loss: 0.0474 - sparse_categorical_accuracy: 0.9854 - val_loss: 0.1197 - val_sparse_categorical_accuracy: 0.9667 - learning_rate: 1.0000e-04
## Epoch 320/500
## 90/90 - 0s - 2ms/step - loss: 0.0405 - sparse_categorical_accuracy: 0.9889 - val_loss: 0.0994 - val_sparse_categorical_accuracy: 0.9736 - learning_rate: 1.0000e-04
## Epoch 321/500
## 90/90 - 0s - 2ms/step - loss: 0.0442 - sparse_categorical_accuracy: 0.9878 - val_loss: 0.1499 - val_sparse_categorical_accuracy: 0.9528 - learning_rate: 1.0000e-04
## Epoch 322/500
## 90/90 - 0s - 2ms/step - loss: 0.0404 - sparse_categorical_accuracy: 0.9889 - val_loss: 0.1253 - val_sparse_categorical_accuracy: 0.9584 - learning_rate: 1.0000e-04
## Epoch 323/500
## 90/90 - 0s - 1ms/step - loss: 0.0468 - sparse_categorical_accuracy: 0.9844 - val_loss: 0.1081 - val_sparse_categorical_accuracy: 0.9709 - learning_rate: 1.0000e-04
## Epoch 324/500
## 90/90 - 0s - 1ms/step - loss: 0.0418 - sparse_categorical_accuracy: 0.9858 - val_loss: 0.1016 - val_sparse_categorical_accuracy: 0.9626 - learning_rate: 1.0000e-04
## Epoch 325/500
## 90/90 - 0s - 1ms/step - loss: 0.0420 - sparse_categorical_accuracy: 0.9868 - val_loss: 0.1290 - val_sparse_categorical_accuracy: 0.9501 - learning_rate: 1.0000e-04
## Epoch 326/500
## 90/90 - 0s - 2ms/step - loss: 0.0436 - sparse_categorical_accuracy: 0.9854 - val_loss: 0.1270 - val_sparse_categorical_accuracy: 0.9515 - learning_rate: 1.0000e-04
## Epoch 327/500
## 90/90 - 0s - 1ms/step - loss: 0.0417 - sparse_categorical_accuracy: 0.9865 - val_loss: 0.1009 - val_sparse_categorical_accuracy: 0.9695 - learning_rate: 1.0000e-04
## Epoch 328/500
## 90/90 - 0s - 2ms/step - loss: 0.0447 - sparse_categorical_accuracy: 0.9868 - val_loss: 0.1141 - val_sparse_categorical_accuracy: 0.9667 - learning_rate: 1.0000e-04
## Epoch 329/500
## 90/90 - 0s - 2ms/step - loss: 0.0422 - sparse_categorical_accuracy: 0.9885 - val_loss: 0.1198 - val_sparse_categorical_accuracy: 0.9570 - learning_rate: 1.0000e-04
## Epoch 330/500
## 90/90 - 0s - 3ms/step - loss: 0.0435 - sparse_categorical_accuracy: 0.9868 - val_loss: 0.0988 - val_sparse_categorical_accuracy: 0.9612 - learning_rate: 1.0000e-04
## Epoch 330: early stopping

Evaluate model on test data

model <- load_model("best_model.keras")

results <- model |> evaluate(x_test, y_test)

## 42/42 - 1s - 15ms/step - loss: 0.0980 - sparse_categorical_accuracy: 0.9682

str(results)

## List of 2
##  $ loss                       : num 0.098
##  $ sparse_categorical_accuracy: num 0.968

cat(
  "Test accuracy: ", results$sparse_categorical_accuracy, "\n",
  "Test loss: ", results$loss, "\n",
  sep = ""
)

## Test accuracy: 0.9681818
## Test loss: 0.09795157

Plot the model's training history

plot(history)

Plot of Training History Metrics

Plot just the training and validation accuracy:

plot(history, metric = "sparse_categorical_accuracy") +
  # scale x axis to actual number of epochs run before early stopping
  ggplot2::xlim(0, length(history$metrics$loss))

Plot of Accuracy During Training

We can see how the training accuracy reaches almost 0.95 after 100 epochs. However, by observing the validation accuracy we can see how the network still needs training until it reaches almost 0.97 for both the validation and the training accuracy after 200 epochs. Beyond the 200th epoch, if we continue on training, the validation accuracy will start decreasing while the training accuracy will continue on increasing: the model starts overfitting.

rstudio/keras documentation built on July 8, 2024, 3:07 p.m.

rdrr.io home R language documentation Run R code online

CRAN packages Bioconductor packages R-Forge packages GitHub packages

Note that we can't provide technical support on individual packages. You should contact the package authors for that.

Tweet to @rdrrHQ

GitHub issue tracker

ian@mutexlabs.com