3-3-10-TFLiteOptimizations.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "3-3-10-TFLiteOptimizations.ipynb",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZQfJvzOBh0e-"
      },
      "source": [
        "# Cats versus Dogs\n",
        "Exploring optimization impact on size, performance and accuracy!\n",
        "\n",
        "**For best performance on this Colab make sure you are using a GPU runtime!** The runtime can be changed by selecting: ```runtime -> change runtime type``` and selecting GPU from the hardware accelerator dropdown."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aW0PlnQDFOij"
      },
      "source": [
        "### We start by importing TensorFlow and our Dataset\n",
        "And splitting our dataset into batches."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "D1J15Vh_1Jih",
        "cellView": "both"
      },
      "source": [
        "try:\n",
        "  # %tensorflow_version only exists in Colab.\n",
        "  %tensorflow_version 2.x\n",
        "except Exception:\n",
        "  pass\n"
      ],
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sqNRQoc7W17k"
      },
      "source": [
        "import numpy as np\n",
        "import matplotlib.pylab as plt\n",
        "\n",
        "import tensorflow as tf\n",
        "import tensorflow_hub as hub\n",
        "import tensorflow_datasets as tfds\n",
        "\n",
        "# format images to have normalized pixels\n",
        "def format_image(image, label):\n",
        "    image = tf.image.resize(image, (224, 224)) / 255.0\n",
        "    return  image, label\n",
        "\n",
        "# load in our dataset\n",
        "(raw_train, raw_validation, raw_test), metadata = tfds.load(\n",
        "    'cats_vs_dogs',\n",
        "    split=['train[:80%]', 'train[80%:90%]', 'train[90%:]'],\n",
        "    with_info=True,\n",
        "    as_supervised=True,\n",
        ")\n",
        "\n",
        "# display how much data we have\n",
        "num_examples = metadata.splits['train'].num_examples\n",
        "num_classes = metadata.features['label'].num_classes\n",
        "print(num_examples)\n",
        "print(num_classes)\n",
        "\n",
        "# split the data in training, validation, and test datasets\n",
        "BATCH_SIZE = 32\n",
        "train_batches = raw_train.shuffle(num_examples // 4).map(format_image).batch(BATCH_SIZE).prefetch(1)\n",
        "validation_batches = raw_validation.map(format_image).batch(BATCH_SIZE).prefetch(1)\n",
        "test_batches = raw_test.map(format_image).batch(1)\n",
        "\n",
        "# display the shape of our data\n",
        "for image_batch, label_batch in train_batches.take(1):\n",
        "    pass\n",
        "image_batch.shape"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kFpqPEIbF9lx"
      },
      "source": [
        "### We next define our (pre-trained) model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "inIK227eZbgV"
      },
      "source": [
        "module_selection = (\"mobilenet_v2\", 224, 1280) \n",
        "handle_base, pixels, FV_SIZE = module_selection\n",
        "MODULE_HANDLE =\"https://tfhub.dev/google/tf2-preview/{}/feature_vector/4\".format(handle_base)\n",
        "IMAGE_SIZE = (pixels, pixels)\n",
        "print(\"Using {} with input size {} and output dimension {}\".format(MODULE_HANDLE, IMAGE_SIZE, FV_SIZE))\n",
        "\n",
        "feature_extractor = hub.KerasLayer(MODULE_HANDLE,\n",
        "                                   input_shape=IMAGE_SIZE + (3,), \n",
        "                                   output_shape=[FV_SIZE],\n",
        "                                   trainable=False)\n",
        "\n",
        "print(\"Building model with\", MODULE_HANDLE)\n",
        "\n",
        "model = tf.keras.Sequential([\n",
        "        feature_extractor,\n",
        "        tf.keras.layers.Dense(num_classes, activation='softmax')\n",
        "])\n",
        "\n",
        "model.summary()\n",
        "\n",
        "model.compile(optimizer='adam',\n",
        "                  loss='sparse_categorical_crossentropy',\n",
        "                  metrics=['accuracy'])"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rFm6YokEGOew"
      },
      "source": [
        "### We then train and save our model\n",
        "Since we are doing transfer learning to fine tune a pre-trained model to our dataset we only need to use 5 Epochs. We will explore transfer learning in detail and study how, why, and where it works later in this course!"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hFdUvxZUGLji"
      },
      "source": [
        "EPOCHS = 5\n",
        "\n",
        "hist = model.fit(train_batches,\n",
        "                 epochs=EPOCHS,\n",
        "                 validation_data=validation_batches)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cML-V4yqtuYI"
      },
      "source": [
        "CATS_VS_DOGS_SAVED_MODEL = \"exp_saved_model\"\n",
        "tf.saved_model.save(model, CATS_VS_DOGS_SAVED_MODEL)\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_BGe8CF7aruS"
      },
      "source": [
        "### Your task starts here:\n",
        "+ Run this code, and you'll have model1.tflite, with no optimization or quantization\n",
        "\n",
        "+ Then, remove the comment on the ```converter.optimizations = []``` line. Change the model name to model2.tflite, and rerun. Model2.tflite will now have optimizations added -- you should see a much smaller file size.\n",
        "\n",
        "+ Finally, remove the comments on the code to add a representative dataset and set the supported ops as shown. Change the model name to model3.tflite, and rerun. Model3.tflite will now have optimizations added, along with quantization from the representative dataset. Note: it might be slightly larger than model2.tflite!",
        "\n",
        "\n",
        "**Note:** tf.lite.Optimize has changed and the OPTIMIZE_FOR_SIZE and OPTIMIZE_FOR_LATENCY options are now deprecated and are the same and DEFAULT: https://www.tensorflow.org/api_docs/python/tf/lite/Optimize"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gButZXqZt3o4"
      },
      "source": [
        "import pathlib\n",
        "converter = tf.lite.TFLiteConverter.from_saved_model(CATS_VS_DOGS_SAVED_MODEL)\n",
        "\n",
        "# These options are for converter optimizaitons\n",
        "# Consider trying the converter without them and\n",
        "# explore model size and accuracy\n",
        "# Then...use them and reconvert the model and explore model\n",
        "# size an accuracy at that point. What differences do you see?\n",
        "\n",
        "# converter.optimizations = [tf.lite.Optimize.DEFAULT]    # Uncomment this line for Model 2 and Model 3\n",
        "\n",
        "# def representative_data_gen():                          # Uncomment the following 5 lines for Model 3\n",
        "#     for input_value, _ in test_batches.take(100):\n",
        "#         yield [input_value]\n",
        "# converter.representative_dataset = representative_data_gen\n",
        "# converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]\n",
        "\n",
        "tflite_model = converter.convert()\n",
        "tflite_models_dir = pathlib.Path(\"/tmp/\")\n",
        "\n",
        "tflite_model_file = tflite_models_dir/'model1.tflite'     # Change the filename here for Model2 and Model3!\n",
        "tflite_model_file.write_bytes(tflite_model)\n",
        "\n",
        "\n",
        "# Without any optimizations I got\n",
        "# 8857848  (model1.tflite)\n",
        "# With the .optimizations property set I got\n",
        "# 2629648 (model2.tflite)\n",
        "# With the .optimizations property and representative data set I got\n",
        "# 2835952 -- Slightly larger!  (model3.tflite)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0RTZmndkcZFP"
      },
      "source": [
        "### Your task continues\n",
        "Now we will test the accuracy of the three models! After you run each model you will get the number of correct predictions and then you can plot which images were correct/incorrect!\n",
        "+ Run this code\n",
        "+ Change the model file to model2.tflite and run it again\n",
        "+ Change the model file to model3.tflite and run it again"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MsNjuPhxuDOx"
      },
      "source": [
        "#@title Run this cell each time to test your model's accuracy (make sure to change the filename)\n",
        "from tqdm import tqdm\n",
        "# Load TFLite model and allocate tensors.\n",
        "tflite_model_file = '/tmp/model1.tflite'                 # Change the filename here for Model 2 and 3\n",
        "interpreter = tf.lite.Interpreter(model_path=tflite_model_file)\n",
        "interpreter.allocate_tensors()\n",
        "\n",
        "input_index = interpreter.get_input_details()[0][\"index\"]\n",
        "output_index = interpreter.get_output_details()[0][\"index\"]\n",
        "\n",
        "predictions = []\n",
        "\n",
        "test_labels, test_imgs = [], []\n",
        "for img, label in tqdm(test_batches.take(100)):\n",
        "    interpreter.set_tensor(input_index, img)\n",
        "    interpreter.invoke()\n",
        "    predictions.append(interpreter.get_tensor(output_index))\n",
        "    \n",
        "    test_labels.append(label.numpy()[0])\n",
        "    test_imgs.append(img)\n",
        "\n",
        "# For model 1, I got 32.25 it/s\n",
        "# For model 2, I got 16.16 it/s\n",
        "# For model 3, I got 1.19s it/s\n",
        "# Note: since the it/s will depend on the computer on which your Colab VM\n",
        "#       instance is running -- we would expect it to vary a bit.\n",
        "# Note2: Changes have been made to the TFLite Interpreter since Laurence filmed the\n",
        "#        previous video that further optimize it for mobile use at the expense of\n",
        "#        speed in Colab. As such, you'll find that while Laurence was able to achieve\n", 
        "#        16 it/s for model 2, you may only see speeds of 1-2 it/s\n",
        "\n",
        "score = 0\n",
        "for item in range(0,100):\n",
        "  prediction=np.argmax(predictions[item])\n",
        "  label = test_labels[item]\n",
        "  if prediction==label:\n",
        "    score=score+1\n",
        "\n",
        "print(\"Out of 100 predictions I got \" + str(score) + \" correct\")\n",
        "\n",
        "# Model 1 - 100 Correct\n",
        "# Model 2 - 99 Correct\n",
        "# Model 3 - 99 Correct\n",
        "# Note: since training starts from a random intialization it would not be \n",
        "#       surprising if your result is off by 1 or 2 correct."
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gRgWsPmDuJEI"
      },
      "source": [
        "#@title Define utility functions once for plotting\n",
        "# Utilities for plotting\n",
        "\n",
        "class_names = ['cat', 'dog']\n",
        "\n",
        "def plot_image(i, predictions_array, true_label, img):\n",
        "    predictions_array, true_label, img = predictions_array[i], true_label[i], img[i]\n",
        "    plt.grid(False)\n",
        "    plt.xticks([])\n",
        "    plt.yticks([])\n",
        "    \n",
        "    img = np.squeeze(img)\n",
        "\n",
        "    plt.imshow(img, cmap=plt.cm.binary)\n",
        "    \n",
        "    predicted_label = np.argmax(predictions_array)\n",
        "    \n",
        "    if predicted_label == true_label:\n",
        "        color = 'green'\n",
        "    else:\n",
        "        color = 'red'\n",
        "    \n",
        "    plt.xlabel(\"{} {:2.0f}% ({})\".format(class_names[predicted_label],\n",
        "                                         100*np.max(predictions_array),\n",
        "                                         class_names[true_label]), color=color)"
      ],
      "execution_count": 8,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TylXFi0quLEw"
      },
      "source": [
        "#@title Visualize the outputs each time { run: \"auto\" }\n",
        "max_index = 15 #@param {type:\"slider\", min:1, max:100, step:1}\n",
        "for index in range(0,max_index):\n",
        "  plt.figure(figsize=(6,3))\n",
        "  plt.subplot(1,2,1)\n",
        "  plot_image(index, predictions, test_labels, test_imgs)\n",
        "  plt.show()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "G5Sathr4KBXp"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}