{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## TensorFlow 튜토리얼 - 전문가를 위한 TensorFlow 2 빠른 시작\n---\n참고 : [전문가를 위한 TensorFlow 2 빠른 시작](https://www.tensorflow.org/tutorials/quickstart/advanced)","metadata":{}},{"cell_type":"markdown","source":"## 데이터셋 준비\n---","metadata":{}},{"cell_type":"code","source":"import tensorflow as tf\n\nmnist = tf.keras.datasets.mnist\n\n(x_train, y_train), (x_test, y_test) = mnist.load_data()\nx_train, x_test = x_train / 255.0, x_test / 255.0\n\nprint(x_train.shape)\n\n# Add a channels dimension\n# 컨볼루션할 수 있게 한 차원 추가 : (28, 28) -> (28, 28, 1)\nx_train = x_train[..., tf.newaxis].astype(\"float32\")\nx_test = x_test[..., tf.newaxis].astype(\"float32\")\n\nprint(x_train.shape)","metadata":{"scrolled":true,"execution":{"iopub.status.busy":"2022-05-28T02:42:28.196275Z","iopub.execute_input":"2022-05-28T02:42:28.197403Z","iopub.status.idle":"2022-05-28T02:42:31.318197Z","shell.execute_reply.started":"2022-05-28T02:42:28.197329Z","shell.execute_reply":"2022-05-28T02:42:31.317046Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"데이터셋 섞고 배치 만들기   \n- [`from_tensor_slices()`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset#from_tensor_slices) : Creates a `Dataset` whose elements are slices of the given tensors.","metadata":{}},{"cell_type":"code","source":"train_ds = tf.data.Dataset.from_tensor_slices((x_train, y_train)).shuffle(10000).batch(32)\ntest_ds = tf.data.Dataset.from_tensor_slices((x_test, y_test)).batch(32)\n\nprint(type(train_ds))  # <class 'tensorflow.python.data.ops.dataset_ops.BatchDataset'>","metadata":{"execution":{"iopub.status.busy":"2022-05-28T02:42:31.324541Z","iopub.execute_input":"2022-05-28T02:42:31.325340Z","iopub.status.idle":"2022-05-28T02:42:31.566390Z","shell.execute_reply.started":"2022-05-28T02:42:31.325291Z","shell.execute_reply":"2022-05-28T02:42:31.565374Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 모델\n---\n케라스(Keras)의 [모델 서브클래싱(subclassing) API](https://www.tensorflow.org/guide/keras#model_subclassing)를 사용하여 tf.keras 모델을 만든다.   \n참고 : [하위 클래스화를 통한 새로운 레이어 및 모델 만들기](https://www.tensorflow.org/guide/keras/custom_layers_and_models), [기능적 API](https://www.tensorflow.org/guide/keras/functional/)","metadata":{}},{"cell_type":"code","source":"from tensorflow.keras import Model\nfrom tensorflow.keras.layers import Dense, Flatten, Conv2D\n\nclass MyModel(Model):\n    def __init__(self):\n        super(MyModel, self).__init__()\n        self.conv1 = Conv2D(32, 3, activation='relu')\n        self.flatten = Flatten()\n        self.d1 = Dense(128, activation='relu')\n        self.d2 = Dense(10)\n\n    def call(self, x):\n        x = self.conv1(x)\n        x = self.flatten(x)\n        x = self.d1(x)\n        return self.d2(x)\n\n# Create an instance of the model\nmodel = MyModel()","metadata":{"execution":{"iopub.status.busy":"2022-05-28T02:42:31.567688Z","iopub.execute_input":"2022-05-28T02:42:31.568659Z","iopub.status.idle":"2022-05-28T02:42:31.595745Z","shell.execute_reply.started":"2022-05-28T02:42:31.568619Z","shell.execute_reply":"2022-05-28T02:42:31.594850Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"훈련에 필요한 옵티마이저(optimizer)와 손실 함수 선택   \n- [`tf.keras.losses.SparseCategoricalCrossentropy()`](https://www.tensorflow.org/api_docs/python/tf/keras/losses/SparseCategoricalCrossentropy) : 라벨이 `(0, 1, 2, 3, 4...)`와 같이 정수로 주어지는 멀티 클래스 분류에 사용   \n    - `from_logits` : Whether `y_pred` is expected to be a logits tensor. By default, we assume that `y_pred` encodes a probability distribution.\n        - [`logit`](https://i-am-eden.tistory.com/21) : 아주 간단하게 말해 시그모이드의 역함수","metadata":{}},{"cell_type":"code","source":"loss_object = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)\noptimizer = tf.keras.optimizers.Adam()","metadata":{"execution":{"iopub.status.busy":"2022-05-28T02:42:31.597685Z","iopub.execute_input":"2022-05-28T02:42:31.598487Z","iopub.status.idle":"2022-05-28T02:42:31.602994Z","shell.execute_reply.started":"2022-05-28T02:42:31.598450Z","shell.execute_reply":"2022-05-28T02:42:31.602128Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"모델의 손실과 성능을 측정할 지표를 선택한다. 에포크가 진행되는 동안 수집된 측정 지표를 바탕으로 최종 결과를 출력한다.   \n- [`tf.keras.metrics.Mean()`](https://www.tensorflow.org/api_docs/python/tf/keras/metrics/Mean) : Computes the (weighted) mean of the given values.   \n- [`tf.keras.metrics.SparseCategoricalAccuracy()`](https://www.tensorflow.org/api_docs/python/tf/keras/metrics/SparseCategoricalAccuracy) : Calculates how often predictions match integer labels.","metadata":{}},{"cell_type":"code","source":"train_loss = tf.keras.metrics.Mean(name='train_loss')\ntrain_accuracy = tf.keras.metrics.SparseCategoricalAccuracy(name='train_accuracy')\n\ntest_loss = tf.keras.metrics.Mean(name='test_loss')\ntest_accuracy = tf.keras.metrics.SparseCategoricalAccuracy(name='test_accuracy')","metadata":{"execution":{"iopub.status.busy":"2022-05-28T02:42:31.604300Z","iopub.execute_input":"2022-05-28T02:42:31.604740Z","iopub.status.idle":"2022-05-28T02:42:31.827680Z","shell.execute_reply.started":"2022-05-28T02:42:31.604704Z","shell.execute_reply":"2022-05-28T02:42:31.826655Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"`tf.GradientTape`를 사용하여 모델을 훈련한다.\n- [`tf.GradientTape()`](https://www.tensorflow.org/api_docs/python/tf/GradientTape) : Record operations for automatic differentiation. -> 자동으로 미분을 해준다는 것 같음.   \n- 참고 : [그래디언트 및 자동 미분 소개](https://www.tensorflow.org/guide/autodiff?hl=ko)","metadata":{}},{"cell_type":"code","source":"@tf.function\ndef train_step(images, labels):\n    with tf.GradientTape() as tape:\n        # training=True is only needed if there are layers with different\n        # behavior during training versus inference (e.g. Dropout).\n        predictions = model(images, training=True)\n        loss = loss_object(labels, predictions)\n    gradients = tape.gradient(loss, model.trainable_variables)\n    optimizer.apply_gradients(zip(gradients, model.trainable_variables))\n\n    train_loss(loss)\n    train_accuracy(labels, predictions)","metadata":{"execution":{"iopub.status.busy":"2022-05-28T02:42:31.829079Z","iopub.execute_input":"2022-05-28T02:42:31.829407Z","iopub.status.idle":"2022-05-28T02:42:31.837318Z","shell.execute_reply.started":"2022-05-28T02:42:31.829377Z","shell.execute_reply":"2022-05-28T02:42:31.836557Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"모델 테스트","metadata":{}},{"cell_type":"code","source":"@tf.function\ndef test_step(images, labels):\n    # training=False is only needed if there are layers with different\n    # behavior during training versus inference (e.g. Dropout).\n    predictions = model(images, training=False)\n    t_loss = loss_object(labels, predictions)\n\n    test_loss(t_loss)\n    test_accuracy(labels, predictions)","metadata":{"execution":{"iopub.status.busy":"2022-05-28T02:42:31.838262Z","iopub.execute_input":"2022-05-28T02:42:31.838849Z","iopub.status.idle":"2022-05-28T02:42:31.856335Z","shell.execute_reply.started":"2022-05-28T02:42:31.838809Z","shell.execute_reply":"2022-05-28T02:42:31.855304Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- [`reset_states()`](https://www.tensorflow.org/guide/keras/customizing_what_happens_in_fit?hl=ko#%EB%8D%94_%EB%82%AE%EC%9D%80_%EC%88%98%EC%A4%80%EC%9C%BC%EB%A1%9C_%EA%B5%AC%EC%84%B1%ED%95%98%EA%B8%B0) : 다음으로 학습할 데이터가 이전의 학습 데이터와 완전히 무관할 때 사용한다. 즉 각각의 배치가 서로 독립적일 때 사용한다. ([출처](https://hoony-gunputer.tistory.com/entry/Rnnlstm%EC%9D%84-%EC%9D%B4%EC%9A%A9%ED%95%B4%EC%84%9C-%EC%95%85%EB%B3%B4%EC%98%88%EC%B8%A1%ED%95%B4%EB%B3%B4%EA%B8%B0))","metadata":{}},{"cell_type":"code","source":"import numpy as np\n\nEPOCHS = 5\n\nfor epoch in range(EPOCHS):\n    # Reset the metrics at the start of the next epoch\n    train_loss.reset_states()\n    train_accuracy.reset_states()\n    test_loss.reset_states()\n    test_accuracy.reset_states()\n\n    for images, labels in train_ds:\n        train_step(images, labels)\n\n    for test_images, test_labels in test_ds:\n        test_step(test_images, test_labels)\n\n    print(\n        f'Epoch {epoch + 1}, '\n        f'Loss: {train_loss.result().numpy():.4f}, '\n        f'Accuracy: {(train_accuracy.result().numpy() * 100):.2f}, '\n        f'Test Loss: {test_loss.result().numpy():.4f}, '\n        f'Test Accuracy: {(test_accuracy.result().numpy() * 100):.2f}'\n    )","metadata":{"execution":{"iopub.status.busy":"2022-05-28T02:42:31.858367Z","iopub.execute_input":"2022-05-28T02:42:31.859721Z","iopub.status.idle":"2022-05-28T02:45:07.242484Z","shell.execute_reply.started":"2022-05-28T02:42:31.859678Z","shell.execute_reply":"2022-05-28T02:45:07.241422Z"},"trusted":true},"execution_count":null,"outputs":[]}]}