{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":77476,"databundleVersionId":8462666,"sourceType":"competition"}],"dockerImageVersionId":30698,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## Intro\n[NOTION](https://kj49.notion.site/24-5-9-1-a128d9c5c4f44de8b056c573d81e993a?pvs=73)  \n  \nZalando의 Fashion MNIST는 Zalando의 의류 이미지 데이터셋으로,  \n60,000개의 훈련용 예제와 10,000개의 테스트용 예제로 구성되어 있습니다.  \n각 예제는 28x28 크기의 흑백 이미지로, 10개의 클래스 중 하나와 연관되어 있습니다.  \nZalando는 Fashion-MNIST를 기계 학습 알고리즘 벤치마킹을 위한  \n기존 MNIST 데이터셋의 직접적인 대체재로 사용할 목적으로 제공합니다.  \n이 데이터셋은 이미지 크기와 훈련 및 테스트 분할 구조가 기존 MNIST 데이터셋과 동일합니다.  \n  \n원래의 MNIST 데이터셋은 많은 수의 손으로 쓴 숫자들을 포함하고 있으며,  \nAI/ML/데이터 과학 커뮤니티의 많은 구성원들이 이 데이터셋을 좋아하고  \n자신들의 알고리즘을 검증하기 위한 벤치마크로 사용합니다.  \n사실, MNIST는 연구자들이 처음 시도하는 데이터셋 중 하나입니다.  \n\"MNIST에서 작동하지 않는다면, 어떤 것도 작동하지 않을 것이라고\" 그들은 말합니다.  \n\"그렇다고 해서 MNIST에서 작동한다고 해서 다른 것들에서도 성공할 것이라는 보장은 없습니다.\"  \n  \nZalando는 원래의 MNIST 데이터셋을 대체하고자 합니다.  \n  \n  \n[REFERENCE 0](https://www.kaggle.com/code/heuisubhyeon/heesub-fashion-mnist)  \n\n[REFERENCE 1](https://brunch.co.kr/@hansungdev/30)  \n[REFERENCE 2](https://dacon.io/codeshare/4562)  \n[REFERENCE 3](https://hul980.tistory.com/108)  \n  \n[REFERENCE 4](https://federicoarenasl.github.io/PCA-on-Fashion-MNIST/)  \n[REFERENCE 5](https://solearn.tistory.com/152)  \n[REFERENCE 6](https://itstory1592.tistory.com/20)  ","metadata":{}},{"cell_type":"markdown","source":"#### The task\n#### Classify the fashion items!\n#### There are 10 categories.\n\n0. T-shirt/top\n\n1. Trouser\n\n2. Pullover\n\n3. Dress\n\n4. Coat\n\n5. Sandal\n\n6. Shirt\n\n7. Sneaker\n\n8. Bag\n\n9. Ankle boot\n\n#### Convolutional neural networks will come in handy here.","metadata":{}},{"cell_type":"markdown","source":"## EDA","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport seaborn as sns\n\nimport tensorflow as tf\n\nimport matplotlib.pyplot as plt\n%matplotlib inline","metadata":{"execution":{"iopub.status.busy":"2024-05-14T19:50:12.572422Z","iopub.execute_input":"2024-05-14T19:50:12.572868Z","iopub.status.idle":"2024-05-14T19:50:12.581321Z","shell.execute_reply.started":"2024-05-14T19:50:12.572829Z","shell.execute_reply":"2024-05-14T19:50:12.579906Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df = pd.read_csv('/kaggle/input/2024-smarcle-ks-3-fashion-mnist2/train.csv')\ntest_df = pd.read_csv('/kaggle/input/2024-smarcle-ks-3-fashion-mnist2/test.csv')","metadata":{"execution":{"iopub.status.busy":"2024-05-14T19:50:12.583774Z","iopub.execute_input":"2024-05-14T19:50:12.584200Z","iopub.status.idle":"2024-05-14T19:50:18.483464Z","shell.execute_reply.started":"2024-05-14T19:50:12.584166Z","shell.execute_reply":"2024-05-14T19:50:18.482318Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_df.head()","metadata":{"execution":{"iopub.status.busy":"2024-05-14T19:50:18.484739Z","iopub.execute_input":"2024-05-14T19:50:18.485082Z","iopub.status.idle":"2024-05-14T19:50:18.505304Z","shell.execute_reply.started":"2024-05-14T19:50:18.485046Z","shell.execute_reply":"2024-05-14T19:50:18.504125Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.head()","metadata":{"execution":{"iopub.status.busy":"2024-05-14T19:50:18.508272Z","iopub.execute_input":"2024-05-14T19:50:18.508823Z","iopub.status.idle":"2024-05-14T19:50:18.529178Z","shell.execute_reply.started":"2024-05-14T19:50:18.508787Z","shell.execute_reply":"2024-05-14T19:50:18.527800Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_df.shape","metadata":{"execution":{"iopub.status.busy":"2024-05-14T19:50:18.530767Z","iopub.execute_input":"2024-05-14T19:50:18.531098Z","iopub.status.idle":"2024-05-14T19:50:18.538783Z","shell.execute_reply.started":"2024-05-14T19:50:18.531069Z","shell.execute_reply":"2024-05-14T19:50:18.537525Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.shape","metadata":{"execution":{"iopub.status.busy":"2024-05-14T19:50:18.540331Z","iopub.execute_input":"2024-05-14T19:50:18.540674Z","iopub.status.idle":"2024-05-14T19:50:18.549853Z","shell.execute_reply.started":"2024-05-14T19:50:18.540647Z","shell.execute_reply":"2024-05-14T19:50:18.548570Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# delete label\nx_train = train_df.loc[:,'pixel1':].values.reshape(-1, 28, 28)\nx_train.shape","metadata":{"execution":{"iopub.status.busy":"2024-05-14T19:50:18.551232Z","iopub.execute_input":"2024-05-14T19:50:18.551571Z","iopub.status.idle":"2024-05-14T19:50:18.559681Z","shell.execute_reply.started":"2024-05-14T19:50:18.551544Z","shell.execute_reply":"2024-05-14T19:50:18.558428Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"x_test = test_df.loc[:, 'pixel1':].values.reshape(-1, 28, 28)\nx_test.shape","metadata":{"execution":{"iopub.status.busy":"2024-05-14T19:50:18.561103Z","iopub.execute_input":"2024-05-14T19:50:18.561457Z","iopub.status.idle":"2024-05-14T19:50:18.568304Z","shell.execute_reply.started":"2024-05-14T19:50:18.561405Z","shell.execute_reply":"2024-05-14T19:50:18.567217Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.imshow(x_train[3])","metadata":{"execution":{"iopub.status.busy":"2024-05-14T19:50:18.572428Z","iopub.execute_input":"2024-05-14T19:50:18.572775Z","iopub.status.idle":"2024-05-14T19:50:18.828600Z","shell.execute_reply.started":"2024-05-14T19:50:18.572746Z","shell.execute_reply":"2024-05-14T19:50:18.827579Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.imshow(x_test[3])","metadata":{"execution":{"iopub.status.busy":"2024-05-14T19:50:18.829774Z","iopub.execute_input":"2024-05-14T19:50:18.830094Z","iopub.status.idle":"2024-05-14T19:50:19.104643Z","shell.execute_reply.started":"2024-05-14T19:50:18.830058Z","shell.execute_reply":"2024-05-14T19:50:19.103564Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"y_train = train_df.loc[:, 'label']\ny_train.unique()","metadata":{"execution":{"iopub.status.busy":"2024-05-14T19:50:19.106270Z","iopub.execute_input":"2024-05-14T19:50:19.107011Z","iopub.status.idle":"2024-05-14T19:50:19.114949Z","shell.execute_reply.started":"2024-05-14T19:50:19.106968Z","shell.execute_reply":"2024-05-14T19:50:19.113729Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"target = {0:'T-shirt/top', \n            1:'Trouser',\n            2:'Pullover', \n            3:'Dress', \n            4:'Coat', \n            5:'Sandal', \n            6:'Shirt', \n            7:'Sneaker', \n            8:'Bag',\n            9:'ANkle boot'}\n\nprint(y_train[3])\nprint(target[y_train[3]])","metadata":{"execution":{"iopub.status.busy":"2024-05-14T19:50:19.116390Z","iopub.execute_input":"2024-05-14T19:50:19.116733Z","iopub.status.idle":"2024-05-14T19:50:19.123694Z","shell.execute_reply.started":"2024-05-14T19:50:19.116705Z","shell.execute_reply":"2024-05-14T19:50:19.122634Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# 결측값이 포함된 열의 인덱스를 찾는 코드\n# x_train은 이미 numpy 형태이므로 이렇게 찾았다. 지피티야 고마워~\nmissing_value_columns_indices = np.where(np.isnan(x_train).any(axis=0))[0]\n\n# 결측값이 있는 열의 인덱스 출력\nprint(\"NULL in columns indices:\", missing_value_columns_indices)","metadata":{"execution":{"iopub.status.busy":"2024-05-14T19:50:19.125346Z","iopub.execute_input":"2024-05-14T19:50:19.125680Z","iopub.status.idle":"2024-05-14T19:50:19.145703Z","shell.execute_reply.started":"2024-05-14T19:50:19.125653Z","shell.execute_reply":"2024-05-14T19:50:19.144674Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(x_train[3])","metadata":{"execution":{"iopub.status.busy":"2024-05-14T19:50:19.146928Z","iopub.execute_input":"2024-05-14T19:50:19.147280Z","iopub.status.idle":"2024-05-14T19:50:19.156564Z","shell.execute_reply.started":"2024-05-14T19:50:19.147251Z","shell.execute_reply":"2024-05-14T19:50:19.155521Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig = plt.figure(figsize=(6, 6))\nfor i in range(30): # 30개의 이미지를 위한 반복문\n    # 6x5 그리드에 서브플롯 추가\n    ax = fig.add_subplot(6, 5, i + 1, xticks=[], yticks=[])\n    ax.imshow(x_train[i], cmap=plt.cm.binary, interpolation='nearest')\n    # 'target' 딕셔너리를 이용해 이미지에 라벨을 붙임\n    label = target[y_train[i]]\n    ax.text(0, 7, label, color=\"red\", fontsize=8)","metadata":{"execution":{"iopub.status.busy":"2024-05-14T19:50:19.158052Z","iopub.execute_input":"2024-05-14T19:50:19.158360Z","iopub.status.idle":"2024-05-14T19:50:20.682304Z","shell.execute_reply.started":"2024-05-14T19:50:19.158333Z","shell.execute_reply":"2024-05-14T19:50:20.681023Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Feature Scaling\nx_train = x_train.astype('float64')\nx_test = x_test.astype('float64') \nx_train /= 255\nx_test /= 255\n\nprint(x_train[3])","metadata":{"execution":{"iopub.status.busy":"2024-05-14T19:50:20.684093Z","iopub.execute_input":"2024-05-14T19:50:20.684770Z","iopub.status.idle":"2024-05-14T19:50:21.059793Z","shell.execute_reply.started":"2024-05-14T19:50:20.684736Z","shell.execute_reply":"2024-05-14T19:50:21.058702Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## PCA\nPCA(Principal Component Analysis)는  \n차원 감소, 특징 추출, 데이터 시각화, 노이즈 감소와 같은 다양한 응용 분야에 널리 사용됩니다.  \n가장 유익한 기능을 식별하고 중복 정보를 제거하며  \n중요한 정보의 손실을 최소화하면서 데이터를 압축하는 데 도움이 됩니다.  ","metadata":{}},{"cell_type":"code","source":"from sklearn.decomposition import PCA\n\n# x_train이 (샘플 수, 높이, 너비) 형태를 가질 때,\n# (샘플 수, 높이 * 너비)로 변환\nx_train_for_pca = x_train.reshape(x_train.shape[0], -1)\nprint(f'Before PCA: {x_train_for_pca.shape}')\n\n# PCA 모델 초기화 및 학습과 동시에 데이터 변환\nn_components=0.95\npca_model = PCA(n_components)\nx_train_pca = pca_model.fit_transform(x_train_for_pca)\nprint(f'After PCA: {x_train_pca.shape}')","metadata":{"execution":{"iopub.status.busy":"2024-05-14T19:50:21.061121Z","iopub.execute_input":"2024-05-14T19:50:21.061454Z","iopub.status.idle":"2024-05-14T19:50:21.491965Z","shell.execute_reply.started":"2024-05-14T19:50:21.061409Z","shell.execute_reply":"2024-05-14T19:50:21.489388Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# PCA 그래프 그리기 by GPT\n\n# 첫 번째와 두 번째 주성분에 대한 산점도 그리기\nplt.figure(figsize=(8, 6))\nscatter = plt.scatter(x_train_pca[:, 0], x_train_pca[:, 1], c=y_train, alpha=0.5, cmap='inferno')\n# 색상 범례 추가\nplt.legend(handles=scatter.legend_elements()[0], labels=[target[i] for i in np.unique(y_train)], title=\"Classes\")\n\nplt.xlabel('1st Principal Component')\nplt.ylabel('2nd Principal Component')\nplt.title('PCA of x_train')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-05-14T19:50:21.492973Z","iopub.status.idle":"2024-05-14T19:50:21.493834Z","shell.execute_reply.started":"2024-05-14T19:50:21.493541Z","shell.execute_reply":"2024-05-14T19:50:21.493567Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"첫 번째 주성분은 데이터의 분산을 가장 많이 설명하는 방향을 나타내고,  \n두 번째 주성분은 첫 번째 주성분과 직교(orthogonal)하는 벡터 중에서  \n분산을 가장 많이 설명하는 방향입니다.  \n이렇게 각 주성분들은 서로 직교하는 성질을 가지며,  \n데이터의 분산을 설명하는 순서대로 나열됩니다.  ","metadata":{}},{"cell_type":"code","source":"# 원-핫 인코딩\nfrom keras.utils import to_categorical\nprint(\"Before: \", y_train[0])\ny_train2 = to_categorical(y_train)\nprint(\"After: \", y_train2[0])","metadata":{"execution":{"iopub.status.busy":"2024-05-14T19:50:21.495263Z","iopub.status.idle":"2024-05-14T19:50:21.495670Z","shell.execute_reply.started":"2024-05-14T19:50:21.495476Z","shell.execute_reply":"2024-05-14T19:50:21.495494Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Modelling\nPCA와 CNN을 별개로 진행해보겠습니다.","metadata":{}},{"cell_type":"markdown","source":"## Model - Tensorflow","metadata":{}},{"cell_type":"code","source":"# explained_variance_ratio_ 출력\nprint(np.sum(pca_model.explained_variance_ratio_))\n\n# explained_variance_ratio_의 누적 합을 그래프로 표시\nplt.plot(np.cumsum(pca_model.explained_variance_ratio_))\nplt.xlabel('Number of Components')\nplt.ylabel('Cumulative Explained Variance')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-05-14T19:50:21.497513Z","iopub.status.idle":"2024-05-14T19:50:21.497924Z","shell.execute_reply.started":"2024-05-14T19:50:21.497729Z","shell.execute_reply":"2024-05-14T19:50:21.497746Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from tensorflow.keras.models import Sequential\nfrom tensorflow.keras.layers import Dense, Dropout\nfrom tensorflow.keras.optimizers import Adam\n\n# 모델 구성\nmodel = Sequential([\n    Dense(512, activation='relu', input_shape=(x_train_pca.shape[1],)),  # 입력층\n    Dropout(0.2),  # 과적합 방지를 위한 Dropout\n    Dense(128, activation='relu'),  # 은닉층\n    Dropout(0.2),\n    Dense(10, activation='softmax')  # 출력층, 클래스가 10개이므로 softmax 사용\n])","metadata":{"execution":{"iopub.status.busy":"2024-05-14T19:50:21.499109Z","iopub.status.idle":"2024-05-14T19:50:21.499510Z","shell.execute_reply.started":"2024-05-14T19:50:21.499310Z","shell.execute_reply":"2024-05-14T19:50:21.499327Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# 모델 컴파일\nmodel.compile(optimizer=Adam(), loss='categorical_crossentropy', metrics=['accuracy'])","metadata":{"execution":{"iopub.status.busy":"2024-05-14T19:50:21.501861Z","iopub.status.idle":"2024-05-14T19:50:21.502322Z","shell.execute_reply.started":"2024-05-14T19:50:21.502121Z","shell.execute_reply":"2024-05-14T19:50:21.502139Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# 모델 요약 출력\nmodel.summary()","metadata":{"execution":{"iopub.status.busy":"2024-05-14T19:50:21.503867Z","iopub.status.idle":"2024-05-14T19:50:21.504344Z","shell.execute_reply.started":"2024-05-14T19:50:21.504147Z","shell.execute_reply":"2024-05-14T19:50:21.504165Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# 모델 훈련\nhistory = model.fit(x_train_pca, y_train2, epochs=50, batch_size=128, validation_split=0.2, verbose=1)","metadata":{"execution":{"iopub.status.busy":"2024-05-14T19:50:21.506052Z","iopub.status.idle":"2024-05-14T19:50:21.506482Z","shell.execute_reply.started":"2024-05-14T19:50:21.506290Z","shell.execute_reply":"2024-05-14T19:50:21.506307Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# 훈련 과정 그래프로 표시\nplt.figure(figsize=(12, 4))\n\nplt.subplot(1, 2, 1)\nplt.plot(history.history['loss'], label='Training Loss')\nplt.plot(history.history['val_loss'], label='Validation Loss')\nplt.legend()\nplt.title('Loss')\n\nplt.subplot(1, 2, 2)\nplt.plot(history.history['accuracy'], label='Training Accuracy')\nplt.plot(history.history['val_accuracy'], label='Validation Accuracy')\nplt.legend()\nplt.title('Accuracy')\n\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-05-14T19:50:21.508231Z","iopub.status.idle":"2024-05-14T19:50:21.508675Z","shell.execute_reply.started":"2024-05-14T19:50:21.508477Z","shell.execute_reply":"2024-05-14T19:50:21.508496Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# 테스트 데이터 준비\n# 테스트 데이터도 동일한 전처리\nx_test = test_df.loc[:, 'pixel1':].values.reshape(-1, 28, 28)  # Reshape\nx_test = x_test.astype('float64') / 255  # Feature Scaling\nx_test = x_test.reshape(x_test.shape[0], -1)  # Flatten\nx_test_pca = pca_model.transform(x_test)  # PCA 적용","metadata":{"execution":{"iopub.status.busy":"2024-05-14T19:50:21.510026Z","iopub.status.idle":"2024-05-14T19:50:21.510397Z","shell.execute_reply.started":"2024-05-14T19:50:21.510217Z","shell.execute_reply":"2024-05-14T19:50:21.510233Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# 테스트 데이터에 대한 예측 수행\npredictions = model.predict(x_test_pca)\n\n# 예측 결과 확인\npredicted_classes = np.argmax(predictions, axis=1)\nprint(predicted_classes[:10])","metadata":{"execution":{"iopub.status.busy":"2024-05-14T19:50:21.511725Z","iopub.status.idle":"2024-05-14T19:50:21.512111Z","shell.execute_reply.started":"2024-05-14T19:50:21.511904Z","shell.execute_reply":"2024-05-14T19:50:21.511919Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# 제출 파일 로드\nsubmission = pd.read_csv('/kaggle/input/2024-smarcle-ks-3-fashion-mnist2/sample_submission.csv')\n\n# 예측한 클래스를 'label' 열에 저장\nsubmission['label'] = predicted_classes\n\n# 제출 파일 저장\nsubmission.to_csv('submission_tensorflow.csv', index=False)\n\n# 제출 파일 확인\nprint(submission.head())","metadata":{"execution":{"iopub.status.busy":"2024-05-14T19:50:21.513224Z","iopub.status.idle":"2024-05-14T19:50:21.513589Z","shell.execute_reply.started":"2024-05-14T19:50:21.513401Z","shell.execute_reply":"2024-05-14T19:50:21.513416Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Model - CNN\n앞서 PCA 후 tensorflow를 통해 점수(0.89720)를 확인해보았으니,  \n이제는 CNN으로도 한 번 돌려보겠다.","metadata":{}},{"cell_type":"code","source":"# 원본 테스트 데이터 준비 (CNN 모델용)\n# x_test를 PCA 적용 전의 원본 데이터로 다시 로드하고, 4차원으로 재변환\nx_test_original = test_df.loc[:, 'pixel1':].values.reshape(-1, 28, 28)  # 원본 데이터 로드\nx_test_original = x_test_original.astype('float64') / 255  # Feature Scaling\nx_test2 = x_test_original.reshape(-1, 28, 28, 1)  # 4차원으로 변경: (샘플 수, 높이, 너비, 채널 수)\n\nprint(x_test2.shape)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# CNN 사용을 위한 4차원으로 변경\nx_train2 = x_train.reshape(-1, 28, 28, 1) # 4차원으로 변경: (sample 수, 높이, 너비, 채널 수)\nx_test2 = x_test.reshape(-1, 28, 28, 1)\n\nprint(x_train2.shape)\nprint(x_test2.shape)","metadata":{"execution":{"iopub.status.busy":"2024-05-14T19:50:21.514408Z","iopub.status.idle":"2024-05-14T19:50:21.514752Z","shell.execute_reply.started":"2024-05-14T19:50:21.514578Z","shell.execute_reply":"2024-05-14T19:50:21.514593Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from tensorflow.keras.models import Sequential\nfrom tensorflow.keras.layers import Conv2D, MaxPooling2D, Flatten, Dense, Dropout\n\n# 모델 정의\nmodel = Sequential([\n    # 첫 번째 합성곱 레이어: 64개의 필터, 3x3 커널, relu 활성화 함수 사용\n    # 입력 형태는 (28, 28, 1)이며, 이는 이미지가 28x28 크기에 1개의 채널을 가지고 있음을 의미\n    Conv2D(64, (3, 3), activation='relu', input_shape=(28, 28, 1)),\n    MaxPooling2D(2, 2),\n    # 두 번째 합성곱 레이어: 64개의 필터, 3x3 커널, relu 활성화 함수 사용\n    Conv2D(64, (3, 3), activation='relu'),\n    MaxPooling2D(2,2),\n    # Flatten 레이어: 2D 합성곱 레이어에서 나온 3D 출력을 1D 벡터로 변환\n    Flatten(),\n    # 첫 번째 완전 연결 레이어: 128개의 뉴런, relu 활성화 함수 사용\n    Dense(128, activation='relu'),\n    # 드롭아웃 레이어: 과적합을 방지하기 위해 사용, 여기서는 50%의 뉴런을 무작위로 꺼준다.\n    Dropout(0.5),\n    # 출력 레이어: 10개의 뉴런, 각 뉴런은 하나의 클래스를 나타냄, softmax 활성화 함수 사용\n    Dense(10, activation='softmax')\n])","metadata":{"execution":{"iopub.status.busy":"2024-05-14T19:50:21.516118Z","iopub.status.idle":"2024-05-14T19:50:21.516472Z","shell.execute_reply.started":"2024-05-14T19:50:21.516295Z","shell.execute_reply":"2024-05-14T19:50:21.516311Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# 모델 컴파일\n# 옵티마이저로는 'adam'을 사용하고, 손실 함수로는 'sparse_categorical_crossentropy'를 사용\n# 이는 각 이미지에 대해 하나의 레이블이 주어지는 경우에 적합한 손실 함수이다.\n# 평가 지표로는 'accuracy'를 사용\nmodel.compile(optimizer='adam', \n              loss='sparse_categorical_crossentropy', \n              metrics=['accuracy'])","metadata":{"execution":{"iopub.status.busy":"2024-05-14T19:50:21.517970Z","iopub.status.idle":"2024-05-14T19:50:21.518322Z","shell.execute_reply.started":"2024-05-14T19:50:21.518146Z","shell.execute_reply":"2024-05-14T19:50:21.518161Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# 모델 요약 출력\nmodel.summary()","metadata":{"execution":{"iopub.status.busy":"2024-05-14T19:50:21.519141Z","iopub.status.idle":"2024-05-14T19:50:21.519473Z","shell.execute_reply.started":"2024-05-14T19:50:21.519301Z","shell.execute_reply":"2024-05-14T19:50:21.519315Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# x_test2에 대한 예측 수행\npredictions_cnn = model.predict(x_test2)\n\n# 예측값 중 가장 높은 확률을 가진 클래스 선택\npredicted_classes_cnn = np.argmax(predictions, axis=1)\n\n# 제출\nsubmission.to_csv(\"submission_cnn.csv\", index=False)\nprint(submission.head())","metadata":{"execution":{"iopub.status.busy":"2024-05-14T19:50:21.520302Z","iopub.status.idle":"2024-05-14T19:50:21.520648Z","shell.execute_reply.started":"2024-05-14T19:50:21.520472Z","shell.execute_reply":"2024-05-14T19:50:21.520488Z"},"trusted":true},"execution_count":null,"outputs":[]}]}