{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":3004,"databundleVersionId":861823,"sourceType":"competition"}],"dockerImageVersionId":30587,"isInternetEnabled":false,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"Phần 1: Nhập các thư viện cần thiết\n------","metadata":{"_uuid":"b0093541-188e-43e9-acb2-d917ae3a5010","_cell_guid":"cadbfcb2-82c3-470f-bad6-eade54542125","trusted":true}},{"cell_type":"markdown","source":"
\nCơ sở dữ liệu MNIST (Modified National Institute of Standards and Technology database) là một cơ sở dữ liệu lớn về chữ số viết tay thường được sử dụng để huấn luyện các hệ thống xử lý hình ảnh khác nhau. Khi các kỹ thuật học máy mới xuất hiện, MNIST vẫn là một nguồn tài nguyên đáng tin cậy cho các nhà nghiên cứu và người học.

\nCác hình ảnh đen trắng gốc đã được chuẩn hóa để vừa với một hộp giới hạn 28x28 pixel và được làm mềm, điều này đã tạo ra các mức độ xám. Cơ sở dữ liệu MNIST gốc chứa 60,000 hình ảnh huấn luyện và 10,000 hình ảnh kiểm tra, nhưng trong cuộc thi này, chúng ta có 42,000 hình ảnh huấn luyện (70%) và 18,000 hình ảnh kiểm tra (30%).

\nCác kỹ thuật tân tiến nhất để tiếp cận bất kỳ vấn đề Phân loại Hình ảnh nào thường dựa vào Mạng Nơron Tích chập. Kiến trúc này rất giỏi trong việc trích xuất các mẫu từ hình ảnh. Tuy nhiên, vì MNIST là một vấn đề đơn giản, trong notebook này, chúng ta sẽ áp dụng các thuật toán Học máy đơn giản (và chúng ta sẽ đạt được độ chính xác khá tốt!).

\nĐầu tiên, chúng ta sẽ thực hiện Phân tích Thành phần Chính để giảm chiều của dữ liệu (từ 784 giá trị pixel xuống còn 10 thành phần chính). Sau đó, chúng ta sẽ chạy thuật toán học lười yêu thích của chúng ta: K-Nearest Neighbors. Sự kết hợp này đạt được Độ chính xác Kiểm tra là 0.9271, điều này khá đáng chú ý !!!

\nMục lục:\n\n
","metadata":{"_uuid":"85d40f0d-34d6-4e02-90f3-7ef7a60e5573","_cell_guid":"ddf0a964-5157-492a-9699-253bc2caec79","trusted":true}},{"cell_type":"code","source":"## IMPORT THE NECESSARY PYTHON LIBRARIES\n\n# Read & Explore Data\nimport numpy as np\nimport pandas as pd\nimport seaborn as sns\nimport matplotlib.pyplot as plt\nfrom sklearn.preprocessing import MinMaxScaler\n\n# Principal Component Analysis (PCA)\nfrom sklearn.decomposition import PCA\nfrom sklearn.pipeline import Pipeline\nimport plotly.express as px\n\n# K-Nearest Neighbor Classifier (KNN)\nfrom sklearn.neighbors import KNeighborsClassifier\nfrom sklearn.model_selection import GridSearchCV\nfrom sklearn.metrics import classification_report, \\\n confusion_matrix, ConfusionMatrixDisplay \n\n# VHT\nfrom scipy.ndimage import rotate","metadata":{"_uuid":"c382c9dc-2109-4ce6-b779-dc8c447da240","_cell_guid":"e169ec97-1f64-46cd-8099-1d09c757dccc","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-06-13T16:18:42.435127Z","iopub.execute_input":"2024-06-13T16:18:42.435541Z","iopub.status.idle":"2024-06-13T16:18:45.938350Z","shell.execute_reply.started":"2024-06-13T16:18:42.435504Z","shell.execute_reply":"2024-06-13T16:18:45.937109Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def func_array1d_rotate2d(Narray1d,rotate_angle,sizePixel):\n\n N = Narray1d.shape[0]\n\n Narray1d_rotate = np.zeros(Narray1d.shape)\n \n for i in range(N):\n image = np.reshape(Narray1d[i,:],(sizePixel,sizePixel))\n rotated_image = rotate(image,rotate_angle, reshape=False)\n rotated_array1d = np.reshape(rotated_image,(1,-1))\n Narray1d_rotate[i,:] = rotated_array1d\n \n return Narray1d_rotate","metadata":{"_uuid":"b362968d-9828-4257-92ec-b9e4e49a9cb0","_cell_guid":"8ace7936-3ee9-4fe9-b71e-8d7e3b7eee15","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-06-13T16:18:45.940452Z","iopub.execute_input":"2024-06-13T16:18:45.940999Z","iopub.status.idle":"2024-06-13T16:18:45.950453Z","shell.execute_reply.started":"2024-06-13T16:18:45.940958Z","shell.execute_reply":"2024-06-13T16:18:45.948529Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Phần 2: Đọc dữ liệu và khai thác dữ liệu","metadata":{"_uuid":"a7abf7cf-fbfb-44f8-a5eb-77885d1283fb","_cell_guid":"1e5c33e8-406f-4da5-929d-034f87443736","trusted":true}},{"cell_type":"markdown","source":"
\nTrong phần này, chúng ta sẽ sử dụng thư viện Python nổi tiếng Pandas để đọc dữ liệu từ các tệp .csv.

\nChúng ta sẽ trực quan hóa phân phối của các giá trị pixel (biểu đồ histogram) sử dụng các thư viện Python Seaborn và Matplotlib. Chúng ta cũng sẽ kiểm tra rằng không có giá trị thiếu, và các lớp mục tiêu được cân bằng.

\nCuối cùng, chúng ta sẽ trực quan hóa các hình ảnh bằng cách vẽ một hình ảnh từ mỗi lớp.\n
\n","metadata":{"_uuid":"d4c82fac-2a10-4c14-a95a-653bd16872cd","_cell_guid":"deaebcb3-115c-4e6b-b843-4522382a052d","trusted":true}},{"cell_type":"code","source":"# Read the data files using Pandas\ndf_train = pd.read_csv('/kaggle/input/digit-recognizer/train.csv')\ndf_test = pd.read_csv('/kaggle/input/digit-recognizer/test.csv')","metadata":{"_uuid":"4da57b46-cc50-4ae9-9495-d0dbeb6be5ee","_cell_guid":"8967621a-0c55-49ff-b6e2-05d9efdeedf2","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-06-13T16:18:45.952140Z","iopub.execute_input":"2024-06-13T16:18:45.952538Z","iopub.status.idle":"2024-06-13T16:18:53.239909Z","shell.execute_reply.started":"2024-06-13T16:18:45.952500Z","shell.execute_reply":"2024-06-13T16:18:53.238702Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Check the size of the train and test DataFrames\nprint(\"df_train shape:\", df_train.shape) # 70%\nprint(\"df_test shape: \", df_test.shape) # 30%","metadata":{"_uuid":"5465a268-9ec4-4482-a0d7-e531d25f4917","_cell_guid":"da0e04b7-5ce3-48ec-be64-cee70c71af07","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-06-13T16:18:53.243249Z","iopub.execute_input":"2024-06-13T16:18:53.243744Z","iopub.status.idle":"2024-06-13T16:18:53.250833Z","shell.execute_reply.started":"2024-06-13T16:18:53.243701Z","shell.execute_reply":"2024-06-13T16:18:53.249561Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Fortunately there are no missing values in our data\nprint(\n \"Missing values in df_train: \", \\\n df_train.isna().sum().sum()\n)\nprint(\n \"Missing values in df_test: \", \\\n df_test.isna().sum().sum()\n)","metadata":{"_uuid":"f8324800-d007-4d21-9eb6-39bba50ce548","_cell_guid":"758f94a2-69c7-4c9b-b1a3-22cb0f0b1a43","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-06-13T16:18:53.252442Z","iopub.execute_input":"2024-06-13T16:18:53.253183Z","iopub.status.idle":"2024-06-13T16:18:53.311888Z","shell.execute_reply.started":"2024-06-13T16:18:53.253138Z","shell.execute_reply":"2024-06-13T16:18:53.310724Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Let's have a look at the first five training rows\ndf_train.head()","metadata":{"_uuid":"b1a7f2dc-6cec-4610-ba22-6ad5a7fb06ac","_cell_guid":"f1a8c252-31ba-46a1-b25c-c6ffff3fa2a9","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-06-13T16:18:53.313382Z","iopub.execute_input":"2024-06-13T16:18:53.313780Z","iopub.status.idle":"2024-06-13T16:18:53.346267Z","shell.execute_reply.started":"2024-06-13T16:18:53.313747Z","shell.execute_reply":"2024-06-13T16:18:53.345011Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# We can see that the first colum is the label,\n# and the rest of the columns are the 784 pixels\n# that represent 28x28 gray-scale images.\n# Let's separate X and Y\n\nX_train = df_train.drop(columns=['label']).values\ny_train = df_train['label'].values","metadata":{"_uuid":"48085649-c225-4e39-824f-82474ea004ff","_cell_guid":"939c1008-c85e-4ee0-bdab-bcccc4a252e4","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-06-13T16:18:53.348122Z","iopub.execute_input":"2024-06-13T16:18:53.348612Z","iopub.status.idle":"2024-06-13T16:18:53.479529Z","shell.execute_reply.started":"2024-06-13T16:18:53.348568Z","shell.execute_reply":"2024-06-13T16:18:53.478277Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Let's visualize the distribution of the pixel values\n\nax = sns.histplot(x=X_train.flatten(), color='#4895EE')\n_ = ax.set(title='Pixel Value Histogram', xlabel='Pixel Value')\n\n# As we can see, the 0 value represents the background\n# and the >0 values represent the digits in the images","metadata":{"_uuid":"f724afe4-d098-4f65-aaca-e97a096b8a32","_cell_guid":"36ec9986-72eb-4f04-abf8-84e2df3e3bfa","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-06-13T16:18:53.481030Z","iopub.execute_input":"2024-06-13T16:18:53.481418Z","iopub.status.idle":"2024-06-13T16:19:11.745411Z","shell.execute_reply.started":"2024-06-13T16:18:53.481386Z","shell.execute_reply":"2024-06-13T16:19:11.744288Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Let's check that the target is balanced.\n# It means that the ten categories in the target\n# have roughly the same representation in the data.\n\nax = sns.countplot(x=y_train, color='#4895EE')\nax.set(\n title='Target distribution is balanced? Yes!',\n xlabel='Label'\n)\n# The expected number of samples for a balanced \n# class distribution is 42000/10 = 4200\n_ = ax.axhline(y=4200, color='#12086F', linestyle='--')\n\n# Since the target classes are balances, we don't have\n# worry about sampling methods and biases in our models.","metadata":{"_uuid":"cd488e69-2d9d-4209-ac52-f11e13ab443b","_cell_guid":"d681064d-2dbe-4c2b-8075-9ce2c254d755","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-06-13T16:19:11.746733Z","iopub.execute_input":"2024-06-13T16:19:11.747056Z","iopub.status.idle":"2024-06-13T16:19:12.090357Z","shell.execute_reply.started":"2024-06-13T16:19:11.747028Z","shell.execute_reply":"2024-06-13T16:19:12.089125Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Next, let's visualize one image from each class.\n# We will use the matplotlib 'imshow' function.\n\nfig, axs = plt.subplots(nrows=2, ncols=5, figsize=(10,5))\n\nfor i in range(10):\n idx = np.where(y_train==i)[0][2]\n img = axs[int(i >= 5), i%5].imshow(\n X=X_train[idx].reshape(28, 28),\n cmap='Blues'\n )\n img.axes.get_yaxis().set_visible(False)\n img.axes.get_xaxis().set_visible(False)\n img.axes.set_title(f'Label: {i}')","metadata":{"_uuid":"9d6c909c-609f-4b5d-a907-2d75a3c17401","_cell_guid":"1fa8f38f-f989-4007-86c5-590612877157","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-06-13T16:19:12.097439Z","iopub.execute_input":"2024-06-13T16:19:12.097850Z","iopub.status.idle":"2024-06-13T16:19:12.866660Z","shell.execute_reply.started":"2024-06-13T16:19:12.097817Z","shell.execute_reply":"2024-06-13T16:19:12.865549Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\ntransform data","metadata":{"_uuid":"dd9d9d94-7ea8-4f8d-aa91-866959d703e3","_cell_guid":"48cf8b51-1a3e-4d0e-94e5-da53b3aa7c90","trusted":true}},{"cell_type":"code","source":"# -------------------------------------------------- declare\nnumRotate_prior = 10\n\nangleOffset_prior = -20\nangleRange_prior = 40\n\nX_train_pick2rotate = {\n 'index': np.array([300,301,302,303,304,\n 305,306,307,308,309,\n 310,311,312,313,314,\n 315,316,317,318,319,\n 320,321,322,323,324,\n 325,326,327,328,329,\n 330,331,332,333,334,\n 335,336,337,338,339,\n 340,341,342,343,344,\n 345,346,347,348,349,\n 350,351,352,353,354,\n 355,356,357,358,359,\n 360,361,362,363,364,\n 365,366,367,368,369,\n 370,371,372,373,374,\n 375,376,377,378,379,\n 380,381,382,383,384,\n 385,386,387,388,389,\n 390,391,392,393,394,\n 395,396,397,398,399,\n 2100,2101,2102,2103,2104,\n 2105,2106,2107,2108,2109,\n 2110,2111,2112,2113,2114,\n 2115,2116,2117,2118,2119,\n 2020,2021,2022,2023,2024,\n 2025,2026,2027,2028,2029,\n 2030,2031,2032,2033,2034,\n 2035,2036,2037,2038,2039,\n 2040,2041,2042,2043,2044,\n 2045,2046,2047,2048,2049,\n 2050,2051,2052,2053,2054,\n 2055,2056,2057,2058,2059,\n 2060,2061,2062,2063,2064,\n 2065,2066,2067,2068,2069,\n 2070,2071,2072,2073,2074,\n 2075,2076,2077,2078,2079,\n 2080,2081,2082,2083,2084,\n 2085,2086,2087,2088,2089,\n 2090,2091,2092,2093,2094,\n 2095,2096,2097,2098,2099,\n 2500,2501,2502,2503,2504, \n 2505,2506,2507,2508,2509, \n 2510,2511,2512,2513,2514,\n 2515,2516,2517,2518,2519,\n 2520,2521,2522,2523,2524,\n 2525,2526,2527,2528,2529,\n 2530,2531,2532,2533,2534,\n 2535,2536,2537,2538,2539,\n 2540,2541,2542,2543,2544,\n 2545,2546,2547,2548,2549,\n 2550,2551,2552,2553,2554,\n 2555,2556,2557,2558,2559,\n 2560,2561,2562,2563,2564,\n 2565,2566,2567,2568,2569,\n 2570,2571,2572,2573,2574,\n 2575,2576,2577,2578,2579,\n 2580,2581,2582,2583,2584,\n 2585,2586,2587,2588,2589,\n 2590,2591,2592,2593,2594,\n 2595,2596,2597,2598,2599,]),\n 'offset': np.array([\n [50,0,-30,0,0, \n 0,0,0,0,25],\n [40,15,-15,0,0, \n 10,0,0,-30,31],\n [0,0,0,-30,30, \n 30,-40,10,-10,7],\n [0,0,0,0,0, \n 30,-40,30,0,11],\n [30,30,0,30,0, \n 30,30,30,30,-15],\n \n [50,0,0,50,0,\n 0,0,0,0,-10],\n [0,0,-30,0,0,\n 0,0,0,20,20],\n [0,0,-10,0,0,\n 0,0,20,0,11],\n [0,-10,0,0,0,\n 0,-20,0,0,0],\n [10,40,-10,30,0,\n 30,-30,0,40,7],\n \n [0,10,0,0,10,\n 0,0,0,0,0],\n [0,0,0,0,0,\n 50,5,10,20,35],\n [30,0,0,0,0,\n 20,0,0,30,0],\n [0,10,0,-10,30,\n 20,0,0,20,0],\n [0,0,0,0,0,\n 0,-20,-5,0,40],\n \n [0,-10,-10,0,0,\n 0,0,0,-10,34],\n [0,0,5,30,30,\n 30,0,0,0,10],\n [0,0,0,0,0,\n 30,0,0,10,10],\n [40,-5,20,30,10,\n 0,-30,0,0,6],\n [0,20,5,10,0,\n 30,-5,0,5,0],\n \n [0,30,-15,0,20,\n 0,0,0,0,30],\n [0,30,0,0,20,\n 0,0,0,5,-5],\n [0,0,0,0,0,\n 0,0,0,0,-72],\n [0,35,0,30,20,\n 20,0,0,20,-6],\n [30,0,0,0,0,\n 0,0,0,30,0],\n \n [30,-2,20,16,0,\n 0,0,0,18,18],\n [0,0,-7,0,0,\n 10,15,20,20,30],\n [0,20,0,0,0,\n 10,30,30,0,0],\n [30,30,0,0,0,\n 0,-20,0,20,18],\n [0,4,0,-5,20,\n 0,0,0,0,5],\n \n [10,10,0,0,15,\n 10,0,0,0,0],\n [-40,0,0,0,20,\n 0,0,20,20,0],\n [0,-5,-5,0,0,\n 40,0,30,0,0],\n [0,0,0,-20,15,\n 10,-20,-10,0,0],\n [20,35,0,0,10,\n 10,0,0,10,10],\n \n [60,31,0,0,0,\n 0,0,0,0,0],\n [0,5,0,0,40,\n 30,-40,30,40,0],\n [30,31,0,-21,0,\n 0,0,0,0,-5],\n [10,-5,-5,10,15,\n 0,-10,0,20,0],\n [20,10,0,0,0,\n 0,0,0,0,0],\n \n [0,7,0,30,0,\n 50,0,50,50,30],\n [40,-5,0,10,0,\n 50,0,50,10,0],\n [40,40,0,40,-5,\n 50,0,0,10,50],\n [0,0,0,0,45,\n 0,0,10,40,0],\n [-50,-4,0,0,18,\n 0,3,30,10,9],\n \n [0,20,10,26,29,\n 40,0,0,15,0],\n [0,18,-5,0,40,\n 0,-25,0,15,0],\n [30,38,0,27,21,\n 0,-31,0,22,14],\n [5,-2,0,0,11,\n 0,17,6,12,3],\n [-17,2,-2,-2,0,\n 32,-48,40,-3,-2],\n \n [0,7,0,19,4,\n 11,0,12,11,12],\n [0,42,-13,0,0,\n 53,-8,10,0,5],\n [0,21,0,0,28,\n 17,0,0,6,14],\n [0,3,0,0,7,\n 6,0,0,2,3],\n [0,0,0,0,2,\n 0,-32,0,48,12],\n \n [21,20,0,-11,31,\n -8,-32,0,37,39],\n [5,3,31,0,0,\n 0,-3,0,8,7],\n [0,0,0,0,0,\n 7,0,18,-21,0],\n [0,0,-18,0,0,\n 0,0,11,0,8],\n [55,-8,-14,-9,17,\n 0,-17,0,0,12],\n \n [0,0,-7,-6,37,\n 0,10,8,11,11],\n [0,11,0,25,42,\n 13,-48,16,23,11],\n [0,8,-33,-12,8,\n 0,-47,5,27,30],\n [0,0,0,-3,18,\n 13,-8,0,11,5],\n [0,37,0,-18,21,\n -5,-22,0,0,8],\n \n [55,36,0,0,13,\n 51,-27,11,2,23],\n [22,12,18,16,13,\n 6,-3,11,4,-5],\n [55,0,0,-3,5,\n 0,0,11,0,2],\n [0,8,0,0,40,\n 13,31,0,11,34],\n [0,15,0,0,41,\n 41,7,41,8,7],\n \n [42,17,-4,5,-16,\n 43,0,5,-3,6],\n [0,0,0,0,0,\n 14,-5,0,11,10],\n [0,17,3,37,38,\n 21,-31,6,7,3],\n [0,47,0,-17,11,\n 8,7,6,-8,-27],\n [0,-2,0,0,5,\n 46,4,0,8,12],\n \n [0,5,-15,0,4,\n 0,0,4,-3,0],\n [32,-5,-5,-7,7,\n 0,11,14,0,0],\n [21,27,-14,5,5,\n 11,-22,0,4,5],\n [0,-5,-5,5,7,\n 21,15,28,5,11],\n [-78,22,0,11,13,\n 23,-23,14,0,11],\n \n [0,-11,-11,11,0,\n 15,-21,0,0,22],\n [0,0,0,5,5,\n 11,0,0,0,22],\n [11,22,0,0,0,\n 0,0,0,15,15],\n [0,21,0,11,0,\n 0,0,0,21,9],\n [0,0,0,-15,18,\n 11,-17,0,21,21],\n \n [11,11,0,11,-3,\n 21,-63,11,11,5],\n [5,5,0,0,21,\n 0,-27,0,8,9],\n [20,11,0,5,-11,\n 11,18,0,5,0],\n [18,35,0,11,15,\n 0,-11,0,5,11],\n [0,0,5,5,11,\n 0,0,5,-21,5],\n \n [11,11,0,0,15,\n 11,0,21,32,32],\n [11,15,0,0,0,\n 15,0,11,0,11],\n [21,21,0,11,25,\n 0,11,12,11,0],\n [0,11,-21,-32,11,\n 0,0,11,5,0],\n [21,11,-5,0,-3,\n 21,11,-5,0,-11],\n \n [33,16,-11,-21,15,\n 0,0,5,5,-5],\n [0,28,0,11,0,\n 0,0,5,0,0],\n [0,0,-5,0,5,\n 21,0,0,5,11],\n [25,4,-5,0,22,\n 31,25,5,0,0],\n [11,0,0,11,-5,\n 24,0,0,5,16],\n \n #---------------------------------------------------------\n \n [26,10,-5,-10,5, \n -15,-20,0,20,30],\n [0,-15,0,0,30, \n 30,-10,0,0,-20],\n [10,15,0,0,0, \n -10,0,0,5,0],\n [30,0,10,-5,5, \n 0,-15,0,-10,0],\n [0,25,-10,10,27, \n 10,15,0,20,-10],\n \n \n [0,29,0,0,5, \n 20,0,0,0,15],\n [15,5,-10,-15,15, \n 0,10,5,0,-10],\n [30,0,0,0,0, \n 0,0,0,13,10],\n [15,10,0,0,0, \n 0,30,0,0,0],\n [0,31,0,0,0, \n 0,0,10,22,0],\n \n\n [10,15,0,5,-10, \n 5,-35,0,15,-41],\n [50,36,7,8,8, \n 25,10,0,10,10],\n [30,-3,-10,8,8, \n 0,-24,-15,20,0],\n [-45,10,-20,0,0, \n 0,-15,0,-35,0],\n [30,-5,0,0,10, \n 10,-25,0,20,0],\n \n\n [-5,-22,-10,0,35, \n 10,25,0,20,-32],\n [0,-7,5,13,0, \n -5,-10,0,10,22],\n [-11,5,-7,-17,30, \n 0,-15,0,42,27],\n [11,21,-20,0,0, \n 0,-30,0,10,0],\n [44,15,-14,0,10, \n 10,-25,-30,0,0], \n \n\n [44,4,0,0,5, \n 5,20,6,6,15],\n [40,15,-3,5,22, \n 20,10,5,0,14],\n [30,-11,-7,0,0, \n 0,0,0,5,10],\n [15,-10,22,0,0, \n 0,30,7,0,0],\n [60,6,0,0,-10, \n 0,0,3,6,0],\n \n\n [37,7,0,0,5, \n 5,5,-14,6,-11],\n [77,24,-3,-16,-14, \n 13,-20,5,0,10],\n [30,-11,-7,-20,7, \n 7,10,0,5,10],\n [15,2,0,7,0, \n 0,-15,12,5,0],\n [-30,21,-6,3,24, \n 15,-20,3,6,-5],\n \n \n [44,45,0,10,5, \n 15,-3,1,4,5],\n [40,13,-3,3,19, \n 18,0,-15,15,19],\n [30,16,7,0,0, \n 20,20,0,10,0],\n [-15,0,-22,0,4, \n 10,3,7,18,-15],\n [30,-10,2,-20,0, \n 0,6,-4,3,23],\n \n\n [44,8,2,-39,30, \n 15,7,6,16,15],\n [4,20,-3,0,20, \n 18,3,10,14,17],\n [35,5,-9,1,-1, \n 6,2,0,28,6],\n [15,21,-10,10,-2, \n 7,-16,0,33,9],\n [25,22,-4,0,-8, \n 0,-6,-7,-2,13],\n \n\n [47,7,-4,20,25, \n 5,5,3,6,15],\n [-10,17,-6,3,14, \n 20,-30,0,3,0],\n [30,27,-12,-6,0, \n 0,-17,7,8,15],\n [0,23,-22,6,0, \n 0,10,0,-1,0],\n [60,39,0,0,20, \n 0,-10,3,6,19],\n \n\n [10,10,-10,-15,5, \n 5,0,0,4,15],\n [15,15,10,10,15, \n 0,10,5,0,-5],\n [30,3,0,0,0, \n 0,0,7,13,13],\n [15,7,0,0,8, \n 9,20,0,0,0],\n [15,-7,-10,0,0, \n 10,0,10,22,-9],\n \n\n [20,24,-10,0,5, \n 17,0,7,0,15],\n [20,25,-10,-7,15, \n 10,-10,0,0,10],\n [17,-4,3,0,20, \n 5,-6,0,13,10],\n [15,0,-8,0,10, \n 0,10,0,0,9],\n [30,13,-2,0,0, \n 0,0,5,14,5],\n \n\n [40,-5,0,0,5, \n 15,0,0,0,15],\n [-40,20,-10,-15,15, \n 8,10,-3,0,-2],\n [40,-3,-2,0,0, \n 0,0,0,13,3],\n [35,2,-2,0,3, \n -2,7,0,-5,0],\n [20,11,-4,7,5, \n 5,-6,10,14,4],\n \n\n [10,10,0,-10,5, \n 15,-7,0,0,15],\n [0,22,-10,0,10, \n 8,0,-3,0,-2],\n [20,3,-2,0,0, \n 0,0,0,13,3],\n [35,20,-2,0,3, \n -2,-12,0,-5,0],\n [20,20,-4,7,5, \n 5,-6,10,14,4],\n \n\n [22,15,0,0,5, \n 15,0,0,0,15],\n [20,20,-10,-15,15, \n 8,10,-3,0,-2],\n [40,16,-2,11,11, \n 0,-11,0,13,3],\n [35,2,-2,0,19, \n 12,7,0,15,0],\n [20,11,-4,7,5, \n 5,19,0,14,4],\n \n\n [20,-5,0,-11,5, \n 15,10,0,-18,-5],\n [0,11,-10,-15,0, \n 8,10,-3,0,-2],\n [40,13,-2,0,0, \n 0,0,0,13,3],\n [35,2,-2,0,-3, \n -2,7,0,-5,10],\n [20,11,-4,7,5, \n 5,-6,10,14,4],\n \n\n [40,-10,0,0,15, \n 15,-10,0,0,-15],\n [0,10,-10,-15,15, \n -8,10,-3,0,-2],\n [40,-3,-2,0,0, \n 10,-10,0,3,3],\n [35,2,-2,10,3, \n -2,7,0,5,5],\n [20,11,-4,-20,5, \n -5,-6,0,14,4],\n \n\n [40,0,-10,0,15, \n 15,-20,-30,0,5],\n [0,10,0,5,-5, \n 8,-10,-3,-5,12],\n [40,-3,-2,0,-20, \n 10,10,0,3,3],\n [50,-20,-2,10,3, \n -2,7,0,5,5],\n [20,11,-4,-20,5, \n -5,11,0,14,-14],\n \n\n [50,-20,0,0,15, \n 15,-10,0,0,-10],\n [0,10,-10,5,15, \n -8,20,-3,0,-2],\n [50,23,-2,0,0, \n 10,-10,0,23,9],\n [35,2,-2,10,3, \n -2,7,10,5,5],\n [20,8,-4,0,5, \n 5,-16,0,14,4],\n \n\n [40,-10,0,0,15, \n 15,-10,0,10,15],\n [0,0,10,-15,0, \n 18,10,-3,0,-2],\n [40,-3,-2,0,10, \n 20,-10,0,3,3],\n [35,2,-2,10,3, \n -2,-37,0,5,5],\n [30,11,-4,0,5, \n -5,-6,0,14,-4],\n \n\n [40,-10,0,0,15, \n 15,-10,0,0,5],\n [10,15,-10,-15,15, \n -8,-10,-3,0,-2],\n [40,-3,-2,0,0, \n 10,-10,0,3,3],\n [35,-2,-2,10,3, \n -2,7,0,5,5],\n [20,5,-4,-20,5, \n -5,-6,0,14,4],\n \n #---------------------------------------------------------------\n \n [60,15,-5,10,-5, \n 60,-30,35,20,25],\n [0,15,15,-45,-5,\n -5,-10,0,10,10],\n [10,5,0,0,0, \n 0,10,0,35,0], \n [15,10,0,0,20, \n 0,-15,-15,30,-35],\n [0,15,0,0,10, \n 10,-25,0,20,0], \n [0,12,0,0,5, \n -20,-25,0,0,0], \n [0,20,0,0,0, \n 0,15,0,0,-28],\n [40,15,0,5,0, \n 0,0,0,0,-40],\n [0,0,0,15,0, \n 0,-35,-20,-10,0], \n [15,15,0,0,5, \n 0,0,-10,20,0], \n [0,25,15,0,5, \n 0,-65,0,30,20], \n [13,15,0,0,20, \n 15,0,0,10,0],\n [14,5,0,0,20, \n 15,-30,0,10,40], \n [16,15,0,0,5, \n 15,-30,0,10,0], \n [15,25,0,-20,5, \n 15,-15,15,0,20],\n [0,25,-25,10,-5, \n 0,-45,-10,-10,0], \n [-10,-5,-20,0,-5, \n -10,0,15,-20,15], \n [14,5,-55,-20,0, \n 30,10,0,10,5], \n [16,5,0,0,25, \n 15,-10,0,0,0], \n [15,15,10,20,35, \n 15,35,15,45,20],\n [-30,10,-10,0,30, \n 0,-25,0,20,-10], \n [0,15,0,5,20, \n 15,-30,0,-5,0],\n [30,15,-10,-5,0, \n 30,0,15,0,0], \n [16,15,0,-30,25, \n 5,-30,15,10,0], \n [25,5,0,5,20, \n 15,-15,0,15,20],\n [0,25,-25,-15,5, \n 20,-15,-10,10,0], \n [-10,20,10,20,20, \n -10,0,5,0,5], \n [14,-5,0,0,25, \n 10,0,0,10,5], \n [16,-5,0,0,0, \n -5,-10,0,35,0], \n [25,0,-20,-15,5, \n -5,-15,0,0,-5],\n [15,5,15,-5,20, \n 0,-35,-10,-10,25], \n [-10,40,-20,-30,20, \n -10,-30,30,-20,15], \n [14,-10,0,5,15, \n 0,-40,0,10,25], \n [25,25,0,-15,-5, \n 15,-30,15,15,0], \n [25,15,-5,0,0, \n 0,-60,5,15,30],\n [20,25,-5,10,15, \n 26,-5,12,8,5],\n [-10,10,-5,-7,40, \n -5,-10,-5,-10,15], \n [14,5,0,-5,0, \n 5,-18,-5,-8,5], \n [16,12,-8,0,0, \n -10,-68,-6,-3,0], \n [15,0,10,0,10, \n 15,-5,0,28,13],#40\n #####\n [60,15,-5,20,-5, \n 20,-30,-20,20,25], \n [0,15,0,-5,20,\n 20,-10,0,10,10], \n [10,5,0,15,0, \n 15,-10,0,-5,20], \n [15,0,10,15,5, \n 0,0,-15,30,5],\n [0,5,-10,12,18, \n 10,-60,0,0,0], \n [15,12,-10,0,5, \n 0,-30,0,0,0], \n [0,20,-13,0,0, \n 0,-30,0,8,-20],\n [40,5,0,5,0, \n 0,0,0,0,10], \n [0,0,0,-10,0, \n 5,-20,-20,20,0], \n [15,0,0,-15,20, \n -25,0,-10,20,0], #50\n #####\n [0,15,0,5,30, \n 0,-25,0,-13,-10], \n [30,10,0,10,-20, \n -10,-15,0,0,-10], \n [16,28,0,30,-20, \n 5,-30,0,10,0], \n [25,25,0,5,17, \n 15,-25,-10,5,0],\n [0,30,5,0,5, \n 0,-65,15,25,0], \n [0,-5,0,20,20, \n 20,-15,0,30,-15], \n [14,5,20,15,0, \n 15,-25,0,20,5], \n [16,5,5,10,5, \n -5,-75,0,30,10], \n [25,0,-10,-5,-5, \n 15,5,-15,-15,-5],\n [15,5,15,-5,20, \n 15,-20,-10,-10,25], \n \n #####\n #60\n [15,25,-5,-5,0, \n 15,-20,-10,0,15], \n [0,15,0,5,0, \n 0,-10,0,-5,-10],\n [30,20,0,-5,-10, \n 30,0,15,0,-5], \n [25,28,0,5,15, \n 15,-15,0,-20,0], \n [25,5,0,0,30, \n 0,-65,10,25,0],\n [0,-5,5,-15,15, \n 20,-10,0,25,0], \n [0,-10,-5,0,0, \n 0,0,15,30,20], \n [14,-5,-10,-10,10, \n -18,-25,0,0,5], \n [16,15,5,10,-5, \n 20,-10,0,5,10], \n [25,0,10,5,-5, \n 15,15,-15,15,-20],\n ###Du\n \n #70 \n [15,10,15,-5,20, \n 20,-35,0,5,-10], \n [20,30,10,-5,0, \n -10,0,10,5,10], \n [14,-10,0,-15,15, \n 0,0,0,10,25], \n [25,25,0,0,10, \n 15,-30,15,15,0], \n [25,15,-5,0,25, \n 0,-10,5,15,10],\n [20,25,-20,-5,15, \n -5,-5,0,8,5],\n [-10,10,-5,-7,10, \n 15,-30,-5,-10,-25], \n [14,5,0,15,18, \n -10,-18,-5,10,5], \n [16,12,-8,0,0, \n 5,-15,-6,-3,0], \n [15,0,10,0,10, \n -5,-5,0,28,13],#80\n ##############\n [-30,10,-10,0,0, \n 0,-25,0,20,-10], \n [0,15,0,-15,20, \n 15,-30,0,-5,0],\n [30,15,-10,-5,0, \n 30,0,15,0,0], \n [16,15,0,-30,25, \n 5,-30,15,10,0], \n [25,5,0,5,20, \n 15,-15,0,15,20],\n [0,25,-25,-15,5, \n 20,-15,-10,10,0], \n [-10,20,10,20,20, \n -10,0,5,0,5], \n [14,-5,0,0,25, \n 10,0,0,10,5], \n [16,-5,0,0,0, \n -5,-10,0,35,0], \n [25,0,-20,-15,5, \n -5,-15,0,0,-5],\n \n [15,5,15,-5,20, \n 0,-35,-10,-10,25], \n [-10,40,-20,-30,20, \n -10,-30,30,-20,15], \n [14,-10,0,5,15, \n 0,-40,0,10,25], \n [25,25,0,-15,-5, \n 15,-30,15,15,0], \n [25,15,-5,0,0, \n 0,-60,5,15,30],\n [20,25,-5,10,15, \n 26,-5,12,8,5],\n [-10,10,-5,-7,40, \n -5,-10,-5,-10,15], \n [14,5,0,-5,0, \n 5,-18,-5,-8,5], \n [16,12,-8,0,0, \n -10,-68,-6,-3,0], \n [15,0,10,0,10, \n 15,-5,0,28,13],#100\n ])\n}\n\nnumPickRotate = len(X_train_pick2rotate['index'])\n\nprint('numPickRotate = ', numPickRotate)\n\n# -------------------------------------------------- rotate\n\nX_sample_rotate = np.zeros((10,numRotate_prior,numPickRotate,X_train.shape[1]))\ny_sample_rotate = np.zeros((10,numRotate_prior,numPickRotate))\n\nfor k0 in range(10):\n for k1 in range(numRotate_prior):\n for k2 in range(numPickRotate):\n \n idxPick_X_train = X_train_pick2rotate['index'][k2]\n \n idx = np.where(y_train==k0)[0][idxPick_X_train]\n \n y_sample_rotate[k0,k1,k2] = k0 * numRotate_prior + k1\n \n angleRotate = (k1/numRotate_prior * angleRange_prior) + angleOffset_prior + X_train_pick2rotate['offset'][k2][k0]\n \n X_sample_rotate[k0,k1,k2,:] = func_array1d_rotate2d(X_train[idx][np.newaxis,:],angleRotate, 28)\n\n \nX_sample_rotate[X_sample_rotate < 10] = 0\n#X_sample_rotate[X_sample_rotate > 300] = 0\n\n# Let's normalize the pixel values to the interval [0,1]\n# Formula: MinMaxscale(X) = (X - X_min) / (X_max - X_min)\n\nprint(\"Before applying MinMax Scaler:\")\nprint(\" - Minimum value:\", X_sample_rotate.min())\nprint(\" - Maximum value:\", X_sample_rotate.max())\n\nminmax_scaler = MinMaxScaler()\n\nfor k0 in range(10):\n for k1 in range(numRotate_prior):\n X_sample_rotate[k0,k1,:,:] = minmax_scaler.fit_transform(X_sample_rotate[k0,k1,:,:]) * 255\n\nprint(\"\\nAfter applying MinMax Scaler:\")\nprint(\" - Minimum value:\", X_sample_rotate.min())\nprint(\" - Maximum value:\", X_sample_rotate.max())\n\n\n\n# Next, let's visualize one image from each class.\n# We will use the matplotlib 'imshow' function.\n\n# for k2 in range(numPickRotate):\nfor k2 in range(5,10):\n k1 = 0\n # k1 = numRotate_prior - 1\n\n fig, axs = plt.subplots(nrows=2, ncols=5, figsize=(10,5))\n\n for k0 in range(10):\n img = axs[int(k0 >= 5), k0%5].imshow(\n X=X_sample_rotate[k0,k1,k2,:].reshape(28, 28),\n cmap='Blues'\n )\n img.axes.get_yaxis().set_visible(False)\n img.axes.get_xaxis().set_visible(False)\n img.axes.set_title(f'Label: {k0}')\n \nX_train_rotate = np.reshape(X_sample_rotate, (-1,784))\ny_train_rotate = np.reshape(y_sample_rotate, (-1,1)).flatten()","metadata":{"_uuid":"2fd8cd6b-8db8-45dc-b5f7-6721c80f11fd","_cell_guid":"6b163e1f-7fee-43a6-b6f6-40c0d402a4fc","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-06-13T16:19:12.868416Z","iopub.execute_input":"2024-06-13T16:19:12.868793Z","iopub.status.idle":"2024-06-13T16:19:28.063480Z","shell.execute_reply.started":"2024-06-13T16:19:12.868761Z","shell.execute_reply":"2024-06-13T16:19:28.062366Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---------\nPhần 3: Phân tích thành phần chính","metadata":{"_uuid":"f67ae6c7-11f8-4b88-a23e-bf62fb9fdb43","_cell_guid":"6e9093b4-f586-4545-b99e-b59d492b7958","trusted":true}},{"cell_type":"markdown","source":"
\n Phân tích Thành phần Chính (PCA) là một kỹ thuật thống kê để giảm chiều dữ liệu. Điều này được thực hiện bằng cách biến đổi tuyến tính dữ liệu thành một hệ tọa độ mới, nơi mà (hầu hết) sự biến đổi trong dữ liệu có thể được mô tả với ít chiều hơn so với dữ liệu ban đầu.

\nThành phần chính đầu tiên có thể được định nghĩa tương đương là một hướng tối đa hóa phương sai của dữ liệu được chiếu. Thành phần chính thứ i có thể được lấy làm một hướng vuông góc với i-1 thành phần chính đầu tiên, mà tối đa hóa phương sai của dữ liệu được chiếu.

\nsklearn.decomposition.PCA: Giảm chiều tuyến tính sử dụng Phân tích Giá trị Kỳ dị của dữ liệu để chiếu nó vào một không gian chiều thấp hơn.

\nỞ đây chúng ta sẽ sử dụng 10 thành phần chính hàng đầu, giải thích gần 50% phương sai trong dữ liệu. Điều này có nghĩa là chúng ta có thể biểu diễn 50% phương sai của dữ liệu với một không gian đặc trưng nhỏ hơn 98% so với không gian ban đầu (10 đặc trưng so với 784 đặc trưng).\n
","metadata":{"_uuid":"c9145741-6e7a-4419-bdb0-baca414ed4e7","_cell_guid":"54390ba7-75cb-4867-b577-97f23c85d6f2","trusted":true}},{"cell_type":"code","source":"# Let's normalize the pixel values to the interval [0,1]\n# Formula: MinMaxscale(X) = (X - X_min) / (X_max - X_min)\n\nprint(\"Before applying MinMax Scaler:\")\nprint(\" - Minimum value:\", X_train.min())\nprint(\" - Maximum value:\", X_train.max())\n\nminmax_scaler = MinMaxScaler()\nX_train_rotate_scaled = minmax_scaler.fit_transform(X_train_rotate)\n\nprint(\"\\nAfter applying MinMax Scaler:\")\nprint(\" - Minimum value:\", X_train_rotate_scaled.min())\nprint(\" - Maximum value:\", X_train_rotate_scaled.max())","metadata":{"_uuid":"aad53bf6-9b1f-4451-844d-ae51f509a68f","_cell_guid":"2f528db2-3b92-425b-a18c-2168dd8862d7","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-06-13T16:19:28.065088Z","iopub.execute_input":"2024-06-13T16:19:28.066499Z","iopub.status.idle":"2024-06-13T16:19:28.350889Z","shell.execute_reply.started":"2024-06-13T16:19:28.066452Z","shell.execute_reply":"2024-06-13T16:19:28.349710Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Actually, we will embed the MinMaxScaler and the PCA\n# modules into a Sklearn Pipeline.\n\npca = Pipeline(steps=[\n ('scaler', MinMaxScaler()),\n ('pca', PCA(n_components=40))\n])\n\nX_train_rotate_pca = pca.fit_transform(X=X_train_rotate)\n\nprint('X_train_pca shape:', X_train_rotate_pca.shape)","metadata":{"_uuid":"6cae5170-3af3-4ed9-a9d8-372ef224e9ba","_cell_guid":"e423b865-ef1d-4c48-9152-917f8b258060","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-06-13T16:19:28.352303Z","iopub.execute_input":"2024-06-13T16:19:28.352758Z","iopub.status.idle":"2024-06-13T16:19:31.837293Z","shell.execute_reply.started":"2024-06-13T16:19:28.352716Z","shell.execute_reply":"2024-06-13T16:19:31.835995Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Let's extract the explained variance ratio\n# And let's compute their cumulative sum\n\npca_object = pca.named_steps['pca']\nvar_ratio = pd.Series(\n data=pca_object.explained_variance_ratio_,\n index=range(1, 41)\n)\n\ncumsum_var_ratio = var_ratio.cumsum()","metadata":{"_uuid":"68b1ffab-f2de-43fe-a655-e04c6060ec60","_cell_guid":"c3cdf97b-6136-45e0-94c6-bdf23e750bf0","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-06-13T16:19:31.838546Z","iopub.execute_input":"2024-06-13T16:19:31.838916Z","iopub.status.idle":"2024-06-13T16:19:31.845341Z","shell.execute_reply.started":"2024-06-13T16:19:31.838886Z","shell.execute_reply":"2024-06-13T16:19:31.844118Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Plot the explained variance ratio for each component (left)\n# And plot their cumulative sum (right)\n\nfig, (ax1, ax2) = plt.subplots(1, 2, figsize=(15,5))\n\nsns.barplot(\n x=var_ratio.index, \n y=var_ratio.values, \n color='#4895EE', \n ax=ax1\n)\nax1.set(\n title='Explained Variance Ratio PCA',\n xlabel='Component',\n ylabel='Var Ratio',\n)\nax1.set_ylim(0, 0.5)\n\nsns.barplot(\n x=cumsum_var_ratio.index, \n y=cumsum_var_ratio.values, \n color='#4895EE', \n ax=ax2\n)\nax2.set(\n title='Cumulative Explained Variance Ratio PCA',\n xlabel='Component',\n ylabel='Cumulative Var Ratio',\n)\n_ = ax2.set_ylim(0, 0.5)","metadata":{"_uuid":"9ea36679-af0c-40f1-bda2-641ef9c73a4e","_cell_guid":"6fe44c61-1ac9-4157-968d-022d42184ff6","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-06-13T16:19:31.846721Z","iopub.execute_input":"2024-06-13T16:19:31.847049Z","iopub.status.idle":"2024-06-13T16:19:33.304658Z","shell.execute_reply.started":"2024-06-13T16:19:31.847022Z","shell.execute_reply":"2024-06-13T16:19:33.303434Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Let's put the components in Pandas DataFrame format\n# This way we can use the seaborn and plotly libraries\n# to visualize the data.\n# (these packages work better with Pandas DataFrames)\n\ndf_train_pca = pd.DataFrame(\n data=X_train_rotate_pca[:,:3], \n columns=['x', 'y', 'z']\n)\n# Let's also add the target to set the colors\ndf_train_pca['target'] = y_train_rotate\n\nprint(df_train_pca.shape)\ndf_train_pca.head()","metadata":{"_uuid":"9796e066-c6af-49c5-af8f-8ebde5e07430","_cell_guid":"4fb22220-cc87-410b-a637-55bc57791987","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-06-13T16:19:33.306242Z","iopub.execute_input":"2024-06-13T16:19:33.306713Z","iopub.status.idle":"2024-06-13T16:19:33.324693Z","shell.execute_reply.started":"2024-06-13T16:19:33.306669Z","shell.execute_reply":"2024-06-13T16:19:33.323557Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Two-dimensional visualization of the data\n# Here the Top2 principal components are plotted\n\nsns.set_style(\"dark\")\nax = sns.scatterplot(\n data=df_train_pca,\n x='x', \n y='y', \n hue='target', \n palette='Blues',\n)\nax.set(\n title='PCA Two-Dimensional Projection',\n xlabel='1st Comp.',\n ylabel='2nd Comp.'\n)\nax.axhline(y=0, linestyle='--', color='#FFBF00', zorder=-1)\nax.axvline(x=0, linestyle='--', color='#FFBF00', zorder=-1)","metadata":{"_uuid":"30135dee-d7d0-4dc8-88f7-3a88a135ad5a","_cell_guid":"147fd5ee-e632-4611-b6de-35d64c6946e9","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-06-13T16:19:33.326372Z","iopub.execute_input":"2024-06-13T16:19:33.327393Z","iopub.status.idle":"2024-06-13T16:19:38.010223Z","shell.execute_reply.started":"2024-06-13T16:19:33.327357Z","shell.execute_reply":"2024-06-13T16:19:38.008706Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Now, let's apply the inverse_transform to compute\n# the approximation of images based on the 10 principal\n# compontents.\n# We can see that with just 10 components we convey a large\n# part of the information of each image.\n\npca_object = pca.named_steps['pca']\n\nfig, axs = plt.subplots(nrows=10, ncols=2, figsize=(6,20))\n\nfor i in range(10):\n idx = np.where(y_train==i)[0][0]\n original_img = axs[i, 0].imshow(\n X=X_train_rotate[idx].reshape(28, 28),\n cmap='Blues'\n )\n original_img.axes.get_yaxis().set_visible(False)\n original_img.axes.get_xaxis().set_visible(False)\n original_img.axes.set_title(f'Original [{i}]:')\n \n approx_img_ = pca_object.inverse_transform(X=X_train_rotate_pca[idx])\n approx_img = axs[i, 1].imshow(\n X=approx_img_.reshape(28, 28),\n cmap='Blues'\n )\n approx_img.axes.get_yaxis().set_visible(False)\n approx_img.axes.get_xaxis().set_visible(False)\n approx_img.axes.set_title(f'Approximation [{i}]:')","metadata":{"_uuid":"77146579-275e-4359-8ec9-7d175da30220","_cell_guid":"329c95b3-2406-402c-b1f8-38487c4fa2f3","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-06-13T16:19:38.011781Z","iopub.execute_input":"2024-06-13T16:19:38.012146Z","iopub.status.idle":"2024-06-13T16:19:39.054440Z","shell.execute_reply.started":"2024-06-13T16:19:38.012116Z","shell.execute_reply":"2024-06-13T16:19:39.053302Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Show that the PCA components are uncorrelated\n# (This statement is true by definition and construction\n# of the Principal Component Analysis algorithm)\n\ncorr = np.corrcoef(x=X_train_rotate_pca, rowvar=False)\ncorr = np.around(corr, decimals=2)\n\nax = sns.heatmap(\n data=corr, \n vmax=1, \n vmin=-1,\n square=True,\n linewidths=.5,\n annot=True,\n cmap='Blues',\n)\n\n_ = ax.set(\n title='Correlation PCA Components [Uncorrelated]',\n xlabel='Components',\n ylabel='Components'\n)","metadata":{"_uuid":"8c3fa8bb-61ab-43dc-bc7e-e5fecc13e6e2","_cell_guid":"ab321270-3628-423b-8fd6-a7ebaf6ccfe5","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-06-13T16:19:39.056034Z","iopub.execute_input":"2024-06-13T16:19:39.056479Z","iopub.status.idle":"2024-06-13T16:19:43.820237Z","shell.execute_reply.started":"2024-06-13T16:19:39.056429Z","shell.execute_reply":"2024-06-13T16:19:43.819109Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"----\n# Phần 4: Sử dụng K-Nearest Neighbors Classifier phân loại 10 thành phần chính hàng đầu từ PCA.","metadata":{"_uuid":"3d69b1a4-faf3-4405-b740-9d5fc82b8424","_cell_guid":"16cefed3-22ad-4c84-ad87-230ed47df34d","trusted":true}},{"cell_type":"markdown","source":"
\nThuật toán K-Nearest Neighbors là một bộ phân loại học có giám sát, không tham số, sử dụng sự gần gũi để đưa ra các phân loại hoặc dự đoán về sự nhóm của một điểm dữ liệu cá nhân. Dự đoán: Một nhãn lớp được gán dựa trên một bỏ phiếu đa số. nhãn được đại diện nhiều nhất xung quanh một điểm dữ liệu nhất định.

\nTrong dự án này, đầu vào của Bộ phân loại KNN sẽ là 10 thành phần chính hàng đầu từ PCA.

\nsklearn.neighbors.KNeighborsClassifier: Bộ phân loại thực hiện bỏ phiếu k-nearest neighbors. Chúng ta sẽ chạy GridSearchCV để tối ưu hóa siêu tham số n_neighbors.\n
","metadata":{"_uuid":"1abd82c5-f4e1-4142-acb2-bea45e42488e","_cell_guid":"b9e0603d-4dc8-4761-baad-a34cf6709975","trusted":true}},{"cell_type":"markdown","source":"---\n# mô hình học máy trên tập dữ liệu Train","metadata":{"_uuid":"334a6d45-a6b2-499e-a95b-2477800b968f","_cell_guid":"e1b5a3d3-ced9-409c-b6f9-b07e7143bc66","trusted":true}},{"cell_type":"code","source":"knn_clf = Pipeline(steps=[\n ('scaler', MinMaxScaler()),\n ('pca', PCA(n_components=10)),\n ('knn', KNeighborsClassifier())\n])\n\nknn_clf_params = {\n 'knn__n_neighbors': [2, 4, 8, 16, 32]\n}\n\nknn_clf_gs = GridSearchCV(\n estimator=knn_clf, \n param_grid=knn_clf_params,\n scoring='accuracy',\n n_jobs=-1,\n cv=5,\n verbose=1,\n return_train_score=True\n)\n\nknn_clf_gs.fit(X=X_train_rotate_pca, y=y_train_rotate)\n\nprint('best PCA+KNN params:', knn_clf_gs.best_params_)\nprint('best PCA+KNN cv accuracy:', knn_clf_gs.best_score_)","metadata":{"_uuid":"7154a244-1406-4e55-9062-2a876ffcfb42","_cell_guid":"f8b23485-498b-446c-83a4-fdf1e50b8285","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-06-13T16:19:43.821483Z","iopub.execute_input":"2024-06-13T16:19:43.821814Z","iopub.status.idle":"2024-06-13T16:20:53.403493Z","shell.execute_reply.started":"2024-06-13T16:19:43.821786Z","shell.execute_reply":"2024-06-13T16:20:53.402160Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Plot the Training and Validation Accuracies\n# for each Hyperparamter value tested during the\n# Grid Search.\n# This plot allows us to check for overfitting.\n\nfig, ax = plt.subplots()\n\nax.plot(\n range(len(knn_clf_params['knn__n_neighbors'])), \n knn_clf_gs.cv_results_['mean_train_score'],\n label='train',\n color='blue'\n)\nax.plot(\n range(len(knn_clf_params['knn__n_neighbors'])),\n knn_clf_gs.cv_results_['mean_test_score'],\n label='test',\n color='orange'\n)\nax.set_xticks(\n ticks=range(len(knn_clf_params['knn__n_neighbors'])),\n labels=knn_clf_params['knn__n_neighbors']\n)\n\nii = knn_clf_params['knn__n_neighbors'].index(\n knn_clf_gs.best_params_['knn__n_neighbors']\n)\n\nplt.axvline(x=ii, linestyle='--', color='red', label='best', alpha=0.3)\nplt.legend()\nplt.xlabel(\"'n_neighbors' vals\")\nplt.ylabel('Accuracy')\nplt.title(\"PCA+KNN GridSearchCV Results\")\nplt.show()","metadata":{"_uuid":"43e1d701-a89a-463f-b860-3afbf14bbf36","_cell_guid":"b1316942-7e1c-4f06-8742-d3eb8f7918c3","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-06-13T16:20:53.405787Z","iopub.execute_input":"2024-06-13T16:20:53.406254Z","iopub.status.idle":"2024-06-13T16:20:53.804949Z","shell.execute_reply.started":"2024-06-13T16:20:53.406200Z","shell.execute_reply":"2024-06-13T16:20:53.803703Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n# Phần 5: Tạo một tệp tin chứa các dự đoán của mô hình học máy trên tập dữ liệu Test","metadata":{"_uuid":"b609f283-8a5c-477c-9c29-941acfedc9aa","_cell_guid":"5d9eb764-a90a-48b5-b962-be1b347e4925","trusted":true}},{"cell_type":"code","source":"from sklearn.metrics import accuracy_score\n\n# Compute and display the Confusion Matrix\n# Note: the entire training dataset is used here\n\nX_val = X_train_rotate_pca\nY_val = y_train_rotate\n\nbest_cv_clf = knn_clf_gs.best_estimator_\ny_pred = best_cv_clf.predict(X=X_val)\n\ny_val_number = Y_val // numRotate_prior\ny_pred_number = y_pred // numRotate_prior\n\ny_val_angle = (Y_val % numRotate_prior)\ny_pred_angle = (y_pred % numRotate_prior)\n\nprint('KNN number + angle, Acc({:.2f}%)', accuracy_score(Y_val, y_pred))\nprint('KNN number, Acc({:.2f}%)', accuracy_score(y_val_number, y_pred_number))\nprint('KNN angle, Acc({:.2f}%)', accuracy_score(y_val_angle, y_pred_angle))\n\n# print('y_val_number =', y_val_number)\n# print('y_pred_number =', y_pred_number)\n\ny_val_angle_degree = y_val_angle/numRotate_prior * angleRange_prior + angleOffset_prior\ny_pred_angle_degree = y_pred_angle/numRotate_prior * angleRange_prior + angleOffset_prior\n\n# print('y_val_angle_degree =', y_val_angle_degree)\n# print('y_pred_angle_degree =', y_pred_angle_degree)\n\nconf_matrix = confusion_matrix(y_val_number, y_pred_number)\nconf_matrix_disp = ConfusionMatrixDisplay(conf_matrix)\nconf_matrix_disp.plot(cmap='Blues')\n\nconf_matrix = confusion_matrix(y_val_angle, y_pred_angle)\nconf_matrix_disp = ConfusionMatrixDisplay(conf_matrix)\nconf_matrix_disp.plot(cmap='Blues')","metadata":{"_uuid":"62d7493a-d5c0-42ca-ad56-df419c3367ee","_cell_guid":"fdfdc4ce-7aa6-4c8a-b7c9-8a069cb98054","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-06-13T16:20:53.806555Z","iopub.execute_input":"2024-06-13T16:20:53.806998Z","iopub.status.idle":"2024-06-13T16:21:04.859278Z","shell.execute_reply.started":"2024-06-13T16:20:53.806959Z","shell.execute_reply":"2024-06-13T16:21:04.858163Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X_val = pca.transform(X_train)\nY_val = y_train\n\nbest_cv_clf = knn_clf_gs.best_estimator_\ny_pred = best_cv_clf.predict(X=X_val)\n\ny_pred_number = y_pred\ny_val_number = Y_val\n\nprint('KNN number, Acc({:.2f}%)', accuracy_score(y_val_number, y_pred_number))\n\nconf_matrix = confusion_matrix(y_val_number, y_pred_number)\nconf_matrix_disp = ConfusionMatrixDisplay(conf_matrix)\nconf_matrix_disp.plot(cmap='Blues')\n\ny_pred_angle = (y_pred % numRotate_prior)\n\ny_pred_angle_degree = y_pred_angle/numRotate_prior * angleRange_prior + angleOffset_prior\n\nprint('y_pred_angle_degree =', y_pred_angle_degree)","metadata":{"_uuid":"da419e38-d545-49ee-a8b6-32f3fa9760c3","_cell_guid":"fa5652c2-d596-4880-8396-b6e77177f4ff","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-06-13T16:21:04.860800Z","iopub.execute_input":"2024-06-13T16:21:04.861207Z","iopub.status.idle":"2024-06-13T16:21:42.155683Z","shell.execute_reply.started":"2024-06-13T16:21:04.861166Z","shell.execute_reply":"2024-06-13T16:21:42.154586Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Now, let's apply the inverse_transform to compute\n# the approximation of images based on the 10 principal\n# compontents.\n# We can see that with just 10 components we convey a large\n# part of the information of each image.\n\npca_object = pca.named_steps['pca']\n\nfig, axs = plt.subplots(nrows=10, ncols=2, figsize=(6,20))\n\nfor i in range(10):\n idx = np.where(y_train == i )[0][0]\n original_img = axs[i, 0].imshow(\n X=X_train[idx].reshape(28, 28),\n cmap='Blues'\n )\n original_img.axes.get_yaxis().set_visible(False)\n original_img.axes.get_xaxis().set_visible(False)\n original_img.axes.set_title(f'Original [{i}]:')\n \n approx_img_ = pca_object.inverse_transform(X=X_val[idx])\n approx_img = axs[i, 1].imshow(\n X=approx_img_.reshape(28, 28),\n cmap='Blues'\n )\n approx_img.axes.get_yaxis().set_visible(False)\n approx_img.axes.get_xaxis().set_visible(False)\n approx_img.axes.set_title(f'Predicted angle [{y_pred_angle_degree[idx]}]:')","metadata":{"_uuid":"bc4e4c18-00b8-4441-900b-83786685d798","_cell_guid":"1393d26a-3a87-4a77-a642-c5b68c94fec3","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-06-13T16:21:42.157161Z","iopub.execute_input":"2024-06-13T16:21:42.157539Z","iopub.status.idle":"2024-06-13T16:21:43.165384Z","shell.execute_reply.started":"2024-06-13T16:21:42.157504Z","shell.execute_reply":"2024-06-13T16:21:43.164267Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n
\n

\nLight CNN Classifier\n

\n
\n
","metadata":{"_uuid":"a1446365-443d-48e8-aa24-514805f2df61","_cell_guid":"f6dca012-b643-484b-8f63-b0e218c6f1e2","trusted":true}},{"cell_type":"markdown","source":"","metadata":{"_uuid":"5a238d9f-7fbe-4d2a-85c7-b38f1793fe47","_cell_guid":"d606a8da-01ba-4650-b234-842037898d97","trusted":true}},{"cell_type":"markdown","source":"### CNN is a very efficient ways for computer vision tasks. \na convolutional neural network (CNN, or ConvNet) is a class of Artificial Neural Network (ANN), most commonly applied to analyze visual imagery. They are also known as Shift Invariant or Space Invariant Artificial Neural Networks (SIANN), based on the shared-weight architecture of the convolution kernels or filters that slide along input features and provide translation equivariant responses known as feature maps. Counter-intuitively, most convolutional neural networks are only equivariant, as opposed to invariant, to translation. They have applications in image and video recognition, recommender systems, image classification, image segmentation, medical image analysis, natural language processing, brain-computer interfaces, and financial time series.","metadata":{"_uuid":"3f62053b-0364-4ff6-bb7b-5deba2a02280","_cell_guid":"1c349770-f9d8-4f7f-8efb-d87b402f2f10","trusted":true}},{"cell_type":"code","source":"import plotly.express as px\nfrom plotly.offline import init_notebook_mode, iplot\nimport plotly.graph_objects as go\nimport warnings\nwarnings.filterwarnings('ignore')","metadata":{"execution":{"iopub.status.busy":"2024-06-13T16:21:43.166964Z","iopub.execute_input":"2024-06-13T16:21:43.167852Z","iopub.status.idle":"2024-06-13T16:21:43.197590Z","shell.execute_reply.started":"2024-06-13T16:21:43.167802Z","shell.execute_reply":"2024-06-13T16:21:43.196477Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def plot_confusion_matrix(cm, labels, title):\n # cm : confusion matrix list(list)\n # labels : name of the data list(str)\n # title : title for the heatmap\n data = go.Heatmap(z=cm, y=labels, x=labels)\n annotations = []\n for i, row in enumerate(cm):\n for j, value in enumerate(row):\n annotations.append(\n {\n \"x\": labels[i],\n \"y\": labels[j],\n \"font\": {\"color\": \"white\"},\n \"text\": str(value),\n \"xref\": \"x1\",\n \"yref\": \"y1\",\n \"showarrow\": False\n }\n )\n layout = {\n \"title\": title,\n \"xaxis\": {\"title\": \"Predicted value\"},\n \"yaxis\": {\"title\": \"Real value\"},\n \"annotations\": annotations\n }\n fig = go.Figure(data=data, layout=layout)\n return fig","metadata":{"execution":{"iopub.status.busy":"2024-06-13T16:21:43.199101Z","iopub.execute_input":"2024-06-13T16:21:43.199528Z","iopub.status.idle":"2024-06-13T16:21:43.209803Z","shell.execute_reply.started":"2024-06-13T16:21:43.199486Z","shell.execute_reply":"2024-06-13T16:21:43.208710Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# from tensorflow.keras.utils import np_utils\nfrom tensorflow.keras import utils\nfrom tensorflow.keras.callbacks import ModelCheckpoint, EarlyStopping\nfrom tensorflow.keras import datasets, layers, models","metadata":{"_uuid":"17fc4707-d986-4950-b0fe-ac152932ac00","_cell_guid":"0d0efe69-4e03-4204-86d8-3ee2e5125830","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-06-13T16:21:43.211242Z","iopub.execute_input":"2024-06-13T16:21:43.211578Z","iopub.status.idle":"2024-06-13T16:21:56.553492Z","shell.execute_reply.started":"2024-06-13T16:21:43.211550Z","shell.execute_reply":"2024-06-13T16:21:56.552352Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X_train_cnn = X_train_rotate.reshape(-1,28,28,1)\n# X_val = np.array(pca.transform(X_train)).reshape(-1,28,28,1)\ny_train_cnn = utils.to_categorical(np.array(y_train_rotate// numRotate_prior))\nX_val = X_train.reshape(-1,28,28,1)\nY_val = utils.to_categorical(np.array(y_train))","metadata":{"_uuid":"598e24f0-4665-4486-ba5d-fb550022bc96","_cell_guid":"b20ca6f3-b288-42fe-8210-9e804c27268e","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-06-13T16:21:56.560070Z","iopub.execute_input":"2024-06-13T16:21:56.560827Z","iopub.status.idle":"2024-06-13T16:21:56.569717Z","shell.execute_reply.started":"2024-06-13T16:21:56.560791Z","shell.execute_reply":"2024-06-13T16:21:56.568484Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X_train_cnn.shape","metadata":{"execution":{"iopub.status.busy":"2024-06-13T16:21:56.571273Z","iopub.execute_input":"2024-06-13T16:21:56.571747Z","iopub.status.idle":"2024-06-13T16:21:56.618298Z","shell.execute_reply.started":"2024-06-13T16:21:56.571701Z","shell.execute_reply":"2024-06-13T16:21:56.617070Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# y_train_rotate// numRotate_prior","metadata":{"execution":{"iopub.status.busy":"2024-06-13T16:21:56.620242Z","iopub.execute_input":"2024-06-13T16:21:56.620714Z","iopub.status.idle":"2024-06-13T16:21:56.628515Z","shell.execute_reply.started":"2024-06-13T16:21:56.620669Z","shell.execute_reply":"2024-06-13T16:21:56.627352Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# X_train_rotate = np.array(X_train_rotate).reshape(-1,28,28,1)\n# # X_val = np.array(pca.transform(X_train)).reshape(-1,28,28,1)\n# X_val = np.array(X_test_crop).reshape(-1,28,28,1)\n# y_train_rotate = utils.to_categorical(np.array(y_train_rotate))\n# Y_val = utils.to_categorical(np.array(y_test_crop))","metadata":{"_uuid":"f978abbd-867f-46a5-b6ec-35f64e740f64","_cell_guid":"9d27fd4b-f9a8-4c46-a856-7e8970246998","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-06-13T16:21:56.630088Z","iopub.execute_input":"2024-06-13T16:21:56.630481Z","iopub.status.idle":"2024-06-13T16:21:56.639899Z","shell.execute_reply.started":"2024-06-13T16:21:56.630448Z","shell.execute_reply":"2024-06-13T16:21:56.638861Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = models.Sequential()\nmodel.add(layers.Conv2D(filters = 32, kernel_size = (5,5),padding = 'Same', \n activation ='relu', input_shape = (28,28,1)))\nmodel.add(layers.Conv2D(filters = 32, kernel_size = (5,5),padding = 'Same', \n activation ='relu'))\nmodel.add(layers.MaxPool2D(pool_size=(2,2)))\nmodel.add(layers.BatchNormalization())\nmodel.add(layers.Dropout(0.25))\nmodel.add(layers.Conv2D(filters = 64, kernel_size = (3,3),padding = 'Same', \n activation ='relu'))\nmodel.add(layers.Conv2D(filters = 64, kernel_size = (3,3),padding = 'Same', \n activation ='relu'))\nmodel.add(layers.MaxPool2D(pool_size=(2,2), strides=(2,2)))\nmodel.add(layers.BatchNormalization())\nmodel.add(layers.Dropout(0.25))\nmodel.add(layers.Flatten())\nmodel.add(layers.Dense(256, activation = \"relu\"))\nmodel.add(layers.Dropout(0.5))\nmodel.add(layers.BatchNormalization())\nmodel.add(layers.Dense(10, activation = \"softmax\"))\n\nmodel.summary()","metadata":{"_uuid":"d30da279-3261-4e97-9b79-6e2a01d703d4","_cell_guid":"43ead369-05c5-4e2d-960c-d5773b8a7f7d","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-06-13T16:21:56.641100Z","iopub.execute_input":"2024-06-13T16:21:56.641434Z","iopub.status.idle":"2024-06-13T16:21:57.028838Z","shell.execute_reply.started":"2024-06-13T16:21:56.641396Z","shell.execute_reply":"2024-06-13T16:21:57.027701Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model_path = './best_model.h5'\ncheckpointer = ModelCheckpoint(\n filepath=model_path,\n monitor='val_accuracy',\n verbose=1,\n mode='max',\n save_best_only=True)\n\nmodel.compile(loss='categorical_crossentropy',\n optimizer='adam',\n metrics=['accuracy'])","metadata":{"_uuid":"d131aa49-918d-43dd-be19-62d3110ca17a","_cell_guid":"db889d42-b8bb-4667-8f07-b831a3fbcf38","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-06-13T16:21:57.030317Z","iopub.execute_input":"2024-06-13T16:21:57.030697Z","iopub.status.idle":"2024-06-13T16:21:57.053732Z","shell.execute_reply.started":"2024-06-13T16:21:57.030661Z","shell.execute_reply":"2024-06-13T16:21:57.052650Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"history = model.fit(X_train_cnn, y_train_cnn, validation_data = (X_val, Y_val), epochs=50, batch_size = 128, callbacks=[checkpointer])","metadata":{"_uuid":"620687bc-89d9-40dc-b133-006fef4e7c78","_cell_guid":"cd8420f8-3c35-407f-93de-b1691d1b534f","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-06-13T16:21:57.055202Z","iopub.execute_input":"2024-06-13T16:21:57.055549Z","iopub.status.idle":"2024-06-13T18:04:04.821409Z","shell.execute_reply.started":"2024-06-13T16:21:57.055519Z","shell.execute_reply":"2024-06-13T18:04:04.820338Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import plotly.graph_objects as go\nfrom plotly.subplots import make_subplots\nimport numpy as np\n\nfig = go.Figure()\n\nlabels = ['Loss', 'Accuracy']\ncolors = ['rgb(67,67,67)', 'rgb(49,130,189)']\n\nmode_size = [8, 8]\nline_size = [2, 2]\n\nx_data = np.vstack((np.arange(1, 51),)*2)\n\ny_data = np.array([\n np.array(history.history['loss']), \n np.array(history.history['accuracy'])\n])\n\nfor i in range(0, 2):\n fig.add_trace(go.Scatter(x=x_data[i], y=y_data[i], mode='lines',\n name=labels[i],\n line=dict(color=colors[i], width=line_size[i]),\n connectgaps=True,\n ))\n\n # endpoints\n fig.add_trace(go.Scatter(\n x=[x_data[i][0], x_data[i][-1]],\n y=[y_data[i][0], y_data[i][-1]],\n mode='markers',\n marker=dict(color=colors[i], size=mode_size[i])\n ))\n\nfig.update_layout(\n xaxis=dict(\n showline=True,\n showgrid=False,\n showticklabels=True,\n linecolor='rgb(204, 204, 204)',\n linewidth=2,\n ticks='outside',\n tickfont=dict(\n family='Arial',\n size=12,\n color='rgb(82, 82, 82)',\n ),\n ),\n yaxis=dict(\n showgrid=False,\n zeroline=False,\n showline=False,\n showticklabels=False,\n ),\n autosize=False,\n margin=dict(\n autoexpand=False,\n l=100,\n r=20,\n t=110,\n ),\n showlegend=False,\n plot_bgcolor='white'\n)\n\nannotations = []\n\n# Adding labels\nfor y_trace, label, color in zip(y_data, labels, colors):\n # labeling the left_side of the plot\n annotations.append(dict(xref='paper', x=0.05, y=y_trace[0],\n xanchor='right', yanchor='middle',\n text=label + ' {:.2f}'.format(y_trace[0]),\n font=dict(family='Arial',\n size=16),\n showarrow=False))\n # labeling the right_side of the plot\n annotations.append(dict(xref='paper', x=0.95, y=y_trace[49],\n xanchor='left', yanchor='middle',\n text='{:.4f}%'.format(y_trace[49]),\n font=dict(family='Arial',\n size=16),\n showarrow=False))\n# Title\nannotations.append(dict(xref='paper', yref='paper', x=0.0, y=1.05,\n xanchor='left', yanchor='bottom',\n text='Training Loss & Accuracy',\n font=dict(family='Arial',\n size=30,\n color='rgb(37,37,37)'),\n showarrow=False))\n# Source\nannotations.append(dict(xref='paper', yref='paper', x=0.5, y=-0.1,\n xanchor='center', yanchor='top',\n text='Epochs',\n font=dict(family='Arial',\n size=12,\n color='rgb(150,150,150)'),\n showarrow=False))\n\nfig.update_layout(annotations=annotations,height=600, width=1000)\n\nfig.show()","metadata":{"_uuid":"b2a98a55-6032-40d1-8bbf-bf6b35840730","_cell_guid":"7fcbf806-1db3-45b9-8bc0-3abf08dfabd9","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-06-13T18:04:04.823356Z","iopub.execute_input":"2024-06-13T18:04:04.823751Z","iopub.status.idle":"2024-06-13T18:04:05.300937Z","shell.execute_reply.started":"2024-06-13T18:04:04.823715Z","shell.execute_reply":"2024-06-13T18:04:05.299727Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"_uuid":"c51476fa-c9d6-4cbc-a5ad-f6ff9706d70e","_cell_guid":"8bdfa1eb-cd3b-4e48-a377-02b2b66506e1","collapsed":false,"jupyter":{"outputs_hidden":false},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from keras.models import load_model\nmodel = load_model('./best_model.h5')\ny_pred = model.predict(X_val)\nM = confusion_matrix(np.argmax(Y_val,1), np.argmax(y_pred,1))\nplot_confusion_matrix(M, sorted(np.unique(np.argmax(Y_val,1))), 'Simple CNN Confusion Matrix, Acc({:.2f}%)'.format(accuracy_score(np.argmax(Y_val,1), np.argmax(y_pred,1))*100))","metadata":{"_uuid":"57a8380a-11c2-45fb-8190-4e9da5bce1ed","_cell_guid":"bdeca8dc-2a69-4598-b508-873212a964dd","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-06-13T18:04:05.302277Z","iopub.execute_input":"2024-06-13T18:04:05.302642Z","iopub.status.idle":"2024-06-13T18:04:43.101820Z","shell.execute_reply.started":"2024-06-13T18:04:05.302594Z","shell.execute_reply":"2024-06-13T18:04:43.100623Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n
\n

\nChecking Failure Cases\n

\n
\n
","metadata":{"_uuid":"8f029ba0-23b9-4fa0-83a3-eae4a5e4b39b","_cell_guid":"d955f566-e62b-4f51-a801-3de790bca12a","trusted":true}},{"cell_type":"code","source":"n = np.where(np.argmax(y_pred,1)!=np.argmax(Y_val,1))\nn = np.array(n).reshape(-1)\nn","metadata":{"_uuid":"042a5e34-2598-420b-bb05-79a34d48eac3","_cell_guid":"3cea1159-64d0-4e09-88e9-7b2f3a4fc1fb","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-06-13T18:04:43.103088Z","iopub.execute_input":"2024-06-13T18:04:43.103428Z","iopub.status.idle":"2024-06-13T18:04:43.119845Z","shell.execute_reply.started":"2024-06-13T18:04:43.103400Z","shell.execute_reply":"2024-06-13T18:04:43.118728Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X_val_vis = np.array(X_val).reshape(X_val.shape[0], 28, 28)\nX_val_vis = X_val_vis\nfig, axis = plt.subplots(int(len(n)/8), 8, figsize=(35, 15))\nfor i, ax in enumerate(axis.flat):\n ax.imshow(X_val_vis[n[i]], cmap='binary')\n digit = np.argmax(y_pred,1)[n[i]]\n digit2 = np.argmax(Y_val,1)[n[i]]\n ax.set(title = f\"Pred Number is {digit}\\n Real Number is {digit2}\");","metadata":{"_uuid":"470a60fd-078e-4a6b-99bf-206ffd71cfb8","_cell_guid":"33ee870d-1eaa-4702-ac0b-7d41a9333040","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-06-13T18:04:43.121265Z","iopub.execute_input":"2024-06-13T18:04:43.121735Z","iopub.status.idle":"2024-06-13T18:06:41.657485Z","shell.execute_reply.started":"2024-06-13T18:04:43.121691Z","shell.execute_reply":"2024-06-13T18:06:41.656339Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### It looks like even human can't get the right answers.😅","metadata":{"_uuid":"7b0368af-6a8a-44d9-8dee-e994819269d4","_cell_guid":"11579e1a-7889-4904-ab0f-b96007a7069b","trusted":true}},{"cell_type":"markdown","source":"
\n
\n

\nSubmit to Leaderboard\n

\n
\n
","metadata":{"_uuid":"b052fb30-6e45-4c66-87af-ce25efdbe87e","_cell_guid":"739fd7c4-6753-4052-8204-a0038c866f87","trusted":true}},{"cell_type":"code","source":"X_test = np.array(df_test).reshape(-1,28,28,1)","metadata":{"execution":{"iopub.status.busy":"2024-06-13T18:06:41.659021Z","iopub.execute_input":"2024-06-13T18:06:41.659434Z","iopub.status.idle":"2024-06-13T18:06:41.731879Z","shell.execute_reply.started":"2024-06-13T18:06:41.659397Z","shell.execute_reply":"2024-06-13T18:06:41.730810Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = load_model('./best_model.h5')\ny_pred = model.predict(X_test)\ny_pred = np.argmax(y_pred,1)\npd_submit = pd.read_csv('../input/digit-recognizer/sample_submission.csv')\npd_submit['Label'] = y_pred\npd_submit.to_csv('submission.csv', header=True, index=False)","metadata":{"execution":{"iopub.status.busy":"2024-06-13T18:06:41.733226Z","iopub.execute_input":"2024-06-13T18:06:41.733566Z","iopub.status.idle":"2024-06-13T18:07:06.851453Z","shell.execute_reply.started":"2024-06-13T18:06:41.733536Z","shell.execute_reply":"2024-06-13T18:07:06.850327Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 📃 References 📃","metadata":{"_uuid":"dc3e2507-cb68-47fb-b3e6-3a0c9c7f84f7","_cell_guid":"e301b5a4-ce01-4ba9-ac3b-40983ec40265","trusted":true}},{"cell_type":"markdown","source":"
\n \n
","metadata":{"_uuid":"a9ad3adf-fc21-4ebc-9eb2-d83f80966629","_cell_guid":"f861cd0f-8e0d-48d3-95f9-029e6cf80e84","trusted":true}}]}