{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":3004,"databundleVersionId":861823,"sourceType":"competition"}],"dockerImageVersionId":30587,"isInternetEnabled":false,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Phần 1: Nhập các thư viện cần thiết\n------","metadata":{"_uuid":"b0093541-188e-43e9-acb2-d917ae3a5010","_cell_guid":"cadbfcb2-82c3-470f-bad6-eade54542125","trusted":true}},{"cell_type":"markdown","source":"
\nThe MNIST database (Modified National Institute of Standards and Technology database) is a large database of handwritten digits that is commonly used for training various image processing systems. As new machine learning techniques emerge, MNIST remains a reliable resource for researchers and learners alike.

\nThe original black and white images were normalized to fit into a 28x28 pixel bounding box and anti-aliased, which introduced grayscale levels. The original MNIST database contains 60,000 training images and 10,000 testing images, but in this competition we are given 42,000 training images (70%) and 18,000 testing images (30%).

\nThe state-of-the-art techniques to approach any Image Classification problem typically rely on Convolutional Neural Networks. This kind of architecture are really good at extracting patterns from images. However, since the MNIST is a simple problem, in this notebook we will aplly simple Machine Learning Algorithms (and we will get a decent accuracy!).

\nFirst, we will perform a Principal Component Analysis to reduce the dimension of the data (from 784 pixel values to 10 principal components). Then, we will run our favourite lazy learner algorithm: K-Nearest Neighbors. This combinatino achieves a Test Accuracy of 0.9271, which is pretty remarkable !!!

\nTable of Contents:\n\n
","metadata":{"_uuid":"85d40f0d-34d6-4e02-90f3-7ef7a60e5573","_cell_guid":"ddf0a964-5157-492a-9699-253bc2caec79","trusted":true}},{"cell_type":"code","source":"## NHẬP THƯ VIỆN PYTHON CẦN THIẾT\n# Đọc và khám phá dữ liệu\nimport numpy as np\nimport pandas as pd\nimport seaborn as sns\nimport matplotlib.pyplot as plt\nfrom sklearn.preprocessing import MinMaxScaler\n\n# Phân tích thành phần chính (PCA)\nfrom sklearn.decomposition import PCA\nfrom sklearn.pipeline import Pipeline\nimport plotly.express as px\n\n# K-Bộ phân loại hàng xóm gần nhất (KNN)\nfrom sklearn.neighbors import KNeighborsClassifier\nfrom sklearn.model_selection import GridSearchCV\nfrom sklearn.metrics import classification_report, \\\n confusion_matrix, ConfusionMatrixDisplay \n\n# VHT\nfrom scipy.ndimage import rotate","metadata":{"_uuid":"c382c9dc-2109-4ce6-b779-dc8c447da240","_cell_guid":"e169ec97-1f64-46cd-8099-1d09c757dccc","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-05-28T03:40:01.913898Z","iopub.execute_input":"2024-05-28T03:40:01.914323Z","iopub.status.idle":"2024-05-28T03:40:05.477506Z","shell.execute_reply.started":"2024-05-28T03:40:01.914287Z","shell.execute_reply":"2024-05-28T03:40:05.476049Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def func_array1d_rotate2d(Narray1d,rotate_angle,sizePixel):\n\n N = Narray1d.shape[0]\n\n Narray1d_rotate = np.zeros(Narray1d.shape)\n \n for i in range(N):\n image = np.reshape(Narray1d[i,:],(sizePixel,sizePixel))\n rotated_image = rotate(image,rotate_angle, reshape=False)\n rotated_array1d = np.reshape(rotated_image,(1,-1))\n Narray1d_rotate[i,:] = rotated_array1d\n \n return Narray1d_rotate","metadata":{"_uuid":"b362968d-9828-4257-92ec-b9e4e49a9cb0","_cell_guid":"8ace7936-3ee9-4fe9-b71e-8d7e3b7eee15","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-05-28T03:40:05.479769Z","iopub.execute_input":"2024-05-28T03:40:05.480497Z","iopub.status.idle":"2024-05-28T03:40:05.488842Z","shell.execute_reply.started":"2024-05-28T03:40:05.480456Z","shell.execute_reply":"2024-05-28T03:40:05.487504Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"----\n# Phần 2: Đọc dữ liệu và khai thác dữ liệu","metadata":{"_uuid":"a7abf7cf-fbfb-44f8-a5eb-77885d1283fb","_cell_guid":"1e5c33e8-406f-4da5-929d-034f87443736","trusted":true}},{"cell_type":"markdown","source":"Trong phần này, chúng tôi sẽ sử dụng thư viện Pandas Python nổi tiếng để đọc dữ liệu từ các tệp .csv.\n\nChúng ta sẽ trực quan hóa việc phân phối các giá trị pixel (biểu đồ) bằng thư viện Seaborn và Matplotlib Python. Chúng tôi cũng sẽ kiểm tra xem không có giá trị nào bị thiếu và các lớp mục tiêu có cân bằng hay không.\n\nCuối cùng, chúng ta sẽ trực quan hóa các hình ảnh bằng cách vẽ một hình ảnh từ mỗi lớp.","metadata":{"_uuid":"d4c82fac-2a10-4c14-a95a-653bd16872cd","_cell_guid":"deaebcb3-115c-4e6b-b843-4522382a052d","trusted":true}},{"cell_type":"code","source":"# Đọc file dữ liệu bằng Pandas\ndf_train = pd.read_csv('/kaggle/input/digit-recognizer/train.csv')\ndf_test = pd.read_csv('/kaggle/input/digit-recognizer/test.csv')","metadata":{"_uuid":"4da57b46-cc50-4ae9-9495-d0dbeb6be5ee","_cell_guid":"8967621a-0c55-49ff-b6e2-05d9efdeedf2","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-05-28T03:40:05.491265Z","iopub.execute_input":"2024-05-28T03:40:05.492741Z","iopub.status.idle":"2024-05-28T03:40:12.890808Z","shell.execute_reply.started":"2024-05-28T03:40:05.492678Z","shell.execute_reply":"2024-05-28T03:40:12.889704Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Kiểm tra kích thước của tàu và kiểm tra DataFrames\nprint(\"df_train shape:\", df_train.shape) # 70%\nprint(\"df_test shape: \", df_test.shape) # 30%","metadata":{"_uuid":"5465a268-9ec4-4482-a0d7-e531d25f4917","_cell_guid":"da0e04b7-5ce3-48ec-be64-cee70c71af07","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-05-28T03:40:12.893787Z","iopub.execute_input":"2024-05-28T03:40:12.894233Z","iopub.status.idle":"2024-05-28T03:40:12.901946Z","shell.execute_reply.started":"2024-05-28T03:40:12.894195Z","shell.execute_reply":"2024-05-28T03:40:12.899845Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# May mắn thay không có giá trị nào bị thiếu trong dữ liệu của chúng tôi\nprint(\n \"Missing values in df_train: \", \\\n df_train.isna().sum().sum()\n)\nprint(\n \"Missing values in df_test: \", \\\n df_test.isna().sum().sum()\n)","metadata":{"_uuid":"f8324800-d007-4d21-9eb6-39bba50ce548","_cell_guid":"758f94a2-69c7-4c9b-b1a3-22cb0f0b1a43","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-05-28T03:40:12.903773Z","iopub.execute_input":"2024-05-28T03:40:12.904311Z","iopub.status.idle":"2024-05-28T03:40:12.968654Z","shell.execute_reply.started":"2024-05-28T03:40:12.904266Z","shell.execute_reply":"2024-05-28T03:40:12.967502Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Cùng nhìn lại năm hàng huấn luyện đầu tiên\ndf_train.head()","metadata":{"_uuid":"b1a7f2dc-6cec-4610-ba22-6ad5a7fb06ac","_cell_guid":"f1a8c252-31ba-46a1-b25c-c6ffff3fa2a9","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-05-28T03:40:12.970408Z","iopub.execute_input":"2024-05-28T03:40:12.970803Z","iopub.status.idle":"2024-05-28T03:40:13.005366Z","shell.execute_reply.started":"2024-05-28T03:40:12.970769Z","shell.execute_reply":"2024-05-28T03:40:13.004212Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Chúng ta có thể thấy cột đầu tiên là nhãn,\n# và các cột còn lại là 784 pixel\n# đại diện cho hình ảnh thang độ xám 28x28.\n# Hãy tách X và Y\nX_train = df_train.drop(columns=['label']).values\ny_train = df_train['label'].values","metadata":{"_uuid":"48085649-c225-4e39-824f-82474ea004ff","_cell_guid":"939c1008-c85e-4ee0-bdab-bcccc4a252e4","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-05-28T03:40:13.006652Z","iopub.execute_input":"2024-05-28T03:40:13.006995Z","iopub.status.idle":"2024-05-28T03:40:13.134824Z","shell.execute_reply.started":"2024-05-28T03:40:13.006964Z","shell.execute_reply":"2024-05-28T03:40:13.133027Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Hãy trực quan hóa việc phân phối các giá trị pixel\n\nax = sns.histplot(x=X_train.flatten(), color='#4895EE')\n_ = ax.set(title='Pixel Value Histogram', xlabel='Pixel Value')\n\n# Như chúng ta có thể thấy, giá trị 0 đại diện cho nền giá trị \n# và >0 đại diện cho các chữ số trong hình ảnh","metadata":{"_uuid":"f724afe4-d098-4f65-aaca-e97a096b8a32","_cell_guid":"36ec9986-72eb-4f04-abf8-84e2df3e3bfa","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-05-28T03:40:13.137499Z","iopub.execute_input":"2024-05-28T03:40:13.138028Z","iopub.status.idle":"2024-05-28T03:40:31.818193Z","shell.execute_reply.started":"2024-05-28T03:40:13.137988Z","shell.execute_reply":"2024-05-28T03:40:31.816966Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Hãy kiểm tra xem mục tiêu có cân bằng không.\n# Nó có nghĩa là mười danh mục trong mục tiêu\n# có sự biểu diễn gần giống nhau trong dữ liệu.\n\nax = sns.countplot(x=y_train, color='#4895EE')\nax.set(\n title='Target distribution is balanced? Yes!',\n xlabel='Label'\n)\n# Số lượng mẫu dự kiến cho cân \n# phân bổ lớp là 42000/10 = 4200\n_ = ax.axhline(y=4200, color='#12086F', linestyle='--')\n\n# Vì các lớp mục tiêu là số dư nên chúng ta không có\n# lo lắng về các phương pháp lấy mẫu và thành kiến trong các mô hình của chúng tôi.","metadata":{"_uuid":"cd488e69-2d9d-4209-ac52-f11e13ab443b","_cell_guid":"d681064d-2dbe-4c2b-8075-9ce2c254d755","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-05-28T03:40:31.819875Z","iopub.execute_input":"2024-05-28T03:40:31.820358Z","iopub.status.idle":"2024-05-28T03:40:32.183475Z","shell.execute_reply.started":"2024-05-28T03:40:31.820315Z","shell.execute_reply":"2024-05-28T03:40:32.182319Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Tiếp theo, hãy hình dung một hình ảnh từ mỗi lớp.\n# Chúng ta sẽ sử dụng hàm 'imshow' của matplotlib.\n\nfig, axs = plt.subplots(nrows=2, ncols=5, figsize=(10,5))\n\nfor i in range(10):\n idx = np.where(y_train==i)[0][2]\n img = axs[int(i >= 5), i%5].imshow(\n X=X_train[idx].reshape(28, 28),\n cmap='Blues'\n )\n img.axes.get_yaxis().set_visible(False)\n img.axes.get_xaxis().set_visible(False)\n img.axes.set_title(f'Label: {i}')","metadata":{"_uuid":"9d6c909c-609f-4b5d-a907-2d75a3c17401","_cell_guid":"1fa8f38f-f989-4007-86c5-590612877157","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-05-28T03:40:32.189062Z","iopub.execute_input":"2024-05-28T03:40:32.189519Z","iopub.status.idle":"2024-05-28T03:40:32.941892Z","shell.execute_reply.started":"2024-05-28T03:40:32.189482Z","shell.execute_reply":"2024-05-28T03:40:32.940550Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":" transform data","metadata":{"_uuid":"dd9d9d94-7ea8-4f8d-aa91-866959d703e3","_cell_guid":"48cf8b51-1a3e-4d0e-94e5-da53b3aa7c90","trusted":true}},{"cell_type":"code","source":"# -------------------------------------------------- declare\nnumRotate_prior = 10\n\nangleOffset_prior = -20\nangleRange_prior = 40\n\nX_train_pick2rotate = {\n 'index': np.array([3600, 3601, 3602, 3603, 3604, 3605, 3606, 3607, 3608, 3609, 3610,\n3611, 3612, 3613, 3614, 3615, 3616, 3617, 3618, 3619, 3620, 3621,\n3622, 3623, 3624, 3625, 3626, 3627, 3628, 3629, 3630, 3631, 3632,\n3633, 3634, 3635, 3636, 3637, 3638, 3639, 3640, 3641, 3642, 3643,\n3644, 3645, 3646, 3647, 3648, 3649, 3650, 3651, 3652, 3653, 3654,\n3655, 3656, 3657, 3658, 3659, 3660, 3661, 3662, 3663, 3664, 3665,\n3666, 3667, 3668, 3669, 3670, 3671, 3672, 3673, 3674, 3675, 3676,\n3677, 3678, 3679, 3680, 3681, 3682, 3683, 3684, 3685, 3686, 3687,\n3688, 3689, 3690, 3691, 3692, 3693, 3694, 3695, 3696, 3697, 3698, 3699]),\n 'offset': np.array([\n #3600\n [60,10,0,-5,20, \n 0,0,20,-5,0],\n #3601\n [50,20,10,-10,0, \n 20,-5,0,10,0],\n #3602\n [50,0,-5,-5,15, \n 0,0,0,5,25],\n #3603\n [30,15,-5,5 ,10, \n 30 ,10 ,0,0,0],\n #3604\n [20 ,0,5,-10,0, \n 10,-30,0,5,15],\n #3605\n [20,-20,-10,10,10, \n 0,-10,0,-10,-10],\n #3606\n [10,0,0,0,20, \n 0,-40,0,5,0],\n #3607\n [25,20,0,0,15, \n 15,-50,5,0,0],\n #3608\n [10,24,0,0,7, \n 0,0,-12,5,10],\n #3609\n [20,20,-5,0,10, \n 7,-16,0,10,0],\n #3610\n [20,10,0,0,15, \n 15,-10,0,20,0],\n #3611\n [0,25,0,0,10, \n 0,-5,0,-20,10],\n #3612\n [20,15,0,-10,-3, \n 10,20,0,5,10],\n #3613\n [20,25,0,10,15, \n 15,-5,0,25,10],\n #3614\n [25,30,-10,0,0, \n 15,10,0,10,0],\n #3615\n [30,5,-5,-5,5, \n 0,-30,10,-5,10],\n #3616\n [10,-5,10,10,25, \n 0,0,0,20,0],\n #3617\n [10,15,-5,0,5, \n 20,0,15,10,-10],\n #3618\n [10,5,0,10,15, \n 20,10,0,0,10],\n #3619\n [10,10,0,-10,0, \n 10,-20,0,0,0],\n #3620\n [20,10,0,0,5, \n 0,20,0,15,20],\n #3621\n [20,13,0,11,15, \n 10,-10,15,5,10],\n #3622\n [20,10,0,0,-5, \n 5,0,-7,-5,0],\n #3623\n [5,4,0,-7,10, \n 2,0,10,5,3],\n #3624\n [40,0,-5,6,0, \n 0,0,0,10,5],\n #3625\n [30,30,0,-7,10, \n 7,-25,-7,8,0],\n #3626\n [0,20,0,-15,5, \n 10,-30,0,5,5],\n #3627\n [30,10,-4,-5,0, \n 5,8,4,5,4],\n #3628\n [10,25,0,0,20, \n 15,-5,12,10,-5],\n #3629\n [30,15,0,13,10, \n 15,-10,7,0,-6],\n #3630\n [0,35,0,10,0, \n 27,-15,0,5,0],\n #3631\n [20,10,0,0,20, \n 10,-20,-10,10,30,],\n #3632\n [10,15,0,0,10, \n 20,10,0,20,0,],\n #3633\n [-10,0,0,0,15, \n 10,0,10,0,-10,],\n #3634\n [30,5,0,-10,5, \n 0,25,0,0,30,],\n #3635\n [0,10,10,-10,33, \n 10,0,0,10,10,],\n #3636\n [10,-15,10,0,10, \n 10,0,5,15,10,],\n #3637\n [50,2,4,0,5, \n 10,10,0,5,0,],\n #3638\n [30,10,-10,10,-15, \n 10,-20,-8,20,10,],\n #3639\n [5,5,10,0,25, \n 0,-30,0,-10,0,],\n #3640\n [35,30,0,20,10, \n 10,0,5,8,15,],\n #3641\n [0,15,-10,0,15, \n 20,-35,0,3,10,],\n #3642\n [-15,20,-15,-7,15, \n -12,-30,15,15,25,],\n #3643\n [-5,10,-10,10,4, \n 0,-30,-7,-5,5,],\n #3644\n [-10,18,0,-3,15, \n 17,-25,0,5,25,],\n #3645\n [10,35,0,-10,10, \n 0,-30,-10,15,15,],\n #3646\n [10,45,20,13,20, \n 0,0,0,-5,-20,],\n #3647\n [10,32,0,0,-10, \n -5,10,0,10,0,],\n #3648\n [20,20,5,5,10, \n 5,-25,-5,-10,10,],\n #3649\n [20,-8,-5,-5,10, \n 15,-5,7,13,-8,],\n #3650\n [30,15,-13,0,0, \n 10,0,13,0,10,],\n #3651\n [0,5,-3,0,5, \n 25,-30,-10,0,30,],\n #3652\n [0,20,10,0,25, \n 14,-30,0,5,14,],\n #3653\n [13,10,0,0,20, \n 7,-4,0,10,15,],\n #3654\n [25,15,-5,7,5, \n 0,-5,-4,0,30],\n #3655\n [10,10,20,10,0, \n 0,0,0,-7,10,],\n #3656\n [10,25,0,-5,20, \n 20,0,-30,10,25,],\n #3657\n [0,10,10,-5,0, \n 5,-30,0,20,0,],\n #3658\n [25,0,-5,0,-10, \n 20,0,0,0,10,],\n #3659\n [15,5,0,17,16, \n 20,-20,0,0,20,],\n #3660\n [-20,35,0,-5,8, \n 10,0,0,6,10,],\n #3661\n [20,10,-5,15,15, \n -3,0,0,-4,10,],\n #3662\n [4,10,10,10,20, \n 0,10,0,17,0,],\n #3663\n [20,30,10,-10,0, \n 10,-5,0,-10,20,],\n #3664\n [20,20,0,0,20, \n 10,-5,0,0,15,],\n #3665\n [20,10,0,0,0, \n 0,0,0,-5,10,],\n #3666\n [0,30,10,-10,5, \n 4,5,0,10,0,],\n #3667\n [5,25,0,10,10, \n 0,0,5,5,0,],\n #3668\n [13,2,-3,0,20, \n 10,5,5,10,0,],\n #3669\n [0,7,0,5,15, \n 0,25,-5,5,10,],\n #3670\n [20,30,-5,0,20, \n 13,0,0,5,-3,],\n #3671\n [20,25,2,-4,-10, \n 0,0,0,5,0,],\n #3672\n [20,15,8,0,10, \n 7,5,-3,0,5,],\n #3673\n [10,0,20,5,15, \n 0,0,0,5,15,],\n #3674\n [15,0,0,3,10, \n 10,-10,0,5,0,],\n #3675\n [20,20,3,0,30, \n 0,-20,10,10,-5,],\n #3676\n [25,20,4,0,10, \n -10,-50,3,20,20,],\n #3677\n [20,2,6,0,-3, \n 15,0,0,-5,20,],\n #3678\n [55,20,0,5,8, \n 0,0,0,5,5,],\n #3679\n [7,4,5,-5,15, \n -5,-20,0,5,10,],\n #3680\n [-5,3,0,2,22, \n 7,-10,-59,2,15,],\n #3681\n [20,5,5,3,-10, \n 5,-6,0,-5,10,],\n #3682\n [20,25,0,5,10, \n 7,-10,0,20,10,],\n #3683\n [30,0,10,0,10, \n 0,0,0,5,0,],\n #3684\n [10,0,-7,10,10, \n 10,-10,10,5,0,],\n #3685\n [10,0,0,0,-5, \n 10,15,15,10,5,],\n #3686\n [40,20,-10,0,5, \n 10,0,0,0,0,],\n #3687\n [0,4,0,0,10, \n 0,10,0,10,-8,],\n #3688\n [30,5,0,0,6, \n 4,0,0,3,15,],\n #3689\n [20,13,-5,3,15, \n 10,0,0,10,-10,],\n #3690\n [20,5,0,0,20, \n 4,0,0,4,15,],\n #3691\n [15,20,5,-3,0, \n 15,0,0,-5,0,],\n #3692\n [0,23,-5,-5,20, \n 30,-10,-7,5,5,],\n #3693\n [20,25,10,0,7, \n -30,50,-7,8,9,],\n #3694\n [25,15,-10,0,25, \n 27,-50,0,8,-15,],\n #3695\n [30,-8,0,4,4, \n -15,0,0,0,20,],\n #3696\n [20,15,0,0,0, \n 0,0,0,10,0,],\n #3697\n [20,-5,-5,-5,5, \n 10,0,0,0,5,],\n #3698\n [20,-7,0,0,0, \n 0,0,7,0,0,],\n #3699\n [10,15,-3,0,0, \n 0,0,-5,0,0,],\n ])\n}\nnumPickRotate = len(X_train_pick2rotate['index'])\n\nprint('numPickRotate = ', numPickRotate)\n\n# -------------------------------------------------- rotate\n\nX_sample_rotate = np.zeros((10,numRotate_prior,numPickRotate,X_train.shape[1]))\ny_sample_rotate = np.zeros((10,numRotate_prior,numPickRotate))\n\nfor k0 in range(10):\n for k1 in range(numRotate_prior):\n for k2 in range(numPickRotate):\n \n idxPick_X_train = X_train_pick2rotate['index'][k2]\n \n idx = np.where(y_train==k0)[0][idxPick_X_train]\n \n y_sample_rotate[k0,k1,k2] = k0 * numRotate_prior + k1\n \n angleRotate = (k1/numRotate_prior * angleRange_prior) + angleOffset_prior + X_train_pick2rotate['offset'][k2][k0]\n \n X_sample_rotate[k0,k1,k2,:] = func_array1d_rotate2d(X_train[idx][np.newaxis,:],angleRotate, 28)\n\n \nX_sample_rotate[X_sample_rotate < 10] = 0\n#X_sample_rotate[X_sample_rotate > 300] = 0\n\n# Hãy chuẩn hóa các giá trị pixel thành khoảng [0,1]\n# Công thức: MinMaxscale(X) = (X - X_min) / (X_max - X_min)\n\nprint(\"Before applying MinMax Scaler:\")\nprint(\" - Minimum value:\", X_sample_rotate.min())\nprint(\" - Maximum value:\", X_sample_rotate.max())\n\nminmax_scaler = MinMaxScaler()\n\nfor k0 in range(10):\n for k1 in range(numRotate_prior):\n X_sample_rotate[k0,k1,:,:] = minmax_scaler.fit_transform(X_sample_rotate[k0,k1,:,:]) * 255\n\nprint(\"\\nAfter applying MinMax Scaler:\")\nprint(\" - Minimum value:\", X_sample_rotate.min())\nprint(\" - Maximum value:\", X_sample_rotate.max())\n\n\n\n# Tiếp theo, hãy hình dung một hình ảnh từ mỗi lớp.\n# Chúng ta sẽ sử dụng hàm 'imshow' của matplotlib.\n\n# for k2 in range(numPickRotate):\nfor k2 in range(5,10):\n k1 = 0\n # k1 = numRotate_prior - 1\n\n fig, axs = plt.subplots(nrows=2, ncols=5, figsize=(10,5))\n\n for k0 in range(10):\n img = axs[int(k0 >= 5), k0%5].imshow(\n X=X_sample_rotate[k0,k1,k2,:].reshape(28, 28),\n cmap='Blues'\n )\n img.axes.get_yaxis().set_visible(False)\n img.axes.get_xaxis().set_visible(False)\n img.axes.set_title(f'Label: {k0}')\n \nX_train_rotate = np.reshape(X_sample_rotate, (-1,784))\ny_train_rotate = np.reshape(y_sample_rotate, (-1,1)).flatten()","metadata":{"_uuid":"2fd8cd6b-8db8-45dc-b5f7-6721c80f11fd","_cell_guid":"6b163e1f-7fee-43a6-b6f6-40c0d402a4fc","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-05-28T03:40:32.944246Z","iopub.execute_input":"2024-05-28T03:40:32.944820Z","iopub.status.idle":"2024-05-28T03:40:40.082387Z","shell.execute_reply.started":"2024-05-28T03:40:32.944772Z","shell.execute_reply":"2024-05-28T03:40:40.081229Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---------\n# Phần 3: Phân tích thành phần chính","metadata":{"_uuid":"f67ae6c7-11f8-4b88-a23e-bf62fb9fdb43","_cell_guid":"6e9093b4-f586-4545-b99e-b59d492b7958","trusted":true}},{"cell_type":"markdown","source":"
\n Principal Comopnent Analysis (PCA) is a statistical technique for reducing the dimensionality of a dataset. This is accomplished by linearly transforming the data into a new coordinate system where (most of) the variation in the data can be described with fewer dimensions than the initial data.

\nThe first principal component can equivalently be defined as a direction that maximizes the variance of the projected data. The i-th principal component can be taken as a direction orthogonal to the first i-1 principal components that maximizes the variance of the projected data.

\nsklearn.decomposition.PCA: Linear dimensionality reduction using Singular Value Decomposition of the data to project it to a lower dimensional space.

\nHere we will use the top 10 principal components, which explain almost 50% of the variance in the data. This means that we can express 50% of the variance of the data with a space of features >98% smaller than the original one (10 features vs. 784 features)\n
","metadata":{"_uuid":"c9145741-6e7a-4419-bdb0-baca414ed4e7","_cell_guid":"54390ba7-75cb-4867-b577-97f23c85d6f2","trusted":true}},{"cell_type":"code","source":"# Hãy chuẩn hóa các giá trị pixel thành khoảng [0,1]\n# Công thức: MinMaxscale(X) = (X - X_min) / (X_max - X_min)\nprint(\"Before applying MinMax Scaler:\")\nprint(\" - Minimum value:\", X_train.min())\nprint(\" - Maximum value:\", X_train.max())\n\nminmax_scaler = MinMaxScaler()\nX_train_rotate_scaled = minmax_scaler.fit_transform(X_train_rotate)\n\nprint(\"\\nAfter applying MinMax Scaler:\")\nprint(\" - Minimum value:\", X_train_rotate_scaled.min())\nprint(\" - Maximum value:\", X_train_rotate_scaled.max())","metadata":{"_uuid":"aad53bf6-9b1f-4451-844d-ae51f509a68f","_cell_guid":"2f528db2-3b92-425b-a18c-2168dd8862d7","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-05-28T03:40:40.084408Z","iopub.execute_input":"2024-05-28T03:40:40.084909Z","iopub.status.idle":"2024-05-28T03:40:40.236125Z","shell.execute_reply.started":"2024-05-28T03:40:40.084862Z","shell.execute_reply":"2024-05-28T03:40:40.234820Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Trên thực tế, chúng tôi sẽ nhúng MinMaxScaler và PCA\n# mô-đun vào Đường ống Sklearn.\n\npca = Pipeline(steps=[\n ('scaler', MinMaxScaler()),\n ('pca', PCA(n_components=40))\n])\n\nX_train_rotate_pca = pca.fit_transform(X=X_train_rotate)\n\nprint('X_train_pca shape:', X_train_rotate_pca.shape)","metadata":{"_uuid":"6cae5170-3af3-4ed9-a9d8-372ef224e9ba","_cell_guid":"e423b865-ef1d-4c48-9152-917f8b258060","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-05-28T03:40:40.237685Z","iopub.execute_input":"2024-05-28T03:40:40.238100Z","iopub.status.idle":"2024-05-28T03:40:41.941520Z","shell.execute_reply.started":"2024-05-28T03:40:40.238062Z","shell.execute_reply":"2024-05-28T03:40:41.939503Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Hãy trích xuất tỷ lệ phương sai được giải thích\n# Và hãy tính tổng tích lũy của chúng\n\npca_object = pca.named_steps['pca']\nvar_ratio = pd.Series(\n data=pca_object.explained_variance_ratio_,\n index=range(1, 41)\n)\n\ncumsum_var_ratio = var_ratio.cumsum()","metadata":{"_uuid":"68b1ffab-f2de-43fe-a655-e04c6060ec60","_cell_guid":"c3cdf97b-6136-45e0-94c6-bdf23e750bf0","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-05-28T03:40:41.952773Z","iopub.execute_input":"2024-05-28T03:40:41.960013Z","iopub.status.idle":"2024-05-28T03:40:41.984640Z","shell.execute_reply.started":"2024-05-28T03:40:41.959875Z","shell.execute_reply":"2024-05-28T03:40:41.982503Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Vẽ biểu đồ tỷ lệ phương sai giải thích cho từng thành phần (trái)\n# Và vẽ tổng tích lũy của chúng (phải)\nfig, (ax1, ax2) = plt.subplots(1, 2, figsize=(15,5))\n\nsns.barplot(\n x=var_ratio.index, \n y=var_ratio.values, \n color='#4895EE', \n ax=ax1\n)\nax1.set(\n title='Explained Variance Ratio PCA',\n xlabel='Component',\n ylabel='Var Ratio',\n)\nax1.set_ylim(0, 0.5)\n\nsns.barplot(\n x=cumsum_var_ratio.index, \n y=cumsum_var_ratio.values, \n color='#4895EE', \n ax=ax2\n)\nax2.set(\n title='Cumulative Explained Variance Ratio PCA',\n xlabel='Component',\n ylabel='Cumulative Var Ratio',\n)\n_ = ax2.set_ylim(0, 0.5)","metadata":{"_uuid":"9ea36679-af0c-40f1-bda2-641ef9c73a4e","_cell_guid":"6fe44c61-1ac9-4157-968d-022d42184ff6","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-05-28T03:40:41.989243Z","iopub.execute_input":"2024-05-28T03:40:41.989767Z","iopub.status.idle":"2024-05-28T03:40:43.605098Z","shell.execute_reply.started":"2024-05-28T03:40:41.989726Z","shell.execute_reply":"2024-05-28T03:40:43.603757Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Hãy đặt các thành phần ở định dạng Pandas DataFrame\n# Bằng cách này chúng ta có thể sử dụng thư viện seaborn và pploly\n# để trực quan hóa dữ liệu.\n# (các gói này hoạt động tốt hơn với Pandas DataFrames)\n\ndf_train_pca = pd.DataFrame(\n data=X_train_rotate_pca[:,:3], \n columns=['x', 'y', 'z']\n)\n# Hãy thêm mục tiêu để đặt màu\ndf_train_pca['target'] = y_train_rotate\n\nprint(df_train_pca.shape)\ndf_train_pca.head()","metadata":{"_uuid":"9796e066-c6af-49c5-af8f-8ebde5e07430","_cell_guid":"4fb22220-cc87-410b-a637-55bc57791987","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-05-28T03:40:43.606796Z","iopub.execute_input":"2024-05-28T03:40:43.607164Z","iopub.status.idle":"2024-05-28T03:40:43.626553Z","shell.execute_reply.started":"2024-05-28T03:40:43.607118Z","shell.execute_reply":"2024-05-28T03:40:43.625027Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Trực quan hóa hai chiều của dữ liệu\n# Ở đây các thành phần chính của Top2 được vẽ\n\nsns.set_style(\"dark\")\nax = sns.scatterplot(\n data=df_train_pca,\n x='x', \n y='y', \n hue='target', \n palette='Blues',\n)\nax.set(\n title='PCA Two-Dimensional Projection',\n xlabel='1st Comp.',\n ylabel='2nd Comp.'\n)\nax.axhline(y=0, linestyle='--', color='#FFBF00', zorder=-1)\nax.axvline(x=0, linestyle='--', color='#FFBF00', zorder=-1)","metadata":{"_uuid":"30135dee-d7d0-4dc8-88f7-3a88a135ad5a","_cell_guid":"147fd5ee-e632-4611-b6de-35d64c6946e9","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-05-28T03:40:43.628060Z","iopub.execute_input":"2024-05-28T03:40:43.628609Z","iopub.status.idle":"2024-05-28T03:40:45.955556Z","shell.execute_reply.started":"2024-05-28T03:40:43.628560Z","shell.execute_reply":"2024-05-28T03:40:45.954172Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Bây giờ, hãy áp dụng biến đổi nghịch đảo để tính toán\n# sự gần đúng của hình ảnh dựa trên nguyên tắc 10\n#thành phần.\n# Chúng ta có thể thấy rằng chỉ với 10 thành phần chúng ta đã truyền tải được một lượng lớn\n# một phần thông tin của mỗi hình ảnh.\n\npca_object = pca.named_steps['pca']\n\nfig, axs = plt.subplots(nrows=10, ncols=2, figsize=(6,20))\n\nfor i in range(10):\n idx = np.where(y_train==i)[0][0]\n original_img = axs[i, 0].imshow(\n X=X_train_rotate[idx].reshape(28, 28),\n cmap='Blues'\n )\n original_img.axes.get_yaxis().set_visible(False)\n original_img.axes.get_xaxis().set_visible(False)\n original_img.axes.set_title(f'Original [{i}]:')\n \n approx_img_ = pca_object.inverse_transform(X=X_train_rotate_pca[idx])\n approx_img = axs[i, 1].imshow(\n X=approx_img_.reshape(28, 28),\n cmap='Blues'\n )\n approx_img.axes.get_yaxis().set_visible(False)\n approx_img.axes.get_xaxis().set_visible(False)\n approx_img.axes.set_title(f'Approximation [{i}]:')","metadata":{"_uuid":"77146579-275e-4359-8ec9-7d175da30220","_cell_guid":"329c95b3-2406-402c-b1f8-38487c4fa2f3","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-05-28T03:40:45.956931Z","iopub.execute_input":"2024-05-28T03:40:45.957349Z","iopub.status.idle":"2024-05-28T03:40:47.197044Z","shell.execute_reply.started":"2024-05-28T03:40:45.957312Z","shell.execute_reply":"2024-05-28T03:40:47.194967Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Chứng tỏ các thành phần PCA không tương quan\n# (Câu nói này đúng về mặt định nghĩa và cách xây dựng\n# của thuật toán Phân tích thành phần chính)\n\ncorr = np.corrcoef(x=X_train_rotate_pca, rowvar=False)\ncorr = np.around(corr, decimals=2)\n\nax = sns.heatmap(\n data=corr, \n vmax=1, \n vmin=-1,\n square=True,\n linewidths=.5,\n annot=True,\n cmap='Blues',\n)\n\n_ = ax.set(\n title='Correlation PCA Components [Uncorrelated]',\n xlabel='Components',\n ylabel='Components'\n)","metadata":{"_uuid":"8c3fa8bb-61ab-43dc-bc7e-e5fecc13e6e2","_cell_guid":"ab321270-3628-423b-8fd6-a7ebaf6ccfe5","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-05-28T03:40:47.198823Z","iopub.execute_input":"2024-05-28T03:40:47.199268Z","iopub.status.idle":"2024-05-28T03:40:52.332924Z","shell.execute_reply.started":"2024-05-28T03:40:47.199231Z","shell.execute_reply":"2024-05-28T03:40:52.331246Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"----\n# Phần 4: Sử dụng K-Nearest Neighbors Classifier phân loại 10 thành phần chính hàng đầu từ PCA.","metadata":{"_uuid":"3d69b1a4-faf3-4405-b740-9d5fc82b8424","_cell_guid":"16cefed3-22ad-4c84-ad87-230ed47df34d","trusted":true}},{"cell_type":"markdown","source":"Thuật toán K-Nearest Neighbors là một thuật toán phân loại học tập có giám sát, không tham số, sử dụng độ gần để phân loại hoặc dự đoán về việc nhóm một điểm dữ liệu riêng lẻ. Dự đoán: Nhãn lớp được chỉ định trên cơ sở đa số phiếu bầu. nhãn được thể hiện thường xuyên nhất xung quanh một điểm dữ liệu nhất định.\n\nTrong dự án này, đầu vào của Bộ phân loại KNN sẽ là 10 thành phần chính hàng đầu từ PCA.\n\nsklearn.neighbors.KNeighborsClassifier: Trình phân loại thực hiện bỏ phiếu k-hàng xóm gần nhất. Chúng tôi sẽ chạy GridSearchCV để tối ưu hóa siêu tham số n_neighbors.","metadata":{"_uuid":"1abd82c5-f4e1-4142-acb2-bea45e42488e","_cell_guid":"b9e0603d-4dc8-4761-baad-a34cf6709975","trusted":true}},{"cell_type":"markdown","source":"--- mô hình học máy trên tập dữ liệu Train","metadata":{"_uuid":"334a6d45-a6b2-499e-a95b-2477800b968f","_cell_guid":"e1b5a3d3-ced9-409c-b6f9-b07e7143bc66","trusted":true}},{"cell_type":"code","source":"knn_clf = Pipeline(steps=[\n ('scaler', MinMaxScaler()),\n ('pca', PCA(n_components=10)),\n ('knn', KNeighborsClassifier())\n])\n\nknn_clf_params = {\n 'knn__n_neighbors': [2, 4, 8, 16, 32]\n}\n\nknn_clf_gs = GridSearchCV(\n estimator=knn_clf, \n param_grid=knn_clf_params,\n scoring='accuracy',\n n_jobs=-1,\n cv=5,\n verbose=1,\n return_train_score=True\n)\n\nknn_clf_gs.fit(X=X_train_rotate_pca, y=y_train_rotate)\n\nprint('best PCA+KNN params:', knn_clf_gs.best_params_)\nprint('best PCA+KNN cv accuracy:', knn_clf_gs.best_score_)","metadata":{"_uuid":"7154a244-1406-4e55-9062-2a876ffcfb42","_cell_guid":"f8b23485-498b-446c-83a4-fdf1e50b8285","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-05-28T03:40:52.334840Z","iopub.execute_input":"2024-05-28T03:40:52.335345Z","iopub.status.idle":"2024-05-28T03:41:11.980403Z","shell.execute_reply.started":"2024-05-28T03:40:52.335298Z","shell.execute_reply":"2024-05-28T03:41:11.978774Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Vẽ sơ đồ độ chính xác trong đào tạo và xác nhận\n# cho mỗi giá trị Siêu tham số được kiểm tra trong quá trình\n# Tìm kiếm lưới.\n# Biểu đồ này cho phép chúng ta kiểm tra xem có trang bị quá mức hay không.\nfig, ax = plt.subplots()\n\nax.plot(\n range(len(knn_clf_params['knn__n_neighbors'])), \n knn_clf_gs.cv_results_['mean_train_score'],\n label='train',\n color='blue'\n)\nax.plot(\n range(len(knn_clf_params['knn__n_neighbors'])),\n knn_clf_gs.cv_results_['mean_test_score'],\n label='test',\n color='orange'\n)\nax.set_xticks(\n ticks=range(len(knn_clf_params['knn__n_neighbors'])),\n labels=knn_clf_params['knn__n_neighbors']\n)\n\nii = knn_clf_params['knn__n_neighbors'].index(\n knn_clf_gs.best_params_['knn__n_neighbors']\n)\n\nplt.axvline(x=ii, linestyle='--', color='red', label='best', alpha=0.3)\nplt.legend()\nplt.xlabel(\"'n_neighbors' vals\")\nplt.ylabel('Accuracy')\nplt.title(\"PCA+KNN GridSearchCV Results\")\nplt.show()","metadata":{"_uuid":"43e1d701-a89a-463f-b860-3afbf14bbf36","_cell_guid":"b1316942-7e1c-4f06-8742-d3eb8f7918c3","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-05-28T03:41:11.982402Z","iopub.execute_input":"2024-05-28T03:41:11.982877Z","iopub.status.idle":"2024-05-28T03:41:12.412038Z","shell.execute_reply.started":"2024-05-28T03:41:11.982824Z","shell.execute_reply":"2024-05-28T03:41:12.410773Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n# Phần 5: Tạo một tệp tin chứa các dự đoán của mô hình học máy trên tập dữ liệu Test","metadata":{"_uuid":"b609f283-8a5c-477c-9c29-941acfedc9aa","_cell_guid":"5d9eb764-a90a-48b5-b962-be1b347e4925","trusted":true}},{"cell_type":"code","source":"from sklearn.metrics import accuracy_score\n\n# Tính toán và hiển thị Ma trận nhầm lẫn\n# Lưu ý: toàn bộ tập dữ liệu huấn luyện được sử dụng ở đây\nX_val = X_train_rotate_pca\nY_val = y_train_rotate\n\nbest_cv_clf = knn_clf_gs.best_estimator_\ny_pred = best_cv_clf.predict(X=X_val)\n\ny_val_number = Y_val // numRotate_prior\ny_pred_number = y_pred // numRotate_prior\n\ny_val_angle = (Y_val % numRotate_prior)\ny_pred_angle = (y_pred % numRotate_prior)\n\nprint('KNN number + angle, Acc({:.2f}%)', accuracy_score(Y_val, y_pred))\nprint('KNN number, Acc({:.2f}%)', accuracy_score(y_val_number, y_pred_number))\nprint('KNN angle, Acc({:.2f}%)', accuracy_score(y_val_angle, y_pred_angle))\n\n# print('y_val_number =', y_val_number)\n# print('y_pred_number =', y_pred_number)\n\ny_val_angle_degree = y_val_angle/numRotate_prior * angleRange_prior + angleOffset_prior\ny_pred_angle_degree = y_pred_angle/numRotate_prior * angleRange_prior + angleOffset_prior\n\n# print('y_val_angle_degree =', y_val_angle_degree)\n# print('y_pred_angle_degree =', y_pred_angle_degree)\n\nconf_matrix = confusion_matrix(y_val_number, y_pred_number)\nconf_matrix_disp = ConfusionMatrixDisplay(conf_matrix)\nconf_matrix_disp.plot(cmap='Blues')\n\nconf_matrix = confusion_matrix(y_val_angle, y_pred_angle)\nconf_matrix_disp = ConfusionMatrixDisplay(conf_matrix)\nconf_matrix_disp.plot(cmap='Blues')","metadata":{"_uuid":"62d7493a-d5c0-42ca-ad56-df419c3367ee","_cell_guid":"fdfdc4ce-7aa6-4c8a-b7c9-8a069cb98054","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-05-28T03:41:12.413736Z","iopub.execute_input":"2024-05-28T03:41:12.414167Z","iopub.status.idle":"2024-05-28T03:41:16.271781Z","shell.execute_reply.started":"2024-05-28T03:41:12.414111Z","shell.execute_reply":"2024-05-28T03:41:16.270435Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X_val = pca.transform(X_train)\nY_val = y_train\n\nbest_cv_clf = knn_clf_gs.best_estimator_\ny_pred = best_cv_clf.predict(X=X_val)\n\ny_pred_number = y_pred\ny_val_number = Y_val\n\nprint('KNN number, Acc({:.2f}%)', accuracy_score(y_val_number, y_pred_number))\n\nconf_matrix = confusion_matrix(y_val_number, y_pred_number)\nconf_matrix_disp = ConfusionMatrixDisplay(conf_matrix)\nconf_matrix_disp.plot(cmap='Blues')\n\ny_pred_angle = (y_pred % numRotate_prior)\n\ny_pred_angle_degree = y_pred_angle/numRotate_prior * angleRange_prior + angleOffset_prior\n\nprint('y_pred_angle_degree =', y_pred_angle_degree)","metadata":{"_uuid":"da419e38-d545-49ee-a8b6-32f3fa9760c3","_cell_guid":"fa5652c2-d596-4880-8396-b6e77177f4ff","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-05-28T03:41:16.273104Z","iopub.execute_input":"2024-05-28T03:41:16.273563Z","iopub.status.idle":"2024-05-28T03:41:51.374897Z","shell.execute_reply.started":"2024-05-28T03:41:16.273527Z","shell.execute_reply":"2024-05-28T03:41:51.373156Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Bây giờ, hãy áp dụng biến đổi nghịch đảo để tính toán\n# sự gần đúng của hình ảnh dựa trên nguyên tắc 10\n#thành phần.\n# Chúng ta có thể thấy rằng chỉ với 10 thành phần chúng ta đã truyền tải được một lượng lớn\n# một phần thông tin của mỗi hình ảnh.\npca_object = pca.named_steps['pca']\n\nfig, axs = plt.subplots(nrows=10, ncols=2, figsize=(6,20))\n\nfor i in range(10):\n idx = np.where(y_train == i )[0][0]\n original_img = axs[i, 0].imshow(\n X=X_train[idx].reshape(28, 28),\n cmap='Blues'\n )\n original_img.axes.get_yaxis().set_visible(False)\n original_img.axes.get_xaxis().set_visible(False)\n original_img.axes.set_title(f'Original [{i}]:')\n \n approx_img_ = pca_object.inverse_transform(X=X_val[idx])\n approx_img = axs[i, 1].imshow(\n X=approx_img_.reshape(28, 28),\n cmap='Blues'\n )\n approx_img.axes.get_yaxis().set_visible(False)\n approx_img.axes.get_xaxis().set_visible(False)\n approx_img.axes.set_title(f'Predicted angle [{y_pred_angle_degree[idx]}]:')","metadata":{"_uuid":"bc4e4c18-00b8-4441-900b-83786685d798","_cell_guid":"1393d26a-3a87-4a77-a642-c5b68c94fec3","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-05-28T03:41:51.376870Z","iopub.execute_input":"2024-05-28T03:41:51.377424Z","iopub.status.idle":"2024-05-28T03:41:52.569874Z","shell.execute_reply.started":"2024-05-28T03:41:51.377372Z","shell.execute_reply":"2024-05-28T03:41:52.568522Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n
\n

\nLight CNN Classifier\n

\n
\n
","metadata":{"_uuid":"a1446365-443d-48e8-aa24-514805f2df61","_cell_guid":"f6dca012-b643-484b-8f63-b0e218c6f1e2","trusted":true}},{"cell_type":"markdown","source":"","metadata":{"_uuid":"5a238d9f-7fbe-4d2a-85c7-b38f1793fe47","_cell_guid":"d606a8da-01ba-4650-b234-842037898d97","trusted":true}},{"cell_type":"markdown","source":"### CNN là một cách rất hiệu quả cho các nhiệm vụ thị giác máy tính. \nMạng thần kinh tích chập (CNN hoặc ConvNet) là một lớp Mạng thần kinh nhân tạo (ANN), được áp dụng phổ biến nhất để phân tích hình ảnh trực quan. Chúng còn được gọi là Mạng thần kinh nhân tạo bất biến thay đổi hoặc bất biến không gian (SIANN), dựa trên kiến trúc trọng lượng chung của các hạt tích chập hoặc các bộ lọc trượt dọc theo các tính năng đầu vào và cung cấp các phản hồi tương đương dịch thuật được gọi là bản đồ tính năng. Ngược lại với trực giác, hầu hết các mạng lưới thần kinh tích chập chỉ tương đương, trái ngược với bất biến, đối với dịch thuật. Chúng có các ứng dụng trong nhận dạng hình ảnh và video, hệ thống gợi ý, phân loại hình ảnh, phân đoạn hình ảnh, phân tích hình ảnh y tế, xử lý ngôn ngữ tự nhiên, giao diện não-máy tính và chuỗi thời gian tài chính.","metadata":{"_uuid":"3f62053b-0364-4ff6-bb7b-5deba2a02280","_cell_guid":"1c349770-f9d8-4f7f-8efb-d87b402f2f10","trusted":true}},{"cell_type":"code","source":"import plotly.express as px\nfrom plotly.offline import init_notebook_mode, iplot\nimport plotly.graph_objects as go\nimport warnings\nwarnings.filterwarnings('ignore')","metadata":{"execution":{"iopub.status.busy":"2024-05-28T03:41:52.571340Z","iopub.execute_input":"2024-05-28T03:41:52.571692Z","iopub.status.idle":"2024-05-28T03:41:52.601062Z","shell.execute_reply.started":"2024-05-28T03:41:52.571653Z","shell.execute_reply":"2024-05-28T03:41:52.599734Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def plot_confusion_matrix(cm, labels, title):\n # cm : danh sách ma trận nhầm lẫn(danh sách)\n # labels : tên danh sách dữ liệu(str)\n #title : tiêu đề cho bản đồ nhiệt\n data = go.Heatmap(z=cm, y=labels, x=labels)\n annotations = []\n for i, row in enumerate(cm):\n for j, value in enumerate(row):\n annotations.append(\n {\n \"x\": labels[i],\n \"y\": labels[j],\n \"font\": {\"color\": \"white\"},\n \"text\": str(value),\n \"xref\": \"x1\",\n \"yref\": \"y1\",\n \"showarrow\": False\n }\n )\n layout = {\n \"title\": title,\n \"xaxis\": {\"title\": \"Predicted value\"},\n \"yaxis\": {\"title\": \"Real value\"},\n \"annotations\": annotations\n }\n fig = go.Figure(data=data, layout=layout)\n return fig","metadata":{"execution":{"iopub.status.busy":"2024-05-28T03:41:52.602658Z","iopub.execute_input":"2024-05-28T03:41:52.603036Z","iopub.status.idle":"2024-05-28T03:41:52.613399Z","shell.execute_reply.started":"2024-05-28T03:41:52.603001Z","shell.execute_reply":"2024-05-28T03:41:52.611850Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# từ tensorflow.keras.utils nhập np_utils\nfrom tensorflow.keras import utils\nfrom tensorflow.keras.callbacks import ModelCheckpoint, EarlyStopping\nfrom tensorflow.keras import datasets, layers, models","metadata":{"_uuid":"17fc4707-d986-4950-b0fe-ac152932ac00","_cell_guid":"0d0efe69-4e03-4204-86d8-3ee2e5125830","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-05-28T03:41:52.615194Z","iopub.execute_input":"2024-05-28T03:41:52.615591Z","iopub.status.idle":"2024-05-28T03:42:07.431023Z","shell.execute_reply.started":"2024-05-28T03:41:52.615551Z","shell.execute_reply":"2024-05-28T03:42:07.429558Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X_train_cnn = X_train_rotate.reshape(-1,28,28,1)\n# X_val = np.array(pca.transform(X_train)).reshape(-1,28,28,1)\ny_train_cnn = utils.to_categorical(np.array(y_train_rotate// numRotate_prior))\nX_val = X_train.reshape(-1,28,28,1)\nY_val = utils.to_categorical(np.array(y_train))","metadata":{"_uuid":"598e24f0-4665-4486-ba5d-fb550022bc96","_cell_guid":"b20ca6f3-b288-42fe-8210-9e804c27268e","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-05-28T03:42:07.437374Z","iopub.execute_input":"2024-05-28T03:42:07.438233Z","iopub.status.idle":"2024-05-28T03:42:07.453398Z","shell.execute_reply.started":"2024-05-28T03:42:07.438190Z","shell.execute_reply":"2024-05-28T03:42:07.451799Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X_train_cnn.shape","metadata":{"execution":{"iopub.status.busy":"2024-05-28T03:42:07.455225Z","iopub.execute_input":"2024-05-28T03:42:07.456115Z","iopub.status.idle":"2024-05-28T03:42:07.503012Z","shell.execute_reply.started":"2024-05-28T03:42:07.456063Z","shell.execute_reply":"2024-05-28T03:42:07.501642Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# y_train_rotate// numRotate_prior","metadata":{"execution":{"iopub.status.busy":"2024-05-28T03:42:07.504692Z","iopub.execute_input":"2024-05-28T03:42:07.505189Z","iopub.status.idle":"2024-05-28T03:42:07.512810Z","shell.execute_reply.started":"2024-05-28T03:42:07.505123Z","shell.execute_reply":"2024-05-28T03:42:07.511423Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# X_train_rotate = np.array(X_train_rotate).reshape(-1,28,28,1)\n# # X_val = np.array(pca.transform(X_train)).reshape(-1,28,28,1)\n# X_val = np.array(X_test_crop).reshape(-1,28,28,1)\n# y_train_rotate = utils.to_categorical(np.array(y_train_rotate))\n# Y_val = utils.to_categorical(np.array(y_test_crop))","metadata":{"_uuid":"f978abbd-867f-46a5-b6ec-35f64e740f64","_cell_guid":"9d27fd4b-f9a8-4c46-a856-7e8970246998","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-05-28T03:42:07.514170Z","iopub.execute_input":"2024-05-28T03:42:07.514571Z","iopub.status.idle":"2024-05-28T03:42:07.526050Z","shell.execute_reply.started":"2024-05-28T03:42:07.514530Z","shell.execute_reply":"2024-05-28T03:42:07.524717Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = models.Sequential()\nmodel.add(layers.Conv2D(filters = 32, kernel_size = (5,5),padding = 'Same', \n activation ='relu', input_shape = (28,28,1)))\nmodel.add(layers.Conv2D(filters = 32, kernel_size = (5,5),padding = 'Same', \n activation ='relu'))\nmodel.add(layers.MaxPool2D(pool_size=(2,2)))\nmodel.add(layers.BatchNormalization())\nmodel.add(layers.Dropout(0.25))\nmodel.add(layers.Conv2D(filters = 64, kernel_size = (3,3),padding = 'Same', \n activation ='relu'))\nmodel.add(layers.Conv2D(filters = 64, kernel_size = (3,3),padding = 'Same', \n activation ='relu'))\nmodel.add(layers.MaxPool2D(pool_size=(2,2), strides=(2,2)))\nmodel.add(layers.BatchNormalization())\nmodel.add(layers.Dropout(0.25))\nmodel.add(layers.Flatten())\nmodel.add(layers.Dense(256, activation = \"relu\"))\nmodel.add(layers.Dropout(0.5))\nmodel.add(layers.BatchNormalization())\nmodel.add(layers.Dense(10, activation = \"softmax\"))\n\nmodel.summary()","metadata":{"_uuid":"d30da279-3261-4e97-9b79-6e2a01d703d4","_cell_guid":"43ead369-05c5-4e2d-960c-d5773b8a7f7d","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-05-28T03:42:07.527752Z","iopub.execute_input":"2024-05-28T03:42:07.528207Z","iopub.status.idle":"2024-05-28T03:42:07.943336Z","shell.execute_reply.started":"2024-05-28T03:42:07.528166Z","shell.execute_reply":"2024-05-28T03:42:07.941929Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model_path = './best_model.h5'\ncheckpointer = ModelCheckpoint(\n filepath=model_path,\n monitor='val_accuracy',\n verbose=1,\n mode='max',\n save_best_only=True)\n\nmodel.compile(loss='categorical_crossentropy',\n optimizer='adam',\n metrics=['accuracy'])","metadata":{"_uuid":"d131aa49-918d-43dd-be19-62d3110ca17a","_cell_guid":"db889d42-b8bb-4667-8f07-b831a3fbcf38","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-05-28T03:42:07.944880Z","iopub.execute_input":"2024-05-28T03:42:07.945273Z","iopub.status.idle":"2024-05-28T03:42:07.973891Z","shell.execute_reply.started":"2024-05-28T03:42:07.945238Z","shell.execute_reply":"2024-05-28T03:42:07.971852Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"history = model.fit(X_train_cnn, y_train_cnn, validation_data = (X_val, Y_val), epochs=80, batch_size = 128, callbacks=[checkpointer])","metadata":{"_uuid":"620687bc-89d9-40dc-b133-006fef4e7c78","_cell_guid":"cd8420f8-3c35-407f-93de-b1691d1b534f","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2024-05-28T03:42:07.975500Z","iopub.execute_input":"2024-05-28T03:42:07.975899Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import plotly.graph_objects as go\nfrom plotly.subplots import make_subplots\nimport numpy as np\n\nfig = go.Figure()\n\nlabels = ['Loss', 'Accuracy']\ncolors = ['rgb(67,67,67)', 'rgb(49,130,189)']\n\nmode_size = [8, 8]\nline_size = [2, 2]\n\nx_data = np.vstack((np.arange(1, 51),)*2)\n\ny_data = np.array([\n np.array(history.history['loss']), \n np.array(history.history['accuracy'])\n])\n\nfor i in range(0, 2):\n fig.add_trace(go.Scatter(x=x_data[i], y=y_data[i], mode='lines',\n name=labels[i],\n line=dict(color=colors[i], width=line_size[i]),\n connectgaps=True,\n ))\n\n # endpoints\n fig.add_trace(go.Scatter(\n x=[x_data[i][0], x_data[i][-1]],\n y=[y_data[i][0], y_data[i][-1]],\n mode='markers',\n marker=dict(color=colors[i], size=mode_size[i])\n ))\n\nfig.update_layout(\n xaxis=dict(\n showline=True,\n showgrid=False,\n showticklabels=True,\n linecolor='rgb(204, 204, 204)',\n linewidth=2,\n ticks='outside',\n tickfont=dict(\n family='Arial',\n size=12,\n color='rgb(82, 82, 82)',\n ),\n ),\n yaxis=dict(\n showgrid=False,\n zeroline=False,\n showline=False,\n showticklabels=False,\n ),\n autosize=False,\n margin=dict(\n autoexpand=False,\n l=100,\n r=20,\n t=110,\n ),\n showlegend=False,\n plot_bgcolor='white'\n)\n\nannotations = []\n\n# Thêm nhãn\nfor y_trace, label, color in zip(y_data, labels, colors):\n# dán nhãn phía bên trái của ô\n annotations.append(dict(xref='paper', x=0.05, y=y_trace[0],\n xanchor='right', yanchor='middle',\n text=label + ' {:.2f}'.format(y_trace[0]),\n font=dict(family='Arial',\n size=16),\n showarrow=False))\n # dán nhãn bên phải của ô\n annotations.append(dict(xref='paper', x=0.95, y=y_trace[49],\n xanchor='left', yanchor='middle',\n text='{:.4f}%'.format(y_trace[49]),\n font=dict(family='Arial',\n size=16),\n showarrow=False))\n# Tiêu đề\nannotations.append(dict(xref='paper', yref='paper', x=0.0, y=1.05,\n xanchor='left', yanchor='bottom',\n text='Training Loss & Accuracy',\n font=dict(family='Arial',\n size=30,\n color='rgb(37,37,37)'),\n showarrow=False))\n# Nguồn\nannotations.append(dict(xref='paper', yref='paper', x=0.5, y=-0.1,\n xanchor='center', yanchor='top',\n text='Epochs',\n font=dict(family='Arial',\n size=12,\n color='rgb(150,150,150)'),\n showarrow=False))\n\nfig.update_layout(annotations=annotations,height=600, width=1000)\n\nfig.show()","metadata":{"_uuid":"b2a98a55-6032-40d1-8bbf-bf6b35840730","_cell_guid":"7fcbf806-1db3-45b9-8bc0-3abf08dfabd9","collapsed":false,"jupyter":{"outputs_hidden":false},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"_uuid":"c51476fa-c9d6-4cbc-a5ad-f6ff9706d70e","_cell_guid":"8bdfa1eb-cd3b-4e48-a377-02b2b66506e1","collapsed":false,"jupyter":{"outputs_hidden":false},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from keras.models import load_model\nmodel = load_model('./best_model.h5')\ny_pred = model.predict(X_val)\nM = confusion_matrix(np.argmax(Y_val,1), np.argmax(y_pred,1))\nplot_confusion_matrix(M, sorted(np.unique(np.argmax(Y_val,1))), 'Simple CNN Confusion Matrix, Acc({:.2f}%)'.format(accuracy_score(np.argmax(Y_val,1), np.argmax(y_pred,1))*100))","metadata":{"_uuid":"57a8380a-11c2-45fb-8190-4e9da5bce1ed","_cell_guid":"bdeca8dc-2a69-4598-b508-873212a964dd","collapsed":false,"jupyter":{"outputs_hidden":false},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n
\n

\nChecking Failure Cases\n

\n
\n
","metadata":{"_uuid":"8f029ba0-23b9-4fa0-83a3-eae4a5e4b39b","_cell_guid":"d955f566-e62b-4f51-a801-3de790bca12a","trusted":true}},{"cell_type":"code","source":"n = np.where(np.argmax(y_pred,1)!=np.argmax(Y_val,1))\nn = np.array(n).reshape(-1)\nn","metadata":{"_uuid":"042a5e34-2598-420b-bb05-79a34d48eac3","_cell_guid":"3cea1159-64d0-4e09-88e9-7b2f3a4fc1fb","collapsed":false,"jupyter":{"outputs_hidden":false},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X_val_vis = np.array(X_val).reshape(X_val.shape[0], 28, 28)\nX_val_vis = X_val_vis\nfig, axis = plt.subplots(int(len(n)/8), 8, figsize=(35, 15))\nfor i, ax in enumerate(axis.flat):\n ax.imshow(X_val_vis[n[i]], cmap='binary')\n digit = np.argmax(y_pred,1)[n[i]]\n digit2 = np.argmax(Y_val,1)[n[i]]\n ax.set(title = f\"Pred Number is {digit}\\n Real Number is {digit2}\");","metadata":{"_uuid":"470a60fd-078e-4a6b-99bf-206ffd71cfb8","_cell_guid":"33ee870d-1eaa-4702-ac0b-7d41a9333040","collapsed":false,"jupyter":{"outputs_hidden":false},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":" It looks like even human can't get the right answers.😅","metadata":{"_uuid":"7b0368af-6a8a-44d9-8dee-e994819269d4","_cell_guid":"11579e1a-7889-4904-ab0f-b96007a7069b","trusted":true}},{"cell_type":"markdown","source":"
\n
\n

\nSubmit to Leaderboard\n

\n
\n
","metadata":{"_uuid":"b052fb30-6e45-4c66-87af-ce25efdbe87e","_cell_guid":"739fd7c4-6753-4052-8204-a0038c866f87","trusted":true}},{"cell_type":"code","source":"X_test = np.array(df_test).reshape(-1,28,28,1)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = load_model('./best_model.h5')\ny_pred = model.predict(X_test)\ny_pred = np.argmax(y_pred,1)\npd_submit = pd.read_csv('../input/digit-recognizer/sample_submission.csv')\npd_submit['Label'] = y_pred\npd_submit.to_csv('submission.csv', header=True, index=False)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 📃 References 📃","metadata":{"_uuid":"dc3e2507-cb68-47fb-b3e6-3a0c9c7f84f7","_cell_guid":"e301b5a4-ce01-4ba9-ac3b-40983ec40265","trusted":true}},{"cell_type":"markdown","source":"
\n \n
","metadata":{"_uuid":"a9ad3adf-fc21-4ebc-9eb2-d83f80966629","_cell_guid":"f861cd0f-8e0d-48d3-95f9-029e6cf80e84","trusted":true}}]}