{"metadata":{"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":3004,"databundleVersionId":861823,"sourceType":"competition"}],"dockerImageVersionId":30732,"isInternetEnabled":false,"language":"python","sourceType":"notebook","isGpuEnabled":true},"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.10.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"papermill":{"default_parameters":{},"duration":14560.215025,"end_time":"2024-06-11T07:06:57.149963","environment_variables":{},"exception":null,"input_path":"__notebook__.ipynb","output_path":"__notebook__.ipynb","parameters":{},"start_time":"2024-06-11T03:04:16.934938","version":"2.5.0"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# 🔶 PCA & KNN 🔶\n### 🔸 XỬ LÝ ẢNH • Principal Component Analysis & K-Nearest Neighbors 🔸","metadata":{"papermill":{"duration":0.018196,"end_time":"2024-06-11T03:04:21.777672","exception":false,"start_time":"2024-06-11T03:04:21.759476","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"\nCơ sở dữ liệu
MNIST (
Cơ sở dữ liệu của Viện Tiêu chuẩn và Công nghệ Quốc gia được Sửa đổi) là một cơ sở dữ liệu lớn chứa các
chữ số viết tay thường được sử dụng để huấn luyện các hệ thống xử lý hình ảnh khác nhau. Khi các kỹ thuật học máy mới xuất hiện, MNIST vẫn là một nguồn tài nguyên đáng tin cậy cho các nhà nghiên cứu và người học.
\nCác
hình ảnh đen trắng gốc đã được chuẩn hóa để vừa với một
hộp giới hạn 28x28 điểm ảnh và được làm mịn, điều này đã giới thiệu các
mức xám. Cơ sở dữ liệu MNIST gốc chứa 60,000 hình ảnh huấn luyện và 10,000 hình ảnh kiểm tra, nhưng trong cuộc thi này, chúng tôi được cung cấp
42,000 hình ảnh huấn luyện (70%) và
18,000 hình ảnh kiểm tra (30%).
\nCác kỹ thuật
tân tiến nhất để tiếp cận bất kỳ vấn đề Phân loại Hình ảnh nào thường phụ thuộc vào
Mạng nơ-ron tích chập. Loại kiến trúc này rất tốt trong việc trích xuất các mẫu từ hình ảnh. Tuy nhiên, vì MNIST là một vấn đề đơn giản, trong sổ ghi chú này, chúng tôi sẽ áp dụng
Các Thuật toán Học máy đơn giản (và chúng tôi sẽ đạt được độ chính xác tốt!).
\nTrước tiên, chúng tôi sẽ thực hiện một
Phân tích thành phần chính (PCA) để giảm chiều dữ liệu (từ 784 giá trị điểm ảnh xuống còn 10 thành phần chính). Sau đó, chúng tôi sẽ chạy thuật toán
lười biếng yêu thích của chúng tôi:
Người láng giềng gần nhất (K-Nearest Neighbors). Kết hợp này đạt được
Độ chính xác Kiểm tra là 0.9271, điều này khá đáng nể!!!
\n
Mục lục:\n
\n - 🔎 Đọc & Khám phá Dữ liệu 🔍:
\n - Kiểm tra xem có giá trị còn thiếu không.
\n - Kiểm tra phạm vi của các giá trị điểm ảnh.
\n - Hiển thị dữ liệu (Hình ảnh).
\n - Kiểm tra xem mục tiêu có cân bằng không.
\n - 🔶 PCA 🔶:
\n - Giới thiệu thuật toán PCA.
\n - Tiền xử lý (chuẩn hóa) dữ liệu.
\n - Thực hiện thuật toán PCA bằng thư viện Sklearn.
\n - Nhúng các mô-đun vào một Ống dẫn Sklearn.
\n - Vẽ biểu đồ phương sai được giải thích của các thành phần.
\n - Hiển thị hai chiều của dữ liệu bằng cách sử dụng seaborn.
\n - Hiển thị ba chiều động của dữ liệu bằng cách sử dụng plotly.
\n - Hiển thị phản chiếu nghịch đảo của dữ liệu.
\n - Kiểm tra xem các thành phần có không tương quan không.
\n - 🔶 Bộ phân loại KNN 🔶:
\n - Xây dựng một Bộ phân loại KNN bằng cách sử dụng Sklearn.
\n - Sử dụng kỹ thuật PCA như một trích xuất đặc trưng cho KNN.
\n - Chạy một Lưới tìm kiếm Chéo để tối ưu hóa các siêu tham số n_neighbors.
\n - So sánh các chỉ số đo cho tập huấn luyện và tập kiểm tra để kiểm tra xem có quá mức không.
\n - Vẽ Ma trận nhầm lẫn.
\n - Tính toán Báo cáo Phân loại.
\n
\n
\nTrong phần này, chúng ta sẽ sử dụng thư viện Pandas nổi tiếng của Python để đọc dữ liệu từ các tệp .csv.\n\nChúng ta sẽ trực quan hóa phân phối các giá trị pixel (biểu đồ histogram) bằng cách sử dụng các thư viện Seaborn và Matplotlib của Python. Chúng ta cũng sẽ kiểm tra để đảm bảo rằng không có giá trị nào bị thiếu và rằng các lớp mục tiêu được cân bằng.\n\nCuối cùng, chúng ta sẽ trực quan hóa các hình ảnh bằng cách vẽ một hình ảnh từ mỗi lớp.\n
\n","metadata":{"papermill":{"duration":0.0171,"end_time":"2024-06-11T03:04:26.462921","exception":false,"start_time":"2024-06-11T03:04:26.445821","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# Đọc các tệp dữ liệu bằng Pandas\n# df_train chứa dữ liệu huấn luyện từ tệp 'train.csv'\n# df_train = pd.read_csv('data/train.csv')\n# df_test = pd.read_csv('data/test.csv')\n\n# Read the data files using Pandas\ndf_train = pd.read_csv('/kaggle/input/digit-recognizer/train.csv')\ndf_test = pd.read_csv('/kaggle/input/digit-recognizer/test.csv')","metadata":{"papermill":{"duration":7.006811,"end_time":"2024-06-11T03:04:33.486934","exception":false,"start_time":"2024-06-11T03:04:26.480123","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-14T06:01:38.097885Z","iopub.execute_input":"2024-06-14T06:01:38.098125Z","iopub.status.idle":"2024-06-14T06:01:42.756355Z","shell.execute_reply.started":"2024-06-14T06:01:38.098104Z","shell.execute_reply":"2024-06-14T06:01:42.755437Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Kiểm tra kích thước của DataFrame huấn luyện và kiểm tra\n# In ra kích thước của df_train, chứa 70% dữ liệu huấn luyện\nprint(\"df_train shape:\", df_train.shape)\n\n# In ra kích thước của df_test, chứa 30% dữ liệu kiểm tra\nprint(\"df_test shape: \", df_test.shape)\n","metadata":{"papermill":{"duration":0.029078,"end_time":"2024-06-11T03:04:33.533657","exception":false,"start_time":"2024-06-11T03:04:33.504579","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-14T06:01:42.758990Z","iopub.execute_input":"2024-06-14T06:01:42.759297Z","iopub.status.idle":"2024-06-14T06:01:42.764714Z","shell.execute_reply.started":"2024-06-14T06:01:42.759271Z","shell.execute_reply":"2024-06-14T06:01:42.763768Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Kiểm tra xem có giá trị thiếu nào trong df_train không và in ra tổng số lượng giá trị thiếu\nprint(\n \"Giá trị bị thiếu trong file df_train: \", \\\n df_train.isna().sum().sum()\n)\n\n# Kiểm tra xem có giá trị thiếu nào trong df_test không và in ra tổng số lượng giá trị thiếu\nprint(\n \"Giá trị bị thiếu trong file df_test: \", \\\n df_test.isna().sum().sum()\n)\n","metadata":{"papermill":{"duration":0.076699,"end_time":"2024-06-11T03:04:33.628093","exception":false,"start_time":"2024-06-11T03:04:33.551394","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-14T06:01:42.765782Z","iopub.execute_input":"2024-06-14T06:01:42.766162Z","iopub.status.idle":"2024-06-14T06:01:42.818306Z","shell.execute_reply.started":"2024-06-14T06:01:42.766138Z","shell.execute_reply":"2024-06-14T06:01:42.817342Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Hiển thị năm hàng đầu tiên của dữ liệu huấn luyện\ndf_train.head()\n","metadata":{"papermill":{"duration":0.0564,"end_time":"2024-06-11T03:04:33.701982","exception":false,"start_time":"2024-06-11T03:04:33.645582","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-14T06:01:42.819492Z","iopub.execute_input":"2024-06-14T06:01:42.819852Z","iopub.status.idle":"2024-06-14T06:01:42.838644Z","shell.execute_reply.started":"2024-06-14T06:01:42.819819Z","shell.execute_reply":"2024-06-14T06:01:42.837572Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Chia dữ liệu thành hai phần: đặc trưng (X_train) và nhãn (y_train)\n# X_train chứa các cột trừ cột 'label' và chuyển đổi sang dạng mảng numpy\nX_train = df_train.drop(columns=['label']).values\n\n# y_train chứa cột 'label' và chuyển đổi sang dạng mảng numpy\ny_train = df_train['label'].values\n","metadata":{"papermill":{"duration":0.15986,"end_time":"2024-06-11T03:04:33.879312","exception":false,"start_time":"2024-06-11T03:04:33.719452","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-14T06:01:42.839881Z","iopub.execute_input":"2024-06-14T06:01:42.840255Z","iopub.status.idle":"2024-06-14T06:01:42.924028Z","shell.execute_reply.started":"2024-06-14T06:01:42.840221Z","shell.execute_reply":"2024-06-14T06:01:42.923203Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Trực quan hóa phân phối các giá trị pixel trong dữ liệu huấn luyện\n# Sử dụng seaborn.histplot để tạo histogram\n# Màu của histogram được thiết lập thành '#4895EE'\nax = sns.histplot(x=X_train.flatten(), color='#4895EE')\n\n# Đặt tiêu đề và nhãn cho biểu đồ\n_ = ax.set(title='Biểu đồ giá trị Pixel', xlabel='Giá trị Pixel')\n\n# Như chúng ta có thể thấy, giá trị 0 đại diện cho nền\n# và các giá trị >0 đại diện cho các chữ số trong hình ảnh\n","metadata":{"papermill":{"duration":18.274145,"end_time":"2024-06-11T03:04:52.239152","exception":false,"start_time":"2024-06-11T03:04:33.965007","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-14T06:01:42.925440Z","iopub.execute_input":"2024-06-14T06:01:42.925874Z","iopub.status.idle":"2024-06-14T06:01:57.804724Z","shell.execute_reply.started":"2024-06-14T06:01:42.925824Z","shell.execute_reply":"2024-06-14T06:01:57.803786Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Kiểm tra xem phân phối của nhãn có cân bằng không\n# Sử dụng seaborn.countplot để đếm số lượng mẫu cho mỗi nhãn\n# Màu của đếm được thiết lập thành '#4895EE'\nax = sns.countplot(x=y_train, color='#4895EE')\n\n# Đặt tiêu đề và nhãn cho biểu đồ\nax.set(\n title='Phân bổ mục tiêu có cân bằng không? Đúng!',\n xlabel='Nhãn'\n)\n\n# Vẽ một đường thẳng ngang biểu thị số lượng mẫu mong đợi cho mỗi lớp trong trường hợp phân phối cân bằng\n_ = ax.axhline(y=4200, color='#12086F', linestyle='--')\n\n# Vì các lớp nhãn được cân bằng, chúng ta không cần phải lo lắng về các phương pháp lấy mẫu và sai lệch trong các mô hình của chúng ta.\n","metadata":{"papermill":{"duration":0.382305,"end_time":"2024-06-11T03:04:52.640687","exception":false,"start_time":"2024-06-11T03:04:52.258382","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-14T06:01:57.806190Z","iopub.execute_input":"2024-06-14T06:01:57.807004Z","iopub.status.idle":"2024-06-14T06:01:58.136068Z","shell.execute_reply.started":"2024-06-14T06:01:57.806967Z","shell.execute_reply":"2024-06-14T06:01:58.135089Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Tiếp theo, hãy trực quan hóa một hình ảnh từ mỗi lớp nhãn.\n# Chúng ta sẽ sử dụng hàm 'imshow' của matplotlib.\n\n# Tạo một lưới các biểu đồ với 2 hàng và 5 cột\nfig, axs = plt.subplots(nrows=2, ncols=5, figsize=(10, 5))\n\n# Duyệt qua các nhãn từ 0 đến 9\nfor i in range(10):\n # Tìm chỉ số của mẫu đầu tiên có nhãn i\n idx = np.where(y_train == i)[0][2]\n \n # Trực quan hóa hình ảnh tương ứng với nhãn i\n # Hình ảnh được chuyển đổi lại thành ma trận 28x28 và sử dụng màu 'Blues'\n img = axs[int(i >= 5), i % 5].imshow(\n X_train[idx].reshape(28, 28),\n cmap='Blues'\n )\n \n # Ẩn trục y và trục x\n img.axes.get_yaxis().set_visible(False)\n img.axes.get_xaxis().set_visible(False)\n \n # Đặt tiêu đề cho mỗi hình ảnh với nhãn tương ứng\n img.axes.set_title(f'Nhãn: {i}')\n","metadata":{"papermill":{"duration":0.882202,"end_time":"2024-06-11T03:04:53.543369","exception":false,"start_time":"2024-06-11T03:04:52.661167","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-14T06:01:58.140845Z","iopub.execute_input":"2024-06-14T06:01:58.141147Z","iopub.status.idle":"2024-06-14T06:01:58.550512Z","shell.execute_reply.started":"2024-06-14T06:01:58.141122Z","shell.execute_reply":"2024-06-14T06:01:58.549523Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"DỮ LIỆU CỦA XỬ LÝ ẢNH","metadata":{}},{"cell_type":"code","source":"# # ---------- Khai báo ----------\n# numRotate_prior = 10 # Số lần xoay trước\n\n# angleOffset_prior = -20 # Góc xoay ban đầu\n# angleRange_prior = 40 # Phạm vi góc xoay\n\n# # Tập hợp các chỉ số hình ảnh và các offset góc tương ứng\n\n# X_train_pick2rotate = {\n# 'index': np.array([2700, 2701, 2702, 2703, 2704, 2705,2706, 2707,2708, 2709,\n# 2710, 2711, 2712, 2713, 2714, 2715,2716, 2717,2718, 2719,\n# 2720, 2721, 2722, 2723, 2724, 2725,2726, 2727,2728, 2729,\n# 2730, 2731, 2732, 2733, 2734, 2735,2736, 2737,2738, 2739,\n# 2740, 2741, 2742, 2743, 2744, 2745,2746, 2747,2748, 2749,\n# 2750, 2751, 2752, 2753, 2754, 2735,2756, 2757,2758, 2759,\n# 2760, 2761, 2762, 2763, 2764, 2765,2766, 2767,2768, 2769,\n# 2770, 2771, 2772, 2773, 2774, 2775,2776, 2777,2778, 2779,\n# 2780, 2781, 2782, 2783, 2784, 2785,2786, 2787,2788, 2789,\n# 2790, 2791, 2792, 2793, 2794, 2795,2796, 2797,2798, 2799\n# ]),\n \n# 'offset': np.array([\n# #2700 - 2709\n# [ 120, -150, 10, 32, 10, 10, 30, 0, 20, 0],\n\n# [30, 10, 20, 20, 20, 30, 10, 30, 10, 30],\n\n# [ 20,0, 10, 20, 50, -170, 30, 30, 30,10],\n\n# [-120, 30, 10, 10, 20, 10, 40, 30, 40, 40],\n\n# [ 50,10 , 10, 50, 50, 10, 20, 10, 0, 40],\n\n# [ 100, 10, 10, 20, 10, 180, 0, 0, 10, 0],\n \n# [40,10, -20,20,20, 30, 20,10,10,20],\n \n# [40,-150, 10,20,30, 40,-60,10,20,10],\n \n# [0,30,10,0,10, 10,20,30,30,30],\n \n# [50,10,10,30,10, 10,30,40,20,0],\n \n# # 2710 - 2719\n \n# [ 40, 180, 10, 32, 30, 20, -10, -30, -10, -50],\n\n# [50, 10, 20, 20, 30, 40, 0, 0, 40, -20],\n\n# [ 20,50, 10, 20, 50, 180, 10, -10, 30,10],\n\n# [ 0, 50, 20, 10, 20, 20, 0, -10, -10, 40],\n\n# [ 0, 50, 10, 30, 10, 20, 20, 0, 0, 40],\n\n# [ 70, 30, 10, 20, 50, 50, 10, 10, 20, 0],\n \n# [50,10, 10,30,20, 20, 20,10,10,20],\n \n# [50,30, 10,20,30, 10,-20,10,20,10],\n \n# [10,10,10,20,10, 10,20,-10,30,30],\n \n# [30,10,10,30,10, 10,30,-10,20,20],\n \n# #2720 - 2729\n# [ 20, 20, 10, 20, 30, 20, -10, -30, -10, -50],\n\n# [60, 10, 10, 10, 30, 40, 0, 0, 40, -20],\n\n# [ 70,20, 10, 10, 50, 180, 10, -10, 30,10],\n\n# [ 90, 20, 10, 20, 20, 20, 0, -10, -10, 40],\n\n# [ 50, 20, 10, 30, 10, 20, 20, 0, 0, 40],\n\n# [ 70, -170, 10, 30, 50, 50, 10, 10, 20, 0],\n \n# [20,10, 10,30,20, 20, 20,10,10,20],\n \n# [-150,50, 10,20,30, 10,-20,10,20,10],\n \n# [20,50,10,30,10, 10,20,-10,30,30],\n \n# [-50,40,180,-20,10, 10,30,-10,20,20],\n \n# #2730 - 2739\n# [ 20, -120, 10, 32, 30, 50, 0, 10, 10, 20],\n\n# [60, -140, 10, 20, 30, 0, -50, 0, -10, 20],\n\n# [ 70,10, 20, 20, 50, 0, 50, 10, 10,10],\n\n# [ 90, 20, 20, 10, 20, 20, -20, 10, 10, 0],\n\n# [ 50, 40, -10, 30, 10, 10, -20, 0, 20, 0],\n\n# [ 70, 40, 0, 20, 50, 20, 40, 0, 10, 20],\n \n# [20,40, -10,30,20, 10, 20,10,10,10],\n \n# [-150,50, 10,20,30, 10,20,0,20,-10],\n \n# [20,-130,10,20,10,20,20,10,30,0],\n \n# [-50,30,-10,30,10, -150,-30,10,30,-20],\n \n# #2740 - 2749\n# [ 50, 40, 0, 0, 0, 10, 10, 0, 20, -10],\n\n# [60, 20, -10, 10, 40, 20, 10, 0, 30, 30],\n\n# [ -40,50, 20, 10, 20, 20, 10, -10, 0,-10],\n\n# [ -80, 20, 10, 10, -10, 20, 10, -20, 0, 20],\n\n# [ -10, 10, 10, 20, 10, -10, 10, 0, 20, -10],\n\n# [ 40, 30, 0, 10,10, 0, 10, -20, 10, 0],\n \n# [50,40, 0,20,10, 30, 10,-20,-140,-10],\n \n# [50,40, -20,20,20, 20,10,0,10,-10],\n \n# [30, 10,0,20,50,10,10,0,20,-20],\n \n# [50,40,0,30,20, 20,10,100,30,-10],\n \n \n# #2750 - 2759\n# [ 20, 10, 0, 10, 10, 10, 10, -10, 30, 0],\n\n# [80, 30, 20, 20, 20, 10, 10, 10, 0, -20],\n\n# [ 40,20, 10, 10, 10, -10, -10, 10, -20,-10],\n\n# [ -150, 30, -10, 20, 20, 20, -30, 10, 0, -20],\n\n# [ 50, 20, -10, -10, 0, 10, 0, -10, 30, -20],\n\n# [ 50, 10, 10, 30, 50, 10, -10, 20, 40, -10],\n \n# [10,40, 30,20,40, 10, 20,10,-20,-10],\n \n# [0,30, 20,10,30, 10,30,30,170,0],\n \n# [30,20,0,-20,40, 10,-20,10,20,0],\n \n# [50,30,10,20,10, 10,0,30,20,140],\n \n# #2760 - 2769\n# [ 30, 50, 10, 0, 10, 10, 20, -10, 30, -10],\n\n# [50, 50, 20, -10, 10, 0, 10, 10, -30, -10],\n\n# [ 10,50, 20, 20, 10, 10, -30, 10, 10,-10],\n\n# [ 70, 40, 30, 20, 20, 20, -30, 10, 10, -10],\n\n# [ 60, 20, 10, 10, 10, 50, -30, -10, 40, 10],\n\n# [ 20, 180, 20, 0, -10, 10, 0, 20, 10, -10],\n \n# [30,10, -10,-10,10, 30, -10,10,30,0],\n \n# [20,30, 10,10,30, 10,-30,30,30,-20],\n \n# [20,20,0,10,40, 10,-10,10,40,-10],\n \n# [20,20,10,10,30, 20,-10,30,10,10],\n \n# #2770 - 2779\n# [ 30, 30, 10, 0, 10, 10, 20, -10, 30, -10],\n\n# [50, -10, 20, -10, 10, 0, 10, 10, -170, 0],\n\n# [ 10,40, 20, 20, 10, 10, -30, 10, 0,0],\n\n# [ 70, 20, 30, 20, 20, 20, -30, 10, -40, 10],\n\n# [ 60, 20, 10, 10, 10, 50, -30, -10, 10, 10],\n\n# [ 20, 10, 20, 0, -10, 10, 0, 20, 20, 10],\n \n# [30,10, -10,-10,10, 30, -10,10,10,20],\n \n# [20,10, 10,10,30, 10,-30,30,-10,30],\n \n# [20,-130,0,10,40, 10,-10,10,30,30],\n \n# [20,50,10,10,30, 20,-10,30,10,10],\n \n# #2780 - 2789\n# [ 0, 10, 10, 0, 10, 10, 20, -10, 30, -10],\n\n# [50, 10, 20, 20, 10, 0, 10, 10, -170, 0],\n\n# [ 10,40, 20, 20, 10, 10, -30, 10, 40,0],\n\n# [ 70, 10, -10, 20, 20, 20, -30, 10, 40, 10],\n\n# [ 60, 20, 0, 10, 10, 50, -30, -10, 10, 10],\n\n# [ 20, 20, 10, 0, -10, 180, 0, 20, 20, 10],\n \n# [30,30, 20,20,10, 180, -10,10,10,20],\n \n# [20,30, 20,10,30, 10,-30,30,-10,30],\n \n# [20,30,10,10,40, 10,-10,10,30,30],\n \n# [20,20,10,10,30, 20,-10,30,10,10],\n \n \n# #2790 - 2799\n# [ 20, -120, 10, 32, 30, 40, -10, -30, -10, -50],\n\n# [60, -140, 10, 20, 30, 10, 0, 0, 40, -20],\n\n# [ 70,10, 20, 20, 50, 30, 10, -10, 30,10],\n\n# [ 90, 20, 20, 10, 20, 10, 0, -10, -10, 40],\n\n# [ 50, 40, -10, 30, 10, 20, 20, 0, 0, 40],\n\n# [ 70, 40, 0, 20, 50, 10, 10, 10, 20, 0],\n \n# [20,40, -10,30,20, 20, 20,10,10,20],\n \n# [-150,50, 10,20,30, 10,-20,10,20,10],\n \n# [20,-130,10,20,10, 10,20,-10,30,30],\n \n# [-50,30,-10,30,10, 20,30,-10,20,20],\n# ])\n# }\n\n# # Số lượng hình ảnh được chọn để xoay\n# num_indices = X_train_pick2rotate['index'].shape[0]\n# num_offsets = X_train_pick2rotate['offset'].shape[0]\n\n# if num_indices == num_offsets:\n# print(\"Số lượng chỉ số và số lượng offset là bằng nhau.\")\n# else:\n# print(\"Số lượng chỉ số và số lượng offset không bằng nhau.\")\n# if num_indices - num_offsets > 0 :\n# print(\"So luong index lon hon offset\")\n# else: print(\"So luong index nho hon offset\")\n \n# numPickRotate = len(X_train_pick2rotate['index'])\n\n# print('numPickRotate = ', numPickRotate)\n# print('numOffsets = ', num_offsets)","metadata":{"execution":{"iopub.status.busy":"2024-06-14T06:01:58.552537Z","iopub.execute_input":"2024-06-14T06:01:58.553250Z","iopub.status.idle":"2024-06-14T06:01:58.577425Z","shell.execute_reply.started":"2024-06-14T06:01:58.553190Z","shell.execute_reply":"2024-06-14T06:01:58.576207Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Dữ liệu học máy","metadata":{}},{"cell_type":"code","source":"# ---------- Khai báo ----------\nnumRotate_prior = 10 # Số lần xoay trước\n\nangleOffset_prior = -20 # Góc xoay ban đầu\nangleRange_prior = 40 # Phạm vi góc xoay\n\n# Tập hợp các chỉ số hình ảnh và các offset góc tương ứng\n\nX_train_pick2rotate = {\n 'index': np.array([\n # 2700, 2701, 2702, 2703, 2704, 2705,2706, 2707,2708, 2709,\n # 2710, 2711, 2712, 2713, 2714, 2715,2716, 2717,2718, 2719,\n # 2720, 2721, 2722, 2723, 2724, 2725,2726, 2727,2728, 2729,\n # 2730, 2731, 2732, 2733, 2734, 2735,2736, 2737,2738, 2739,\n # 2740, 2741, 2742, 2743, 2744, 2745,2746, 2747,2748, 2749,\n # 2750, 2751, 2752, 2753, 2754, 2735,2756, 2757,2758, 2759,\n # 2760, 2761, 2762, 2763, 2764, 2765,2766, 2767,2768, 2769,\n # 2770, 2771, 2772, 2773, 2774, 2775,2776, 2777,2778, 2779,\n # 2780, 2781, 2782, 2783, 2784, 2785,2786, 2787,2788, 2789,\n # 2790, 2791, 2792, 2793, 2794, 2795,2796, 2797,2798, 2799,\n \n\n \n 0, 1, 2, 3, 4, 5, 6, 7, 8,9,\n 10, 11, 12, 13, 14, 15, 16, 17, 18, 19,\n 20, 21, 22, 23, 24, 25, 26, 27, 28, 29,\n 30, 31, 32, 33, 34, 35, 36, 37, 38, 39,\n 40, 41, 42, 43, 44, 45, 46, 47, 48, 49,\n 50, 51, 52, 53, 54, 55, 56, 57, 58, 59,\n 60, 61, 62, 63, 64, 65, 66, 67, 68, 69,\n 70, 71, 72, 73, 74, 75, 76, 77, 78, 79,\n 80, 81, 82, 83, 84, 85, 86, 87, 88, 89,\n 90, 91, 92, 93, 94,95,96,97,98,99,\n \n 3495, 3496, 3497, 3498, 3499,\n 3500, 3501, 3502, 3503, 3504,\n 3505, 3506, 3507, 3508, 3509,\n 3510, 3511, 3512, 3513, 3514,\n 3515, 3516, 3517, 3518, 3519,\n 3520, 3521, 3522, 3523, 3524,\n 3525, 3526, 3527, 3528, 3529,\n 3530, 3531, 3532, 3533, 3534,\n 3535, 3536, 3537, 3538, 3539,\n 3540, 3541, 3542, 3543, 3544,\n 3545, 3546, 3547, 3548, 3549,\n 3550, 3551, 3552, 3553, 3554,\n 3555, 3556, 3557, 3558, 3559,\n 3560, 3561, 3562, 3563, 3564,\n 3565, 3566, 3567, 3568, 3569,\n 3570, 3571, 3572, 3573, 3574,\n 3575, 3576, 3577, 3578, 3579,\n 3580, 3581, 3582, 3583, 3584,\n 3585, 3586, 3587, 3588, 3589,\n 3590, 3591, 3592, 3593, 3594,\n 3595, 3596, 3597, 3598, 3599,\n 3600, 3601, 3602, 3603, 3604,\n 3605, 3606, 3607, 3608, 3609,\n 3610, 3611, 3612, 3613, 3614,\n 3615, 3616, 3617, 3618, 3619,\n 3620, 3621, 3622, 3623, 3624,\n 3625, 3626, 3627, 3628, 3629,\n 3630, 3631, 3632, 3633, 3634,\n 3635, 3636, 3637, 3638, 3639,\n 3640, 3641, 3642, 3643, 3644,\n 3645, 3646, 3647, 3648, 3649,\n 3650, 3651, 3652, 3653, 3654,\n 3655, 3656, 3657, 3658, 3659,\n 3660, 3661, 3662, 3663, 3664,\n 3665, 3666, 3667, 3668, 3669,\n 3670, 3671, 3672, 3673, 3674,\n 3675, 3676, 3677, 3678, 3679,\n 3680, 3681, 3682, 3683, 3684,\n 3685, 3686, 3687, 3688, 3689,\n 3690, 3691, 3692, 3693, 3694,\n 3695, 3696, 3697, 3698, 3699,\n 3700, 3701, 3702, 3703, 3704,\n 3705, 3706, 3707, 3708, 3709,\n 3710, 3711, 3712, 3713, 3714,\n 3715, 3716, 3717, 3718, 3719,\n 3720, 3721, 3722, 3723, 3724,\n 3725, 3726, 3727, 3728, 3729,\n 3730, 3731, 3732, 3733, 3734,\n 3735, 3736, 3737, 3738, 3739,\n 3740, 3741, 3742, 3743, 3744,\n 3745, 3746, 3747, 3748, 3749,\n 3750, 3751, 3752, 3753, 3754,\n 3755, 3756, 3757, 3758, 3759,\n 3760, 3761, 3762, 3763, 3764,\n 3765, 3766, 3767, 3768, 3769,\n 3770, 3771, 3772, 3773, 3774,\n 3775, 3776, 3777, 3778, 3779,\n 3780, 3781, 3782, 3783, 3784,\n 3785, 3786, 3787, 3788, 3789,\n 3790, 3791, 3792, 3793, 3794,\n \n 1800,1801,1802,1803,1804,\n 1805,1806,1807,1808,1809,\n 1810,1811,1812,1813,1814,\n 1815,1816,1817,1818,1819,\n 1820,1821,1822,1823,1824,\n 1825,1826,1827,1828,1829,\n 1830,1831,1832,1833,1834,\n 1835,1836,1837,1838,1839,\n 1840,1841,1842,1843,1844,\n 1845,1846,1847,1848,1849,\n 1850,1851,1852,1853,1854,\n 1855,1856,1857,1858,1859,\n 1860,1861,1862,1863,1864,\n 1865,1866,1867,1868,1869,\n 1870,1871,1872,1873,1874,\n 1875,1876,1877,1878,1879,\n 1880,1881,1882,1883,1884,\n 1885,1886,1887,1888,1889,\n 1890,1891,1892,1893,1894,\n 1895,1896,1897,1898,1899,\n \n 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, \n 2011, 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020,\n 2021, 2022, 2023, 2024, 2025, 2026, 2027, 2028, 2029, 2030,\n 2031, 2032, 2033, 2034, 2035, 2036, 2037, 2038, 2039, 2040,\n 2041, 2042, 2043, 2044, 2045, 2046, 2047, 2048, 2049, 2050,\n 2051, 2052, 2053, 2054, 2055, 2056, 2057, 2058, 2059, 2060,\n 2061, 2062, 2063, 2064, 2065, 2066, 2067, 2068, 2069, 200,\n 2071, 2072, 2073, 2074, 2075, 2076, 2077, 2078, 2079, 2080,\n 2081, 2082, 2083, 2084, 2085, 2086, 2087, 2088, 2089, 2090,\n 2091, 2092, 2093, 2094, 2095, 2096, 2097, 2098, 2099,\n \n \n 3500, 3501, 3502, 3503, 3504, 3505, 3506, 3507, 3508, 3509,\n 3510, 3511, 3512, 3513, 3514, 3515, 3516, 3517, 3518, 3519,\n 3520, 3521, 3522, 3523, 3524, 3525, 3526, 3527, 3528, 3529,\n 3530, 3531, 3532, 3533, 3534, 3535, 3536, 3537, 3538, 3539,\n 3540, 3541, 3542, 3543, 3544, 3545, 3546, 3547, 3548, 3549,\n 3550, 3551, 3552, 3553, 3554, 3555, 3556, 3557, 3558, 3559,\n 3560, 3561, 3562, 3563, 3564, 3565, 3566, 3567, 3568, 3569,\n 3570, 3571, 3572, 3573, 3574, 3575, 3576, 3577, 3578, 3579,\n 3580, 3581, 3582, 3583, 3584, 3585, 3586, 3587, 3588, 3589,\n 3590, 3591, 3592, 3593, 3594, 3595, 3596, 3597, 3598, 3599,\n \n 2600, 2601, 2602, 2603, 2604, 2605, 2606, 2607 ,2608, 2609,\n 2610, 2611, 2612, 2613, 2614, 2615, 2616, 2617, 2618, 2619,\n 2620, 2621, 2622, 2623, 2624, 2625, 2626, 2627, 2628, 2629,\n 2630, 2631, 2632, 2633, 2634, 2635, 2636, 2637, 2638, 2639 , \n 2640, 2641, 2642, 2643, 2644, 2645, 2646, 2647, 2648, 2649,\n 2650, 2651, 2652, 2653, 2654, 2655, 2656, 2657, 2658, 2659,\n 2660, 2661, 2662, 2663, 2664, 2665, 2666, 2667, 2668, 2669,\n 2670, 2671, 2672, 2673, 2674, 2675, 2676, 2677, 2678, 2679,\n 2680, 2681, 2682, 2683, 2684, 2685, 2686, 2687, 2688, 2689,\n 2690, 2691, 2692, 2693, 2694, 2695, 2696, 2697, 2698, 2699,\n \n 2200,2201,2202,2203,2204,2205,2206,2207,2208,2209,2210,\n 2211,2212,2213,2214,2215,2216,2217,2218,2219,2220,\n 2221,2222,2223,2224,2225,2226,2227,2228,2229,2230,\n 2231,2232,2233,2234,2235,2236,2237,2238,2239,2240,\n 2241,2242,2243,2244,2245,2246,2247,2248,2249,2250,\n 2251,2252,2253,2254,2255,2256,2257,2258,2259,2260,\n 2261,2262,2263,2264,2265,2266,2267,2268,2269,2270,\n 2271,2272,2273,2274,2275,2276,2277,2278,2279,2280,\n 2281,2282,2283,2284,2285,2286,2287,2288,2289,2290,\n 2291,2292,2293,2294,2295,2296,2297,2298,2299\n \n ]),\n \n 'offset': np.array([\n# #2700 - 2709\n# [ 120, -150, 10, 32, 10, 10, 30, 0, 20, 0],\n\n# [30, 10, 20, 20, 20, 30, 10, 30, 10, 30],\n\n# [ 20,0, 10, 20, 50, -170, 30, 30, 30,10],\n\n# [-120, 30, 10, 10, 20, 10, 40, 30, 40, 40],\n\n# [ 50,10 , 10, 50, 50, 10, 20, 10, 0, 40],\n\n# [ 100, 10, 10, 20, 10, 180, 0, 0, 10, 0],\n \n# [40,10, -20,20,20, 30, 20,10,10,20],\n \n# [40,-150, 10,20,30, 40,-60,10,20,10],\n \n# [0,30,10,0,10, 10,20,30,30,30],\n \n# [50,10,10,30,10, 10,30,40,20,0],\n \n# # 2710 - 2719\n \n# [ 40, 180, 10, 32, 30, 20, -10, -30, -10, -50],\n\n# [50, 10, 20, 20, 30, 40, 0, 0, 40, -20],\n\n# [ 20,50, 10, 20, 50, 180, 10, -10, 30,10],\n\n# [ 0, 50, 20, 10, 20, 20, 0, -10, -10, 40],\n\n# [ 0, 50, 10, 30, 10, 20, 20, 0, 0, 40],\n\n# [ 70, 30, 10, 20, 50, 50, 10, 10, 20, 0],\n \n# [50,10, 10,30,20, 20, 20,10,10,20],\n \n# [50,30, 10,20,30, 10,-20,10,20,10],\n \n# [10,10,10,20,10, 10,20,-10,30,30],\n \n# [30,10,10,30,10, 10,30,-10,20,20],\n \n# #2720 - 2729\n# [ 20, 20, 10, 20, 30, 20, -10, -30, -10, -50],\n\n# [60, 10, 10, 10, 30, 40, 0, 0, 40, -20],\n\n# [ 70,20, 10, 10, 50, 180, 10, -10, 30,10],\n\n# [ 90, 20, 10, 20, 20, 20, 0, -10, -10, 40],\n\n# [ 50, 20, 10, 30, 10, 20, 20, 0, 0, 40],\n\n# [ 70, -170, 10, 30, 50, 50, 10, 10, 20, 0],\n \n# [20,10, 10,30,20, 20, 20,10,10,20],\n \n# [-150,50, 10,20,30, 10,-20,10,20,10],\n \n# [20,50,10,30,10, 10,20,-10,30,30],\n \n# [-50,40,180,-20,10, 10,30,-10,20,20],\n \n# #2730 - 2739\n# [ 20, -120, 10, 32, 30, 50, 0, 10, 10, 20],\n\n# [60, -140, 10, 20, 30, 0, -50, 0, -10, 20],\n\n# [ 70,10, 20, 20, 50, 0, 50, 10, 10,10],\n\n# [ 90, 20, 20, 10, 20, 20, -20, 10, 10, 0],\n\n# [ 50, 40, -10, 30, 10, 10, -20, 0, 20, 0],\n\n# [ 70, 40, 0, 20, 50, 20, 40, 0, 10, 20],\n \n# [20,40, -10,30,20, 10, 20,10,10,10],\n \n# [-150,50, 10,20,30, 10,20,0,20,-10],\n \n# [20,-130,10,20,10,20,20,10,30,0],\n \n# [-50,30,-10,30,10, -150,-30,10,30,-20],\n \n# #2740 - 2749\n# [ 50, 40, 0, 0, 0, 10, 10, 0, 20, -10],\n\n# [60, 20, -10, 10, 40, 20, 10, 0, 30, 30],\n\n# [ -40,50, 20, 10, 20, 20, 10, -10, 0,-10],\n\n# [ -80, 20, 10, 10, -10, 20, 10, -20, 0, 20],\n\n# [ -10, 10, 10, 20, 10, -10, 10, 0, 20, -10],\n\n# [ 40, 30, 0, 10,10, 0, 10, -20, 10, 0],\n \n# [50,40, 0,20,10, 30, 10,-20,-140,-10],\n \n# [50,40, -20,20,20, 20,10,0,10,-10],\n \n# [30, 10,0,20,50,10,10,0,20,-20],\n \n# [50,40,0,30,20, 20,10,100,30,-10],\n \n \n# #2750 - 2759\n# [ 20, 10, 0, 10, 10, 10, 10, -10, 30, 0],\n\n# [80, 30, 20, 20, 20, 10, 10, 10, 0, -20],\n\n# [ 40,20, 10, 10, 10, -10, -10, 10, -20,-10],\n\n# [ -150, 30, -10, 20, 20, 20, -30, 10, 0, -20],\n\n# [ 50, 20, -10, -10, 0, 10, 0, -10, 30, -20],\n\n# [ 50, 10, 10, 30, 50, 10, -10, 20, 40, -10],\n \n# [10,40, 30,20,40, 10, 20,10,-20,-10],\n \n# [0,30, 20,10,30, 10,30,30,170,0],\n \n# [30,20,0,-20,40, 10,-20,10,20,0],\n \n# [50,30,10,20,10, 10,0,30,20,140],\n \n# #2760 - 2769\n# [ 30, 50, 10, 0, 10, 10, 20, -10, 30, -10],\n\n# [50, 50, 20, -10, 10, 0, 10, 10, -30, -10],\n\n# [ 10,50, 20, 20, 10, 10, -30, 10, 10,-10],\n\n# [ 70, 40, 30, 20, 20, 20, -30, 10, 10, -10],\n\n# [ 60, 20, 10, 10, 10, 50, -30, -10, 40, 10],\n\n# [ 20, 180, 20, 0, -10, 10, 0, 20, 10, -10],\n \n# [30,10, -10,-10,10, 30, -10,10,30,0],\n \n# [20,30, 10,10,30, 10,-30,30,30,-20],\n \n# [20,20,0,10,40, 10,-10,10,40,-10],\n \n# [20,20,10,10,30, 20,-10,30,10,10],\n \n# #2770 - 2779\n# [ 30, 30, 10, 0, 10, 10, 20, -10, 30, -10],\n\n# [50, -10, 20, -10, 10, 0, 10, 10, -170, 0],\n\n# [ 10,40, 20, 20, 10, 10, -30, 10, 0,0],\n\n# [ 70, 20, 30, 20, 20, 20, -30, 10, -40, 10],\n\n# [ 60, 20, 10, 10, 10, 50, -30, -10, 10, 10],\n\n# [ 20, 10, 20, 0, -10, 10, 0, 20, 20, 10],\n \n# [30,10, -10,-10,10, 30, -10,10,10,20],\n \n# [20,10, 10,10,30, 10,-30,30,-10,30],\n \n# [20,-130,0,10,40, 10,-10,10,30,30],\n \n# [20,50,10,10,30, 20,-10,30,10,10],\n \n# #2780 - 2789\n# [ 0, 10, 10, 0, 10, 10, 20, -10, 30, -10],\n\n# [50, 10, 20, 20, 10, 0, 10, 10, -170, 0],\n\n# [ 10,40, 20, 20, 10, 10, -30, 10, 40,0],\n\n# [ 70, 10, -10, 20, 20, 20, -30, 10, 40, 10],\n\n# [ 60, 20, 0, 10, 10, 50, -30, -10, 10, 10],\n\n# [ 20, 20, 10, 0, -10, 180, 0, 20, 20, 10],\n \n# [30,30, 20,20,10, 180, -10,10,10,20],\n \n# [20,30, 20,10,30, 10,-30,30,-10,30],\n \n# [20,30,10,10,40, 10,-10,10,30,30],\n \n# [20,20,10,10,30, 20,-10,30,10,10],\n \n \n# #2790 - 2799\n# [ 20, -120, 10, 32, 30, 40, -10, -30, -10, -50],\n\n# [60, -140, 10, 20, 30, 10, 0, 0, 40, -20],\n\n# [ 70,10, 20, 20, 50, 30, 10, -10, 30,10],\n\n# [ 90, 20, 20, 10, 20, 10, 0, -10, -10, 40],\n\n# [ 50, 40, -10, 30, 10, 20, 20, 0, 0, 40],\n\n# [ 70, 40, 0, 20, 50, 10, 10, 10, 20, 0],\n \n# [20,40, -10,30,20, 20, 20,10,10,20],\n \n# [-150,50, 10,20,30, 10,-20,10,20,10],\n \n# [20,-130,10,20,10, 10,20,-10,30,30],\n \n# [-50,30,-10,30,10, 20,30,-10,20,20],\n \n \n [50, 5, 18, 0, 10, -10, -15, 0, 0, 19], # \n [10, 10, -9, -15, 26, 0, -20, -10, 30, 1], # \n [-125, -9, -5, -5, 27, -20, -25, 15, -13, 5],\n [10, -185, -10, -25, 28, 0, 0, 0, -5, 15], # \n [0, 5, -3, -2, 19, 0, -5, 15, 21, 15], # \n [-40, 20, 0, 5, 0, 0, 0, 0, -2, 10], # \n [35, 19, -3, 0, 15, 0, 0, 5, 0, -15], # \n [30, 10, 0, 0, 4, 5, 0, 11, 1, -8], # \n [20, -18, -11, 12, 2, 2, 0, 0, 15, 15],\n [30, -3, -1, 1, 15, -5, -3, 5, -1, 15],\n [1, 13, -2, 0, 15, 0, 0, 0, 0, 0], \n [0, -11, -5, -9, -14, 1, -8, 0, -3, 20],\n [0, -150, 0, 0, 25, 0, 0, 0, 4, -10], \n [5, 25, 0, 0, 15, 0, -5, -5, 0, 9], \n [10, 13, -5, -3, 0, 0, 0, 0, 1, 0], \n [0, 7, 0, 3, 10, 0, -10, 5, 0, 5], \n [-15, 30, 0, 0, 0, 1, -5, -5, 0, 15],\n [10, 10, 0, 0, 0, 0, -55, 0, 5, 13], \n [0, 37, -10, -5, 10, 0, -6, 0, 0, 5], \n [0, -30, -5, 1, 15, 0, -40, 0, 12, -3],\n [0, 40, 0, -3, 13, 0, 0, 5, 0, 7], \n [0, 40, -5, 0, 0, 0, 0, 0, 0, 12], \n [0, 3, 0, 0, 17, 0, 0, 0, 0, 18], \n [30, 15, -4, -15, 0, 0, -3, 5, 0, 17], \n [45, 5, -5, -5, -4, -5, 0, 0, 10, 10], \n [0, -3, -4, 12, 0, 0, -3, 0, 8, 0], \n [0, 10, -2, -3, 8, 0, -53, 0, 0, 17], \n [15, 30, -5, 0, 0, 0, 0, 0, 19, -1], \n [90, 5, -5, 0, 12, 0, -5, 0, 0, 10], \n [30, 27, 0, 0, 9, 0, 0, 0, 0, 2], \n [-8, 11, 2, 8, 5, 1, 0, 0, -1, 3],\n [15, -13, 5, 0, 29, 1, 1, 7, 5, 13], \n [0, 9, -2, 0, 17, 5, -1, -5, 1, 17], \n [25, 0, -1, -1, 30, 3, -40, 0, -4, 15],\n [0, 18, 0, -1, 15, 1, 0, -5, 7, 1], \n [30, 0, 0, -5, 6, -2, -15, 7, -17, 2],\n [10, 1, 0, 0, 3, -1, 0, 0, -1, 1], \n [8, 1, 0, 0, 5, 1, 1, 0, -3, -14], \n [0, 13, 1, 0, 31, 3, -15, 0, -20, 0], \n [90, 17, 2, 3, 29, -1, -2, 0, 15, -3],\n [40, 35, -4, -2, 6, 9, 0, 0, 1, 16], \n [1, 0, -5, 9, -2, 2, 0, 5, 0, -14], \n [5, 19, -11, -4, -1, -4, -15, 10, 3, 7],\n [3, 37, -5, -6, 2, -1, -15, 0, 0, 20], \n [10, 9, -7, -2, 14, 1, 0, 0, 0, 25], \n [0, 4, -5, -20, 17, 0, -1, 0, 1, -1],\n [0, 5, 0, -15, 24, -21, -15, 0, 1, 18],\n [0, 7, -1, 0, 0, 1, 0, 0, -6, -1], \n [9, 15, -5, 0, 5, 5, 0, 0, 4, 43], \n [60, 23, -2, 1, 27, 0, 0, -5, 17, 11],\n [12, -2, -7, 0, 15, -7, 0, 0, 5, 10],\n [1, 18, -1, 1, 5, 1, 0, -5, 6, 3], \n [55, -5, -2, 0, 0, 1, -6, -3, -12, 13], \n [19, 17, -5, 0, -5, 0, 0, 12, 14, 8], \n [10, -2, -3, -18, 0, 1, -35, 1, 15, 12], \n [10, -5, -1, 3, 10, 0, -15, -5, -17, -5], \n [30, 40, -10, -5, 19, 17, -48, -1, 16, -9], \n [25, 35, -8, -20, 5, 7, 0, -1, 1, 20], \n [0, 0, 0, -3, 0, 0, 0, -1, 4, 0], \n [1, 7, 0, 0, 27, -5, -3, -5, 29, 3], \n [0, 14, 0, 0, -5, 0, -15, 0, -2, 1], \n [0, 13, -3, 21, 11, 0, -13, 0, 1, -37], \n [0, 12, 0, 0, 0, 0, -1, 0, 0, -10], \n [0, 28, -2, 0, -1, 5, -23, -7, 5, 30], \n [0, 0, -6, 15, 17, 0, -25, -1, 2, -3], \n [0, 7, -9, 1, 0, 0, 0, 1, 5, 7], \n [7, 7, 0, 15, 21, 11, -8, -5, 8, 0],\n [20, 1, -3, 3, 0, 1, 0, -3, -8, 0], \n [80, -12, -10, -2, 0, 0, 0, 0, 0, 9], \n [-10, 31, -19, -1, 1, 0, -4, 0, 1, 39], \n [40, 40, -18, 0, 23, 0, -5, -4, -2, 21],\n [0, 5, 0, 13, 20, 0, -9, 0, 10, 0], \n [8, 25, 0, -2, 18, 5, -3, 0, 5, 3], \n [60, -5, 0, 0, -2, -4, 0, 0, 0, 15],\n [15, 15, 20, -2, 9, 0, 0, 0, 0, 5], \n [50, 18, -8, -10, 13, 0, 0, 0, 3, 9],\n [40, 27, 0, -3, 7, 5, -40, 0, 5, 0],\n [10, 5, 4, 0, 0, 20, 0, 0, 10, -4], \n [-3, 5, -1, 0, -1, 0, 5, 0, -5, 1], \n [0, 15, -5, 9, 4, -5, 0, 0, 1, 33], \n [15, 3, 0, 0, 15, 0, 0, 0, -8, 33], \n [40, 0, 0, -15, -9, 3, -10, 0, 9, 1],\n [0, 0, 0, 0, 0, 0, -8, 3, 5, 19], \n [0, 20, 0, 0, 0, 0, -10, -3, 5, 0],\n [65, 10, 0, 0, 0, 0, -5, 0, 4, 5], \n [3, 38, -5, 0, 17, 0, 0, 7, 3, 9], \n [10, 25, 3, 10, 24, 0, 5, -5, -2, 20], \n [0, 19, -5, 5, 8, 0, 3, 0, 1, -9], \n [60, 5, -3, 3, 20, 1, 0, 0, 3, 30], \n [60, 0, 0, 0, 40, 5, 0, -5, 0, 15], \n [0, -1, 0, -3, 9, 3, 1, -1, 0, 29], \n [30, 33, 0, -5, 15, 3, 0, 0, 1, -1],\n [0, 35, 0, -5, 0, 0, -5, 2, 0, 0], \n [0, 18, 2, -1, 0, 0, -2, 0, 16, 19], \n [25, 0, -5, -3, -2, -1, 0, -1, 0, 5], \n\n [-110,35,-10,-10,15,20,0,0,-10,30],\n [50,30,23,0,30,0,10,20,0,20],\n [0,30,20,30,10,0,25,30,10,10],\n [60,0,0,20,-10,4,-12,20,0,30],\n [-5,0,0,-10,25,0,0,20,20,0],\n \n \n [125,40,10,20,35, \n 30,5,25,50,35],\n [0,25,20,0,50, \n 35,-10,0,15,50],\n [50,45,10,40,30, \n 30,0,10,20,50],\n [55,25,15,30,50, \n 20,20,30,40,20],\n [10,20,20,30,50, \n 20,5,20,40,10],\n [15,23,10,20,45, \n 10,5,25,17,15],\n [50,30,10,20,10, \n 35,-10,0,25,10],\n [10,20,40,10,40, \n 50,50,10,20,15],\n [25,25,15,0,40, \n 20,50,20,20,40],\n [50,50,10,10,50, \n 20,5,0,20,50],\n #0-10\n [70,35,25,40,40, \n 20,20,15,30,37],\n [40,50,20,7,20, \n 30,0,20,10,20],\n [30,50,20,20,20, \n 40,20,40,20,30],\n [40,20,20,20,40, \n 30,20,10,20,30],\n [20,20,20,20,20, #14\n 30,30,20,20,35],\n [40, 60, 10, 20, 60, \n 10, 30, 30, 30, 40],\n [40, 44, 10, 20, 30,\n 20, 20, 20, 25, 20],\n [50, 20, 10, 40, 30,\n 40, -10, 0, 30, 20],\n [-100, 40, 30, 40, 20,\n 30, 60, 0, 30, 50],\n [0, 10, 20, 0, 10,\n 30, 40, 20, 30, 30],\n # 10-20\n [50, 40, 10, 10, 30,\n 20,-20, 20, 20, 30],\n [20, 20, 20, 20, 60,\n 60, 0, 20, 30, 50],\n [30, 30, 20, 30, 30,\n 50, 0, 20, 20, 30],\n [0, 50, 0, 40, 30,\n 50, 0, 20, 30, 30],\n [-20, 20, 20, 0, 20, \n 40, 0, 20, 30, 0], \n [0, 30, 20, 20, 20,\n 40, 20, 10, 30, 30],\n [70, 30, 10, 0, 30,\n 40, 30, 20, 50, 0],\n [50, 20, 30, 30, 20,\n 50, 20, 0, 40, 20],\n [-30,30, 30, 20, 40,\n 20, 20, 20, 40, 40],\n [50, 50, 20, 20, 40,\n 20, 20, 20, 30, 50],\n #20-30\n [60, 20, 20, 20, 30, #30\n 30, 20, 0, 20, 20],\n [0, 30, 20, 20, 40,\n 40, -10, 30, 30,50],\n [50, 40,20, 30, 50,\n 10, 30, 30, 30, 30],\n [0, 20, 50, 0, 50,\n 30, 27, 30, 30, 30],\n [60, 44, 30, 50, 50, #34\n 20, 10, 30, 30, 50], \n [30, 30, 20, 40, 40,\n 40, 30, 10, 10, 20],\n [30, 40, 30, 20, 40,\n 30, 20, 20, 40, 30],\n [0, 30, 20, 20, 30,\n 30, 0, 30, 40, 50],\n [0, 50, 20, 30, 40,\n 20, 20, 20, 30, 50],\n [60, 50, 20, 40, 40,\n 20, 20, 40, 20, 40],\n #30-40\n [-10, 20, 30, 10, 10,\n 50, 0, 30, 20, 30],\n [60, 30, 20, 20, 30,\n 40, 30, 30, 40, 10],\n [100, 30, 10, 30, 30,\n 50, 50, 30, 10, 40],\n [50, 30, 30, 30, 40,\n 10, -10, 20, 20, 40],\n [20, 20, 10, 20, 40,\n 30, 10, 20, 30, 0], #44\n [35, 40, 30, 30, 30,\n 40, 0, 30, 25, 30],\n [0, 20, 20, 30, 60,\n 40, 0, 30, 20, 20],\n [40, 20, 30, -10, 60,\n 30, 0, 40, 30, 40],\n [60, 60, 20, 40, 70,\n 30,20, 30, 20, 20],\n [70, 10, 30, 30, 20,\n 40, 0, 50, 20, 40],\n #40-50\n [40, 30, 30, 50, 20,\n 10, 20, 40, 40, 40],\n [30, 40, 40, 0, 10,\n 10, 0, 20, 30, 50],\n [40, 10, 10, 0, 50,\n 50, 20, 30, 40, 20],\n [0, 40, 30, 30, 40,\n 0, 20, 10, 20, 30],\n [50, 40, 0, 10, 40,\n 50, 6, 40, 30, 20], #54\n [50, 30, 10, 0, 30,\n 30, 30, 20, 20, 40],\n [40, 10, 30, 30, 20,\n 0, -20, 30, 30, 40],\n [30, 40, 30, 0, 10,\n 60, -10, 30, 45, 40],\n [40, 30, 30, 30, 10,\n 30, 30, 10, 30, 20],\n [80, 0, 0, 20, 50,\n 40, 30, 20, 30, 60],\n #50-60\n [10, 50, 10, 30, 30,\n 30, 20, 10, 20, 30],\n [50, 0, 20, 30, 10,\n 50, 0, 30, 40, 30],\n [20, 30, 10, 10, 30,\n 10, -10, 20, 20, 10],\n [40, 20, 20, 30, 30,\n 30, 0, 20, 30, 30],\n [30, 40, 40, 0, 30,\n 40, 20, 20, 10, 10], #64\n [30, 40, 20, 30, 40,\n 10, 10, 20, 10, 30],\n [40, 30, 20, 20, 50,\n 50, 30, 30, 40, 30],\n [40, 30, 10, 10, 20,\n 40, 20, 30, 30, 0],\n [30, 10, 20, 30, 50,\n 50, -10, 30, 40, 30],\n [50, 50, 30, 20, 40,\n 50, 20, 40, 30, 40],\n #60-70\n [40, 40, 10, 10, 20,\n 20, 10, 10, 30, 50],\n [30, 40, 30, 30, 40,\n 20, 10, 0, 40, 50],\n [0, 10, 20, 30, 20,\n 40, 30, 20, 40, 50],\n [50, 20, 30, 40, 50,\n 40, 30, 30, 30, 0],\n [30, 30, 10, 20, 60,\n 50, 20, 30, 10, 40], #74\n [70, 40, 30, 20, 40,\n 40, 0, 20, 0, 0],\n [30, 40, 30, 20, 50,\n 40, 10, 30, 20, 40],\n [60, 35, 10, 20, 20,\n 50, 10, 10, 20, 50],\n [40, 30, 0, 10, 30,\n 30, 20, 20, 30, 50],\n [30, 50, 20, 10, 10,\n -10, -10, 30, 10, 40],\n #70-80\n [60, 10, 20, 30, 0,\n 0, 20, 20, 20, 30],\n [40, 40, 10, 10, 30,\n 60, -20, 0, 50, 30],\n [60, 30, 30, 30, 50,\n 30, 0, 20, 30, 30],\n [50, 40, 30, 30, 30,\n 40, 30, 40, 40, 30],\n [40, 30, 30, 20, 20,\n 40, -20, 20, 10, 30], #84\n [30, 55, 40, 40, 60,\n 30, 0, 30, 30, 30],\n [90, 10, 20, 20, 30,\n 60, 10, 20, 50, 50],\n [40, 30, 20, 50, 20,\n 50, 10, 40, 20, 30],\n [40, 50, 40, 20, 20,\n 30, 10, 20, 40, 30],\n [60, 25, 20, 0, 0,\n 15, 10, 20, 30, 40],\n #80-90\n [30, 50, 30, 10, 40,\n 40, 20, 20, 40, 40],\n [50, 40, 50, 40, 60,\n 60, -10, 20, 30, 50],\n [20, 50, 40, 10, 50,\n 30, 30, 40, 20, 40],\n [10, 50, 20, 40, 50,\n 40, -10, 30, 40, 60],\n [70, 30, 30, 20, 40,\n 10, 10, 20, 10, 50], #94\n [60, 30, 20, 0, 50,\n 60, 0, 20, 20, 30],\n [50, 60, 10, 10, 20,\n -10, 0, 30, 30, 60],\n [80, 10, 10, 30, 0,\n 20, 0, 30, 50, 50],\n [20, 50, 20, 10, 40,\n 30, 20, 20, 30, 30],\n [60, 50, 30, 10, 60,\n 10, 10, 20, 30, 30],\n #90-100\n [30, 30, 30, 40, 30,\n 40, 30, 30, 40, 40],\n [40, 30, 20, 30, 30,\n 30, 10, 20, 40, 20],\n [50, 10, 20, 20, 30,\n 40, 0, 20, 30, 20],\n [40, 20, 10, 20, 30,\n 40, 30, 20, 30, 40],\n [50, 40, 40, 20, 30,\n 30, 30, 30, 30, 30], #104\n #\n [15,23,10,20,45, \n 10,5,25,17,35],\n [0,40,20,0,20, \n 15,0,0,25,10],\n [10,5,10,10,50, \n 10,10,10,20,55],\n [45,25,15,0,10, \n 20,20,0,20,20],\n [50,10,20,10,10, \n 20,5,0,20,50],\n [40,-5,25,40,10, \n 20,20,15,30,37],\n [0,20,20,27,20, \n 30,0,20,10,50],\n [0,50,0,20,40, \n 30,-10,40,10,10],\n [0,50,20,20,30, \n 20,20,10,20,50],\n [20,45,20,20,44, #114\n 30,0,20,20,25],\n [20, 15, 30, 30, 20, \n 30, 0, 0, 30, 20],\n [0, 40, 30, 20, 40,\n 20, 0, 20, -5, 30],\n [0, 20, 10, 20, 10,\n 20, 30, 30, 30, 40],\n [-150, 40, 30, 10, 40,\n 30, 30, 0, 30, 50],\n [0, 50, 0, 0, 20,\n 30, 40, 30, 0, 20],\n [30, 20, 20, 10, 40,\n 30,-20, 0, 20, 30],\n [20, 20, 30, 0, 60,\n 30, 20, 20, 30, 10],\n [0, 30, 20, 30, 30,\n 30, 0, 20, 20, 0],\n [0, 20, 20, 40, 30,\n 40, 40, 20, 30, 20],\n [20, 20, 40, 0, 20, #124\n 20, -30, 20, 20, 30], \n [0, 40, 40, 0, 30,\n 40, 40, 0, 50, 50],\n [50, 30, 10, 0, 20,\n 20, 30, 50, 30, 30],\n [20, 40, 20, 30, 20,\n 40, 30, 0, 20, 0],\n [0, 30, 0, 20, 30,\n 20, 20, 20, 40, 40],\n [50, 20, 30, 40, 0,\n 20, 20, 20, 30, 30],\n [60, 60, 20, 20, 30,\n 30, -30, 20, 20, 40],\n [0, 40, 20, 0, 30,\n 20, -20, 30, 20,30],\n [50, 40,30, 30, 20,\n 10, 30, 0, 30, 30],\n [0, 40, 30, 0, 50,\n 30, 27, 0, 30, 0],\n [50, 30, 30, 20, 20, #134\n 20, 10, 0, 30, 30], \n [30, 40, 30, 10, 20,\n 40, 30, 10, 10, 0],\n [30, 30, 30, 0, 40,\n 30, 20, 20, 40, 30],\n [0, 40, 20, 30, 30,\n 30, 0, 30, 20, 20],\n [0, 30, 30, 10, 40,\n 20, 20, 0, 10, 30],\n [60, 30, 30, 20, 20,\n 40, 20, 30, 10, 30],\n [30, 30, 30, 30, 50,\n 30, 0, 30, 20, 0],\n [30, 0, 20, 20, 30,\n 40, 30, 30, 40, 30],\n [60, 20, 30, 30, 30,\n 50, 50, 30, 30, 30],\n [50, 30, 30, 30, 0,\n 30, -10, 20, 20, 40],\n [20, 20, 40, 0, 50,\n 30, 10, 20, 0, 0], #144\n [35, 50, 30, 30, 30,\n 40, 0, 30, 30, 30],\n [0, 20, 0, 30, 30,\n 40, 0, 30, 20, 40],\n [0, 40, 30, 30, 40,\n 30, 0, 40, 30, 30],\n [30, 30, 20, 10, 30,\n 30,-10, 30, 20, 0],\n [30, 40, 30, 0, 30,\n 40, -30, 30, 40, 40],\n [0, 50, 30, 10, 30,\n 30, 20, 20, 40, 10],\n [30, 50, 40, 40, 30,\n 30, 0, 0, 30, 30],\n [20, 50, 40, 40, 20,\n 40, 0, 30, 40, 20],\n [0, 40, 40, 30, 40,\n 30, 20, 10, 20, 5],\n [30, 30, 30, 30, 40,\n 40, 6, 30, 30, 40], #154\n [50, 30, 40, 0, 40,\n 30, 30, 10, 20, 30],\n [40, 30, 30, 30, 40,\n 40, 20, 30, 20, 40],\n [30, 40, 30, 40, 40,\n 40, 30, 30, 0, 40],\n [40, 40, 30, 30, 30,\n 30, 30, 20, 40, 30],\n [40, 40, 30, 20, 30,\n 40, 0, 20, 30, 40],\n [10, 40, 40, 30, 30,\n 40, 20, 10, 20, 30],\n [0, 40, 40, 30, 40,\n 40, 30, 10, 40, 30],\n [20, 30, 30, 40, 30,\n 40, -10, 20, 20, 40],\n [0, 30, 30, 30, 30,\n 30, 0, 20, 30, 30],\n [30, 20, 20, 0, 30,\n 20, 0, 20, 30, 30], #164\n [10, 40, 30, 30, 30,\n 30, 0, 0, 40, 30],\n [0, 30, 20, 20, 40,\n 10, 30, 30, 40, 30],\n [40, 30, 30, 30, 40,\n 40, 20, 30, 30, 40],\n [30, 40, 40, 30, 30,\n 40, 10, 30, 40, 30],\n [50, 40, 30, 30, 40,\n 30, 0, 20, 30, 30],\n [40, 30, 30, 30, 30,\n 30, 20, 10, 30, 20],\n [30, 50, 30, 30, 20,\n 20, 10, 30, 40, 30],\n [0, 50, 20, 40, 40,\n 40, 0, 30, 0, 40],\n [20, 30, 30, 40, 30,\n 40, 0, 30, 40, 40],\n [0, 30, 30, 30, 30,\n 40, 20, 30, 30, 40], #174\n [50, 50, 0, 20, 40,\n 40, 0, 20, 0, 30],\n [30, 40, 30, 20, 20,\n 40, 10, 30, 20, 0],\n [30, 30, 20, -10, 40,\n 30, -10, 10, 20, 30],\n [40, 30, 30, 40, 40,\n 30, 30, 20, 30, 20],\n [40, 20, 20, 10, 40,\n 40, -10, 0, 40, 40],\n [40, 40, 40, 40, 40,\n 40, 30, 20, 40, 30],\n [40, 40, 40, 40, 30,\n 30, -20, 30, 20, 30],\n [40, 30, 30, 30, 30,\n 30, 20, 20, 30, 30],\n [50, 40, 30, 30, 30,\n 40, 30, 40, 40, 30],\n [40, 30, 30, 20, 40,\n 40, -20, 20, 10, 30], #184\n [30, 30, 10, 20, 40,\n 30, 0, 30, 30, 20],\n [50, 30, 20, 20, 30,\n 10, 20, 10, 30, 30],\n [40, 40, 30, 20, 30,\n 40, -10, 10, 20, 30],\n [40, 30, 0, 0, 20,\n 30, 10, 20, 10, 30],\n [30, 30, 20, 30, 30,\n 30, 30, 20, 30, 40],\n [30, 30, 30, 30, 30,\n 20, 20, 40, 40, 40],\n [50, 40, 30, 0, 20,\n 30, 20, 20, 30, 30],\n [20, 30, 20, 10, 30,\n 30, 0, 0, 20, 10],\n [10, 20, 20, 10, 30,\n 40, 0, 10, 40, 30],\n [40, 30, 30, 0, 40,\n 40, 30, 20, 10, 30], #194\n [30, 30, 20, 30, 30,\n 40, 0, 10, 20, 20],\n [10, 50, 30, 10, 10,\n 30, 0, 30, -10, 30],\n [40, 40, 50, 30, 40,\n 30, -20, 30, 30, 30],\n [20, 20, 20, 10, 40,\n 30, 20, 20, 30, 30],\n [30, 30, 30, 10, 30,\n 10, 10, 20, 10, 40],\n [30, 30, 30, 40, 30,\n 40, 30, 30, 40, 40],\n [40, 30, 20, 30, 30,\n 30, 10, 20, 40, 20],\n [50, 10, 20, 20, 30,\n 40, 0, 20, 30, 20],\n [40, 20, 10, 20, 30,\n 40, 30, 20, 30, 40],\n [50, 40, 40, 20, 30,\n 30, 30, 30, 30, 30], #204\n [30, 20, 30, 30, 40, \n 30, 20, 20, 20, 20],\n [40, 40, 20, 20, 30, \n 40, 10, 30, 20, 30],\n [20, 30, 20, 40, 40,\n 40, 20, 30, 20, 20],\n [30, 40, 40, 0, 20,\n 40, 0, 20, 20, 30],\n [50, 40, 30, 20, 30, \n 30, 0, 20, 20, 30],\n [30, 30, 30, 20, 30, \n 30, 20, 20, 20, 20],\n [50, 40, 30, 30, 40,\n 40, 0, 20, 10, 40],\n [50, 20, 30, 30, 20,\n 30, 20, 10, 30, 20],\n [10, 30, 30, 10, 0,\n 40, 30, 30, 30, 30],\n [20, 30, 20, 30, 20,\n 30, -20, 20, 40, 20],#214\n [40, 30, 20, 20, 30, \n 40, -10, 30, 30, 40],\n [50, 30, 30, 30, 30,\n 20, 10, 30, 30, 30],\n [20, 20, 30, 0, 20,\n 20, 0, 10, 40, 40],\n [30, 30, 20, 30, 30,\n 30, 20, 30, 30, 20],\n [40, 40, 20, 30, 20, \n 30, 0, 20, 30, 30],\n [40, 30, 30, 30, 30,\n 30, 20, 20, 30, 30],\n [40, 30, 20, 30, 10, \n 30, 10, 20, 10, 10],\n [40, 40, 40, 30, 30, \n 40, 30, 10, 30, 20],\n [30, 30, 30, 30, 20,\n 50, 30, 30, 30, 30],\n [20, 30, 20, 20, 30, \n 40, 30, 0, 20, 30],\n [50, 30, 20, 20, 30, #225\n 30, 0, 20, 20, 40],\n [40, 30, 30, 20, 10, \n 30, 0, 20, 30, 30],\n [30, 40, 30, 30, 40,\n 30, 20, 10, 30, 30],\n [10, 0, 0, 30, 40,\n 30, 0, 30, 10, 30],\n [30, 30, 20, 20, 10,\n 30, -10, 20, 0, 30],\n [0, 0, 30, 10, 20, \n 40, 0, 30, 30, 20],\n [-30, 20, 20, 0, 10, \n 30, 30, 20, 30, 40],\n [40, 40, 40, 0, 40, \n 40, 30, 20, 30, 30],\n [0, 30, 10, 10, 20, \n 40, 20, 10, 30, 10],\n [50, 10, 10, 20, 40,\n 30, 30, 20, 30, 30],\n [-50, 30, 20, 10, 30, \n 30, 0, 20, 40, 20],\n [30, 30, 20, 30, 30, \n 40, 30, 10, 10, 10],\n [30, 40, 30, 20, 40, #237\n 30, 30, 10, 30, 0],\n [40, 40, 30, 30, 40,\n 40, -20, 20, 40, 10],\n [0, 30, 20, 20, 40, \n 20, 0, 20, 20, 20],\n [30, 30, 20, -10, 30, \n 30, 0, 20, 30, 20],\n [30, 20, 10, 10, 30,\n 30, 0, 30, 30, 30],\n [30, 0, 30, 20, 30,\n 20, 20, 10, 0, 20],\n [40, 30, 10, 10, 10, \n 20, 0, 30, 50, 40],\n [30, 30, 20, 0, 40, \n 30, 0, 30, 20, 30],\n [40, 30, 20, 30, 40, \n 40, 20, 20, 20, 30],\n [50, 40, 20, 20, 30, \n 30, 20, 0, 10, 0],\n [40, 10, 20, 10, 20, \n 40, 0, 30, 30, 0],\n [40, 40, 30, 20, 30,\n 30, 0, 30, 20, 0],\n [30, 40, 30, 40, 30, #249\n 20, 0, 20, 30, 0],\n [30, 30, 20, 20, 40, \n 40, 30, 20, 10, 30],\n [30, 30, 10, 0, 30, \n 40, 0, 40, 20, 10],\n [30, 20, 10, 30, 40,\n 30, -20, 20, 20, 20],\n [30, 20, 20, 10, 40,\n 20, 20, 20, 30, 0],\n [30, 40, 30, 0, 10,\n 10, 0, 10, 30, 20],\n [30, 20, 20, 0, 30, #255\n 30, 0, 20, 0, 30],\n [30, 30, 30, 30, 30, \n 20, 0, 20, 0, 0],\n [40, 20, 10, 10, 30,\n 30, 20, 10, 20, 0],\n [30, 30, 20, 30, 40, \n 40, 0, 30, 10, 20],\n [20, 10, 20, 20, 20, \n 30, 20, 20, 20, 0],\n [0, 10, 10, 30, 30,\n 40, 20, 10, 10, 0],\n [20, 20, 10, 20, 30,\n 40, 30, 30, 40, 30],\n [30, 30, 20, 10, 20, \n 40, 20, 20, 10, 30],\n [50, 40, 30, 40, 40, \n 40, -10, 20, 30, 30],\n [30, 30, 20, 30, 20, \n 30, 0, 20, 30, 20],\n [20, 0, 20, 0, 10, #265\n 40, -10, 0, 10, 30],\n [0, 0, 20, 0, 30, \n 30, 20, 20, 10, 0],\n [10, 20, 10, 20, 30,\n 30, 10, 20, 10, 10],\n [30, 30, 20, 10, 40,\n 40, 20, 20, 0, 30],\n [30, 40, 30, 20, 10,\n 40, 0, 20, 40, 30],\n [0, 50, 40, 40, 40,\n 20, 0, 20, 30, 0],\n [20, 30, 20, 20, 20, \n 20, 0, 20, 30, 20],\n [30, 40, 20, 20, 40,\n 30, 20, 20, 30, 20],\n [40, 30, 30, 30, 30, \n 30, 30, 30, 30, 30],\n [30, 30, 30, 20, 40,\n 40, 10, 20, 30, -10],\n [30, 10, 10, 20, 20, #275\n 20, 0, 30, 0, 40],\n [0, 20, 20, 10, 30,\n 20, 10, 30, 10, 10],\n [30, 30, 20, 20, 40,\n 40, 10, 10, 20, 20],\n [30, 20, 20, 30, 40,\n 40, 30, 20, 30, 40],\n [30, 30, 30, 20, 30, \n 30, -20, 20, 20, 40],\n [30, 30, 30, 30, 40, \n 40, 0, 20, 20, 30],\n [30, 30, 30, 20, 30, \n 40, 20, 30, 30, 30],\n [50, 30, 20, 30, 30, \n 40, 30, 20, 20, 0],\n [50, 40, 40, 30, 30,\n 30, 30, 20, 40, 30],\n [30, 40, 30, 30, 40, \n 40, 30, 30, 30, 0],\n [10, 10, 0, 10, 20, #285\n 40, 20, 10, 30, 0],\n [0, 10, 10, 0, 20, \n 30, 0, 10, 20, 0],\n [30, 35, 30, 30, 20,\n 30, -10, 20, 20, 10],\n [40, 30, 20, 0, 30,\n 40, 0, 30, 30, 0],\n [40, 40, 20, 0, 30, #284\n 30, 20, 20, 30, 20],\n [30, 30, 30, 30, 30,\n 20, 20, 40, 40, 40],\n [20, 40, 30, 0, 20,\n 30, 20, 20, 30, 30],\n [20, 30, 20, 10, 30,\n 30, 0, 10, 30, 20],\n [20, 30, 20, 30, 40,\n 50, 20, 10, 40, 20],\n [50, 50, 10, 0, 40,\n 40, -30, 30, 20, 30], #294\n [50, 40, 20, 30, 40,\n 40, 10, 20, 20, 20],\n [10, 20, 30, 10, 10,\n 40, 10, 30, 20, 30],\n [40, 40, 30, 20, 40,\n 20, -20, 20, 30, 30],\n [30, 20, 20, 20, 40,\n 30, 20, 20, 30, 30],\n [50, 30, 30, 30, 10,\n 20, 10, 20, 10, 40],\n \n [-15,30,5,0,25, \n 35,-12,0,15,5],\n [-135,5,0,38,25, \n 45,-10,5,5,8],\n [10,36,0,-5,15, \n 24,-20,0,36,18],\n [0,24,7,0,35, \n 25,-9,4,0,30],\n [20,0,0,6,25, \n 17,-32,0,0,18],\n \n [-16,27,-11,5,0, \n 14,-14,-15,-30,25],\n [48,19,-5,3,5, \n 28,-25,5,15,-10],\n [38,34,-15,15,19, \n 10,0,0,-5,17],\n [52,0,0,0,0, \n -6,-20,0,0,0],\n [0,0,10,20,30, \n 10,0,-7,17,50],\n \n [20,28,0,0,5, \n 10,15,0,33,10],\n [0,28,-5,0,5, \n 0,-4,7,6,-3],\n [10,20,5,0,17, \n 15,2,8,6,14],\n [27,6,7,4,6, \n 0,-25,-23,25,0],\n [45,20,5,0,10, \n 17,-25,0,25,7],\n \n [4,35,14,8,5, \n 7,0,-5,15,7],\n [0,31,0,27,25, \n 20,-25,20,20,-6],\n [85,20,-12,0,10, \n 10,0,0,13,14],\n [45,0,0,0,0, \n 4,0,0,0,37],\n [0,43,-7,15,15, \n 0,-77,5,-5,6],\n \n #21-25\n \n [35,25,-12,7,5, \n 16,25,0,20,28],\n [-110,3,-3,5,15, \n 25,-20,-5,3,29],\n [10,20,5,5,47, \n 0,10,-4,2,40],\n [-137,10,-7,0,5, \n 10,-22,5,-5,10],\n [0,10,0,10,14, \n 3,-13,0,8,48],\n \n #26-30\n \n [-15,24,-5,9,5, \n 15,-10,-5,20,15],\n [4,32,2,-4,0, \n 12,-25,15,-6,20],\n [30,30,4,20,-7, \n 10,-7,0,15,17],\n [45,15,0,0,0, \n -13,-10,0,0,16],\n [0,0,27,10,25, \n 10,-30,0,24,-10],\n \n #31-35\n \n [5,-2,3,-30,23, \n 18,-35,5,-5,25],\n [-82,0,0,-7,42, \n -10,-350,20,10,-1],\n [10,27,0,40,0, \n 17,10,27,0,40],\n [45,10,-10,0,0, \n 0,-15,0,24,15],\n [0,33,0,5,17, \n 10,-25,0,35,17],\n \n #36-40\n \n [34,53,2,0,5, \n 20,-35,-10,11,18],\n [0,22,-17,0,0, \n 20,15,10,17,5],\n [30,-12,-18,18,10, \n 20,0,0,-10,27],\n [50,20,0,0,10, \n 30,0,7,10,-5],\n [42,45,0,5,25, \n 10,0,28,5,33],\n \n #40-45\n \n [-5,20,-5,37,15, \n 10,-35,0,15,25],\n [0,0,0,15,0, \n 15,-55,0,10,10],\n [-105,40,0,-7,0, \n 0,-10,0,20,0],\n [30,33,-5,0,0, \n 20,-15,0,10,-10],\n [45,10,0,18,10, \n -5,-25,0,27,10],\n \n #46-50\n \n [5,30,10,-8,12, \n 20,0,-10,16,5],\n [47,20,-45,0,0, \n 20,-25,0,10,32],\n [30,27,5,-7,2, \n 17,-8,0,0,25],\n [45,20,-25,20,17, \n 5,-15,0,0,0],\n [48,14,0,10,25, \n 10,0,8,25,45],\n \n #51-55\n \n [-140,15,-5,0,0, \n 10,35,0,-30,30],\n [45,3,0,10,0, \n 15,-10,0,10,10],\n [30,17,7,0,0, \n 17,0,14,20,38],\n [45,4,-5,8,0, \n 0,-15,0,40,45],\n [0,1,0,0,10, \n 20,-25,0,20,21],\n \n #56-60\n \n [-10,18,0,7,5, \n 20,-20,25,10,0],\n [13,10,5,7,10, \n 3,-25,5,5,0],\n [55,36,-4,5,25, \n 17,-36,0,28,14],\n [45,30,0,0,42, \n 30,0,0,0,0],\n [0,5,0,10,25, \n 10,0,0,12,20],\n \n #61-65\n \n [12,35,-5,8,-5, \n 10,-7,0,12,20],\n [236,0,0,0,0, \n 45,-5,0,10,-20],\n [30,0,18,-4,40, \n 36,0,10,20,30],\n [45,35,22,0,0, \n 7,-15,10,5,15],\n [0,2,0,0,10, \n 23,17,0,-5,33],\n \n #65-70\n \n [-20,24,-10,7,11, \n 15,-15,-10,0,0],\n [29,7,10,0,0, \n 20,-25,0,10,20],\n [30,-9,10,10,10, \n 10,5,0,-10,-15],\n [24,23,8,9,3, \n -2,-10,10,16,40],\n [15,47,0,10,18, \n 10,0,0,11,-10],\n \n #71-75\n \n [50,0,-5,-7,20, \n 20,-35,30,-10,15],\n [0,38,0,-10,30, \n 3,0,4,10,3],\n [20,27,0,0,20, \n 0,-10,0,10,20],\n [30,27,10,-5,0, \n 16,-30,10,15,16],\n [7,18,0,0,26, \n 19,-20,0,7,0],\n \n #76-80\n \n [12,52,0,0,40, \n 20,16,-5,15,40],\n [50,53,-10,8,13, \n 9,-25,-16,10,0],\n [24,7,15,-10,16, \n 10,-10,13,15,3],\n [28,33,23,7,52, \n 35,-13,16,34,0],\n [10,46,8,13,20, \n 2,8,0,17,34],\n \n #81-85\n \n [-5,14,3,22,4, \n 10,-35,0,5,20],\n [50,17,0,0,6, \n 47,27,0,23,5],\n [50,13,0,0,20, \n 25,0,10,40,20],\n [30,10,6,0,7, \n 0,-15,0,10,30],\n [0,12,0,5,10, \n 10,-12,0,27,8],\n \n #86-90\n \n [-20,25,-10,6,-12, \n 20,20,-10,-5,-8],\n [-90,39,7,0,13, \n 30,-25,0,23,0],\n [30,15,7,30,10, \n 10,15,40,0,18],\n [45,65,20,0,0, \n -13,-33,0,15,14],\n [20,25,-8,10,30, \n 10,20,0,13,0],\n \n #91-95\n \n [97,22,0,18,-8, \n 10,-15,0,0,18],\n [35,26,-10,-5,8, \n 33,-10,-8,27,10],\n [50,43,0,-6,30, \n 40,-5,0,20,16],\n [55,27,0,0,5, \n 40,0,5,10,0],\n [20,18,0,6,10, \n 10,-25,10,0,17],\n \n [230,19,-4,10,5, \n 14,0,10,6,-15],\n [0,4,0,30,17, \n 20,0,0,5,44],\n [50,14,-5,30,10, \n 20,10,5,10,8],\n [10,18,10,4,18, \n 8,-13,0,28,5],\n [-18,28,0,0,25, \n 10,0,17,15,-10],\n \n \n \n #####\n [0,0,0,-15,0, \n 20,0,20,0,0],\n #1\n [0,0,0,-15,0, \n 20,-65,0,-15,0],\n #2\n [0,20,0,-20,15, \n 0,-55,0,0,-20],\n #3\n [0,30,0,0,10, \n 15,-65,0,0,0],\n #4\n [-90,10,0,-20,0, \n 15,-60,0,-5,10],\n #5\n [0,30,-20,0,20, \n 0,-20,0,-30,-30],\n #6\n [0,-11,0,0,22, \n 0,-40,0,10,0],\n #7\n [0,0,0,0,0, \n 15,-70,-10,0,20],\n #8\n [0,24,0,0,7, \n 0,-70,-12,0,10],\n #9\n [20,20,-5,0,0, \n 7,-16,0,10,0],\n #10\n [30,10,0,0,20, \n 5,-10,0,20,0],\n #11\n [0,0,0,0,0, \n 0,5,0,0,10],\n #12\n [20,30,0,0,20, \n 10,20,-5,-5,10],\n #13\n [20,30,0,15,15, \n 15,-5,0,25,10],\n #14\n [5,20,10,0,0, \n 15,0,0,20,0],\n #15\n [30,40,0,0,0, \n 0,-10,10,-5,10],\n #16\n [10,-10,10,20,10, \n 0,0,0,20,0],\n #17\n [10,30,-10,0,15, \n 20,8,15,10,0],\n #18\n [10,10,0,0,15, \n 20,0,0,-10,20],\n #19\n [10,40,0,-10,0, \n 10,0,0,-10,0],\n #20\n [20,0,0,20,0, \n 0,20,0,10,20],\n #21\n [20,13,0,11,25, \n 25,-20,0,0,10],\n #22\n [20,-12,0,0,-5, \n 0,0,-7,-5,12],\n #23\n [5,-7,5,-7,0, \n 2,0,15,-3,3],\n #24\n [50,5,-5,3,0, \n 0,0,0,0,5],\n #25\n [20,5,0,-7,0, \n 7,-10,-7,8,0],\n #26\n [0,20,0,-15,0, \n 10,-40,0,5,5],\n #27\n [25,-5,-4,-15,5, \n 15,8,4,5,4],\n #28\n [0,0,0,15,0, \n 15,-10,12,10,0],\n #29\n [0,25,0,13,30, \n 19,-90,7,0,-6],\n #30\n [0,43,0,17,0, \n 27,-23,0,5,0],\n #31\n [20,10,0,0,10, \n 10,-90,-10,10,30,],\n #32\n [10,20,0,0,10, \n 20,30,0,20,0,],\n #33\n [-10,0,0,10,0, \n 10,0,15,20,-10,],\n #34\n [10,-20,10,-10,15, \n 0,25,0,0,30,],\n #35\n [0,10,10,-20,30, \n 10,0,0,20,10,],\n #36\n [0,25,10,0,10, \n 10,0,15,10,10,],\n #37\n [20,8,8,0,0, \n 10,10,0,25,0,],\n #38\n [0,30,-10,10,-5, \n 15,-20,-8,40,10,],\n #39\n [5,25,10,0,0, \n 0,-90,0,0,15,],\n #40\n [35,7,0,20,20, \n 10,0,-10,8,15,],\n #41\n [0,20,0,0,15, \n 20,-105,0,3,0,],\n #42\n [-15,25,-15,-7,15, \n -17,-100,15,15,25,],\n #43\n [-10,25,0,20,10, \n -5,-100,-7,-10,10,],\n #44\n [-10,43,0,-3,20, \n 17,-90,0,0,30],\n #45\n [10,8,0,-10,0, \n 0,-90,-10,15,15,],\n #46\n [0,20,20,13,20, \n 0,0,0,-15,10,],\n #47\n [-3,7,0,0,0, \n 0,0,0,0,25,],\n #48\n [0,0,0,5,17, \n 13,-30,0,-10,15,],\n #49\n [20,-8,-10,-5,0, \n 25,0,7,13,-8,],\n #50\n [-90,25,-13,0,-4, \n 10,0,13,0,10,],\n #51\n [-90,25,-3,0,25, \n 25,-90,-10,0,30,],\n #52\n [-7,-7,10,-5,33, \n 19,-90,0,16,14,],\n #53\n [13,-5,0,0,30, \n 7,-24,0,5,15,],\n #54\n [25,0,-5,7,5, \n 0,0,-4,-4,30],\n #55\n [40,-5,20,10,0, \n 0,0,0,-7,10,],\n #56\n [-40,25,0,-5,10, \n 20,0,-30,10,25,],\n #57\n [0,10,10,-5,0, \n 5,30,0,30,0,],\n #58\n [35,-10,-5,0,-15, \n 20,0,0,-10,10,],\n #59\n [15,5,0,17,16, \n 20,-20,0,0,20,],\n #60\n [-20,40,0,-15,10, \n 10,0,0,6,10,],\n #61\n [20,30,-5,15,20, \n -3,-10,0,-4,20,],\n #62\n [4,10,10,10,20, \n 0,10,0,17,0,],\n #63\n [20,30,10,-10,0, \n 10,-5,0,-10,20,],\n #64\n [0,30,0,0,20, \n 10,0,0,0,15,],\n #65\n [0,20,0,0,0, \n 0,0,0,-5,10,],\n #66\n [0,25,10,-10,5, \n 4,5,0,10,0,],\n #67\n [5,15,0,7,10, \n 0,0,5,5,0,],\n #68\n [13,2,-3,0,20, \n 10,5,5,10,0,],\n #69\n [10,7,0,5,15, \n 0,30,-5,5,10,],\n #70\n [0,5,10,0,20, \n 13,0,0,5,-3,],\n #71\n [20,3,2,-4,-10, \n 0,0,0,5,0,],\n #72\n [20,15,8,0,10, \n 7,5,-3,0,5,],\n #73\n [0,0,20,5,10, \n 0,0,0,5,25,],\n #74\n [10,5,0,3,0, \n 10,0,0,5,0,],\n #75\n [10,-10,3,0,30, \n 0,-20,10,10,-5,],\n #76\n [25,-3,4,-5,20, \n -10,10,3,30,20,],\n #77\n [20,2,6,0,-3, \n 15,0,0,-5,20,],\n #78\n [45,0,0,5,8, \n 0,0,0,5,15,],\n #79\n [7,4,5,-5,10, \n -5,0,0,15,10,],\n #80\n [-5,3,0,2,2, \n 7,-10,-59,2,15,],\n #81\n [5,5,5,3,-10, \n 10,-6,0,-5,10,],\n #82\n [0,30,0,5,10, \n 7,0,0,20,10,],\n #83\n [20,0,20,0,10, \n 0,0,0,5,0,],\n #84\n [10,0,-7,10,15, \n 10,-10,10,5,0,],\n #85\n [0,-5,0,0,-25, \n 10,15,15,10,5,],\n #86\n [30,-20,-10,0,5, \n 10,0,0,0,0,],\n #87\n [0,4,0,0,30, \n 0,20,0,30,-8,],\n #88\n [20,-6,0,0,6, \n 4,0,0,3,15,],\n #89\n [20,3,-5,3,10, \n 10,5,0,3,-10,],\n #90\n [15,-12,0,6,20, \n 4,0,0,4,15,],\n #91\n [15,-3,5,-3,0, \n 15,0,0,-5,0,],\n #92\n [7,3,-5,-5,20, \n 50,-10,-7,10,0,],\n #93\n [10,7,0,0,-7, \n 10,-70,-7,8,9,],\n #94\n [32,20,-10,0,25, \n 27,-70,0,8,-15,],\n #95\n [30,-8,0,4,4, \n 5,0,0,8,12,],\n #96\n [20,20,0,0,0, \n 0,-60,0,10,0,],\n #97\n [0,7,-5,-5,0, \n 10,0,0,0,5,],\n #98\n [20,-7,0,0,0, \n -30,-70,7,-5,0,],\n #99\n [0,0,-3,0,0, \n 0,0,-5,0,0,],\n\n \n ####\n [-15,0,5,-10,15,-15,-20,0,10,-10], #1\n [40,0,0,0,0,20,-60,0,10,0], #2\n [10,0,10,0,20,20,-5,-5,-20,0], #3\n [0,5,20,-10,0,0,20,0,10,30], #4\n [30,35,-10,-5,20,0,-25,0,0,25], #5\n \n [40,10,0,10,5,5,-50,30,5,30], #6\n [20,35,0,-15,0,0,-50,0,10,0], #7\n [10,30,0,0,0,25,-30,5,5,10],#8\n [15,5,-5,-10,20,10,0,0,0,5],#9\n [0,20,0,0,0,0,0,0,0,0],#10\n \n [-5,30,-15,0,30,-10,-20,0,20,25],#11\n [0,0,0,0,0,-5,-10,0,10,10],#12\n [10,5,0,0,0,0,10,0,20,0],#13\n [45,10,-20,0,0,0,-15,0,-35,0],#14\n [0,-5,0,0,10,10,-25,0,20,0],#15\n \n [0,12,0,0,5,20,0,0,0,0],#16\n [0,20,0,0,0,0,-80,0,10,-10],#17\n [0,0,0,0,0,0,0,0,0,10],#18\n [0,0,0,0,0,0,0,0,0,0],#19\n [0,0,0,0,0,0,0,0,0,0],#20\n \n [-5,0,5,0,15,10,-35,0,20,25],#21\n [0,0,0,0,0,-5,-10,0,10,10],#22\n [10,5,0,0,0,0,10,0,20,0],#23\n [45,10,-20,0,0,0,-15,0,-35,0],#24\n [0,-5,0,0,10,10,-25,0,20,0],#25\n \n [0,12,0,0,5,20,0,0,0,0],#26\n [0,20,0,0,0,0,-80,0,10,-10],#27\n [0,0,0,0,0,0,0,0,0,10],#28\n [0,0,0,0,0,0,0,0,0,0],#29\n [0,0,0,0,0,0,0,0,0,0],#30\n \n [-5,0,5,0,15,10,-35,0,20,25],#31\n [0,0,0,0,0,-5,-10,0,10,10],#32\n [10,5,0,0,0,0,10,0,20,0],#33\n [45,10,-20,0,0,0,-15,0,-35,0],#34\n [0,-5,0,0,10,10,-25,0,20,0],#35\n \n [0,12,0,0,5,20,0,0,0,0],#36\n [0,20,0,0,0,0,-80,0,10,-10],#37\n [0,0,0,0,0,0,0,0,0,10],#38\n [0,0,0,0,0,0,0,0,0,0],#39\n [0,0,0,0,0,0,0,0,0,0],#40\n \n [-5,0,5,0,15,10,-35,0,20,25],#41\n [0,0,0,0,0,-5,-10,0,10,10],#42\n [10,5,0,0,0,0,10,0,20,0],#43\n [45,10,-20,0,0,0,-15,0,-35,0],#44\n [0,-5,0,0,10,10,-25,0,20,0],#45\n \n [0,12,0,0,5,20,0,0,0,0],#46\n [0,20,0,0,0,0,-80,0,10,-10],#47\n [0,0,0,0,0,0,0,0,0,10],#48\n [0,0,0,0,0,0,0,0,0,0],#49\n [0,0,0,0,0,0,0,0,0,0],#50\n \n [-5,0,5,0,15,10,-35,0,20,25],#51\n [0,0,0,0,0,-5,-10,0,10,10],#52\n [10,5,0,0,0,0,10,0,20,0],#53\n [45,10,-20,0,0,0,-15,0,-35,0],#54\n [0,-5,0,0,10,10,-25,0,20,0],#55\n \n [0,12,0,0,5,20,0,0,0,0],#56\n [0,20,0,0,0,0,-80,0,10,-10],#57\n [0,0,0,0,0,0,0,0,0,10],#58\n [0,0,0,0,0,0,0,0,0,0],#59\n [0,0,0,0,0,0,0,0,0,0],#60\n \n [-5,0,5,0,15,10,-35,0,20,25],#61\n [0,0,0,0,0,-5,-10,0,10,10],#62\n [10,5,0,0,0,0,10,0,20,0],#63\n [45,10,-20,0,0,0,-15,0,-35,0],#64\n [0,-5,0,0,10,10,-25,0,20,0],#65\n \n [0,12,0,0,5,20,0,0,0,0],#66\n [0,20,0,0,0,0,-80,0,10,-10],#67\n [0,0,0,0,0,0,0,0,0,10],#68\n [0,0,0,0,0,0,0,0,0,0],#69\n [0,0,0,0,0,0,0,0,0,0],#70\n \n [-5,0,5,0,15,10,-35,0,20,25],#71\n [0,0,0,0,0,-5,-10,0,10,10],#72\n [10,5,0,0,0,0,10,0,20,0],#73\n [45,10,-20,0,0,0,-15,0,-35,0],#74\n [0,-5,0,0,10,10,-25,0,20,0],#75\n \n [0,12,0,0,5,20,0,0,0,0],#76\n [0,20,0,0,0,0,-80,0,10,-10],#77\n [0,0,0,0,0,0,0,0,0,10],#78\n [0,0,0,0,0,0,0,0,0,0],#79\n [0,0,0,0,0,0,0,0,0,0],#80\n \n [-5,0,5,0,15,10,-35,0,20,25],#81\n [0,0,0,0,0,-5,-10,0,10,10],#82\n [10,5,0,0,0,0,10,0,20,0],#83\n [45,10,-20,0,0,0,-15,0,-35,0],#84\n [0,-5,0,0,10,10,-25,0,20,0],#85\n \n [0,12,0,0,5,20,0,0,0,0],#86\n [0,20,0,0,0,0,-80,0,10,-10],#87\n [0,0,0,0,0,0,0,0,0,10],#88\n [0,0,0,0,0,0,0,0,0,0],#89\n [0,0,0,0,0,0,0,0,0,0],#90\n \n [-5,0,5,0,15,10,-35,0,20,25],#91\n [0,0,0,0,0,-5,-10,0,10,10],#92\n [10,5,0,0,0,0,10,0,20,0],#93\n [45,10,-20,0,0,0,-15,0,-35,0],#94\n [0,-5,0,0,10,10,-25,0,20,0],#95\n \n [0,12,0,0,5,20,0,0,0,0],#96\n [0,20,0,0,0,0,-80,0,10,-10],#97\n [0,0,0,0,0,0,0,0,0,10],#98\n [0,0,0,0,0,0,0,0,0,0],#99\n [0,0,0,0,0,0,0,0,0,0],#100 \n \n ######\n \n [0,30,0,20,20, \n 40,-28,20,10,0],\n #1\n [25,32,0,-15,25, \n 10,-15,0,15,15],\n #2\n [0,20,0,5,15, \n 8,-55,0,8,8],\n #3\n [10,-5,0,5,10, \n 15,-30,0,11,0],\n #4\n [-90,-10,0,-20,3, \n 15,20,0,15,-10],\n #5\n [20,-10,5,0,20, \n 30,-20,0,30,20],\n #6\n [0,10,0,0,22, \n 5,-5,0,5,0],\n #7\n [-10,0,0,0,0, \n 15,-10,-10,0,20],\n #8\n [0,-5,0,0,-2, \n 0,-10,2,0,10],\n #9\n [20,20,-5,4,4, \n 7,-16,0,-2,0],\n #10\n [30,30,-5,0,20, \n 10,-10,0,5,0],\n #11\n [30,10,0,-10,0, \n 0,-10,0,0,-15],\n #12\n [20,30,0,5,20, \n 10,-30,-5,10,-10],\n #13\n [20,30,-5,-10,-20, \n -15,5,0,25,10],\n #14\n [5,-7,-10,0,15,\n -10,-20,0,5,10],\n #15\n [-10,15,10,0,10, \n 0,-10,10,-5,10],\n #16\n [5,10,-10,-10,-5, \n 20,0,0,5,0],\n #17\n [40,10,-10,0,15, \n 20,-5,15,10,0],\n #18\n [10,-5,0,0,15, \n -5,-5,0,5,-10],\n #19\n [30,-5,0,-10,5, \n 10,-40,0,10,0],\n #20\n [20,0,0,-20,0, \n 0,20,0,-10,20],\n #21\n [40,-10,0,-11,-10, \n -10,-20,0,0,10],\n #22\n [20,12,0,0,-5, \n 0,0,-7,-10,15],\n #23\n [5,25,5,-10,15, \n 30,0,-15,15,-25],\n #24\n [-10,5,-5,-10,5, \n 5,-30,10,0,5],\n #25\n [-5,10,0,-7,0, \n 7,15,-7,8,0],\n #26\n [10,20,0,-15,10, \n 20,40,0,5,20],\n #27\n [25,5,-4,-15,5, \n 25,-8,-4,12,-10],\n #28\n [0,10,0,8,5, \n -10,10,12,10,0],\n #29\n [10,25,-5,-13,-30, \n 19,-50,7,0,-6],\n #30\n [0,-10,0,-10,-5, \n -27,-23,0,5,0],\n #31\n [20,-5,0,0,10, \n 15,-10,5,-10,30,],\n #32\n [10,5,0,0,-10, \n 20,-30,-20,5,0,],\n #33\n [10,19,0,-2,10, \n -2,0,15,20,-10,],\n #34\n [10,10,10,-10,-10, \n 0,-13,0,10,30,],\n #35\n [0,-2,10,-20,30, \n 10,-4,0,-2,10,],\n #36\n [20,-5,-2,-2,10, \n -2,-4,15,10,10,],\n #37\n [20,8,8,-2,4, \n 10,-10,0,25,0,],\n #38\n [0,-20,-10,-10,-5, \n -5,-20,10,2,-5,],\n #39\n [-5,-5,-5,0,5, \n 5,-10,0,5,15,],\n #40\n [35,7,0,20,20, \n 10,0,-10,8,15,],\n #41\n [20,20,-4,3,5, \n 25,-15,0,-30,2,],\n #42\n [-15,-10,10,-7,15, \n 4,5,-4,15,-10,],\n #43\n [-10,4,0,-5,15, \n 10,-25,4,10,-10,],\n #44\n [-10,25,0,-3,-5, \n -5,10,0,8,-10],\n #45\n [-5,-5,0,10,0, \n -5,-10,-10,-10,15,],\n #46\n [5,25,20,-4,-15, \n 0,-9,0,15,10,],\n #47\n [5,-4,0,0,0, \n 5,-40,0,0,25,],\n #48\n [5,-5,0,-4,-5, \n 13,30,0,-10,15,],\n #49\n [20,20,5,5,0, \n -5,-10,7,13,-8,],\n #50\n [-110,25,-5,0,-4, \n 10,-10,13,20,-10,],\n #51\n [-145,-25,-3,0,25, \n 25,-20,-10,0,-20,],\n #52\n [7,-7,-4,-5,33, \n -10,-40,0,20,-14,],\n #53\n [13,-10,0,10,-30, \n 10,20,0,5,20,],\n #54\n [25,15,-5,-13,5, \n 10,0,-4,-7,10],\n #55\n [40,2,-10,10,0, \n 0,0,0,2,10,],\n #56\n [30,2,0,3,10, \n 20,0,-20,-5,25,],\n #57\n [0,16,-15,-5,7, \n 5,-10,0,-30,0,],\n #58\n [35,20,-5,9,-15, \n 20,-5,0,2,2,],\n #59\n [-5,10,0,-5,16, \n -5,20,0,0,20,],\n #60\n [10,-2,0,-15,10, \n 25,-9,0,-7,-2,],\n #61\n [26,8,-5,-7,4, \n 3,-15,0,-4,20,],\n #62\n [25,11,10,14,20, \n 10,10,0,-3,0,],\n #63\n [20,30,10,-10,0, \n 10,-42,0,-11,-5,],\n #64\n [0,-5,0,0,20, \n 10,-2,0,-7,15,],\n #65\n [3,-5,0,2,10, \n 3,0,0,-5,10,],\n #66\n [-4,-12,-5,-10,8, \n 8,-70,0,10,0,],\n #67\n [5,-5,0,-10,10, \n 0,0,5,5,0,],\n #68\n [13,2,-3,-10,20, \n 10,5,5,-5,0,],\n #69\n [10,9,0,8,15, \n 5,-5,-5,10,10,],\n #70\n [0,-13,10,0,20, \n 13,-10,0,5,3,],\n #71\n [-3,-3,-5,4,10, \n 10,0,0,-5,0,],\n #72\n [47,-5,-5,0,20, \n -5,5,-3,15,5,],\n #73\n [0,10,20,10,15, \n 0,0,0,7,25,],\n #74\n [10,30,0,3,-2, \n 10,0,3,30,5,],\n #75\n [17,10,3,0,-2, \n 0,20,10,20,-10,],\n #76\n [25,5,-4,-5,20, \n 10,10,3,2,2,],\n #77\n [20,2,6,22,-3, \n 15,-85,0,-5,22,],\n #78\n [45,10,0,-5,8, \n 0,-2,0,-5,15,],\n #79\n [12,4,5,-5,10, \n -5,0,5,15,13,],\n #80\n [-5,3,0,2,2, \n 7,-10,-10,2,15,],\n #81\n [5,15,5,3,-10, \n 10,-6,0,15,12,],\n #82\n [20,-3,5,5,-10, \n -5,2,0,-10,12,],\n #83\n [20,0,20,2,10, \n 0,0,10,5,0,],\n #84\n [10,0,-7,-3,-5, \n 10,-12,12,7,-9,],\n #85\n [10,20,0,-3,25, \n -10,15,-5,10,10,],\n #86\n [30,-5,10,0,10, \n 10,0,10,0,0,],\n #87\n [5,4,0,-2,-10, \n 5,-10,0,-10,10,],\n #88\n [25,22,0,3,6, \n 4,0,0,3,15,],\n #89\n [20,3,-5,3,10, \n 10,5,0,3,10,],\n #90\n [15,14,0,6,20, \n 4,-45,0,4,15,],\n #91\n [15,-3,5,3,2, \n -4,0,5,5,0,],\n #92\n [7,-5,-5,5,-5, \n 10,-10,-7,-10,0,],\n #93\n [10,7,0,0,4, \n 10,5,-7,8,9,],\n #94\n [50,20,5,0,5, \n 3,-35,0,8,-15,],\n #95\n [38,8,-3,4,25, \n 7,-5,5,-5,12,],\n #96\n [20,2,0,3,0, \n 7,7,-5,10,-10,],\n #97\n [30,10,-5,3,20, \n -6,4,0,12,-5,],\n #98\n [25,5,0,-25,5, \n 5,0,-2,-7,0,],\n #99\n [-40,25,-3,0,0, \n 0,-2,-5,0,0],\n \n ####\n \n [10,10,-5,0,5,\n -15,-35,0,20,25],\n [0,-5,0,20,10,\n 15,-30,0,-5,-10],\n [40,5,0,0,0,\n 0,-10,0,0,0],\n [45,10,0,0,0,\n 0,-25,0,-5,0],\n [0,20,0,0,10,\n 10,-25,0,20,0],\n \n [20,30,0,0,0,\n 0,-20,-20,0,15],\n [0,15,0,-5,25,\n 0,-5,0,0,0],\n [0,5,0,0,0,\n 0,0,0,10,10],\n [0,10,0,0,0,\n 0,5,0,-15,0],\n [0,15,0,0,0,\n 0,-5,0,5,0],\n \n [-10,20,-5,0,-5,\n 10,-35,0,10,5],\n [0,30,0,0,0,\n -5,-10,0,10,10],\n [40,0,0,0,10,\n 0,-10,0,20,0],\n [45,10,-20,0,10,\n 5,-15,0,-5,10],\n [0,0,0,0,10,\n 10,0,0,5,0],\n \n [-15,-5,-5,0,5,\n 10,5,0,20,-15],\n [0,0,0,5,5,\n -5,-10,0,10,20],\n [10,5,0,0,20,\n 0,-10,0,50,20],\n [25,0,-20,0,0,\n -5,-35,0,5,5],\n [10,10,0,5,20,\n 10,-25,-5,5,5],\n \n [5,5,-5,0,5,\n 10,-35,0,15,25],\n [0,0,-10,0,15,\n 5,-30,0,10,10],\n [55,0,0,0,5,\n 0,10,0,10,10],\n [55,-10,-20,0,5,\n 10,5,0,-5,5],\n [10,-10,0,0,-5,\n 10,-25,0,40,5],\n \n [-5,10,-5,-10,5,\n 10,-15,0,15,25],\n [0,-5,0,0,5,\n -5,-10,-5,10,25],\n [15,0,0,0,-5,\n 0,-15,0,20,-5],\n [45,5,-5,-5,5,\n 0,-15,-5,5,5],\n [0,-5,0,0,10,\n 10,-30,0,15,5],\n \n [-5,15,-5,0,5,\n 15,-30,0,30,5],\n [10,5,0,5,0,\n 5,-15,0,5,10],\n [30,15,0,-5,0,\n 0,10,0,10,20],\n [25,10,-10,0,0,\n 0,-15,0,-5,5],\n [10,0,0,0,5,\n 0,-25,0,15,0],\n \n [-40,15,-5,0,-15,\n 10,-5,0,10,20],\n [0,15,0,-5,20,\n 0,-15,0,10,10],\n [40,5,0,5,0,\n 0,10,0,10,-5],\n [20,25,-10,0,10,\n 0,-15,0,-5,5],\n [0,-5,0,0,10,\n 10,5,0,5,5],\n \n [-5,10,-5,-5,5,\n 10,-35,0,30,5],\n [0,0,-5,0,5,\n -5,-10,0,5,15],\n [40,5,0,0,5,\n 0,10,0,10,0],\n [45,5,-20,0,0,\n 0,-15,0,-25,-10],\n [0,0,0,0,10,\n 10,-25,0,20,0],\n \n [-40,15,-5,0,5,\n 10,-30,0,15,20],\n [0,5,0,0,0,\n -5,-15,0,10,15],\n [40,5,0,5,5,\n 0,5,0,10,0],\n [45,20,-20,0,-5,\n 0,-15,0,-5,0],\n [0,-5,0,0,10,\n 10,-25,0,40,0],\n \n [-40,10,-5,0,-10,\n 10,-35,0,10,10],\n [0,-5,0,0,0,\n -5,-10,0,10,0],\n [40,5,0,0,0,\n 0,10,0,50,5],\n [45,10,-20,0,20,\n 0,-15,0,-5,15],\n [0,-5,0,0,10,\n 10,-25,0,20,0],\n \n [-20,50,-5,0,-15,\n 10,-15,0,15,25],\n [0,-15,0,0,20,\n -5,-10,0,-10,10],\n [60,0,0,0,0,\n 0,10,0,10,0],\n [45,10,-10,0,0,\n 0,15,0,0,0],\n [0,-5,0,0,15,\n 10,-25,0,15,5],\n \n [-40,30,-5,0,-15,\n 10,-35,0,20,25],\n [0,-5,0,0,15,\n -5,-10,0,-10,5],\n [40,15,0,0,5,\n 5,10,0,30,-5],\n [45,15,-20,0,0,\n 0,-15,0,-25,5],\n [0,-5,0,0,25,\n 10,-25,0,40,5],\n \n [-10,50,-10,5,-25,\n 10,-35,0,10,25],\n [20,-20,0,0,0,\n -5,-5,0,-10,10],\n [40,-5,0,0,5,\n 0,-5,0,30,0],\n [35,10,-20,0,0,\n 0,-15,0,-40,5],\n [-10,10,0,5,0,\n 10,-25,0,20,0],\n \n [-25,50,-5,0,-15,\n 10,-35,0,15,30],\n [0,-20,0,0,20,\n -5,-10,0,-20,10],\n [40,25,-5,0,-5,\n 0,10,0,50,0],\n [15,10,-20,0,0,\n 0,-15,0,-35,0],\n [0,-15,0,0,25,\n 10,-25,0,80,0],\n \n [-60,50,-5,0,-10,\n 10,-35,0,10,25],\n [-5,20,0,0,0,\n -5,-10,0,5,5],\n [30,0,0,0,0,\n 0,10,0,25,0],\n [45,10,-20,0,0,\n 10,-20,0,-45,0],\n [-10,-5,0,0,20,\n 10,-25,0,30,0],\n \n [-50,70,-5,0,-35,\n 10,-35,0,10,35],\n [20,-10,10,0,0,\n -5,-10,0,-10,10],\n [40,15,0,0,0,\n 0,10,0,30,0],\n [25,15,-20,5,5,\n 0,-35,0,-40,10],\n [-5,25,0,0,10,\n 10,-25,0,25,5],\n \n [-40,10,-5,0,-15,\n 10,-15,0,0,30],\n [0,15,0,0,0,\n -5,-10,0,-10,15],\n [40,25,0,0,0,\n 0,10,0,20,0],\n [45,25,-20,0,10,\n 0,-35,0,-45,0],\n [0,20,5,0,25,\n 10,-15,0,25,0],\n \n [-40,10,-10,0,-5,\n 10,-10,0,20,25],\n [20,5,0,0,5,\n -5,-10,0,0,10],\n [40,15,0,0,10,\n 0,10,0,50,0],\n [25,10,-20,0,20,\n 0,-15,0,-15,5],\n [-20,-5,5,0,25,\n 10,-25,0,60,20],\n \n [-40,0,-5,0,-20,\n 10,-15,0,40,-5],\n [20,0,0,5,5,\n -5,-10,0,0,-5],\n [40,5,0,0,0,\n 10,-10,-5,0,0],\n [45,5,-20,0,0,\n 5,-20,0,-5,0],\n [0,-5,0,0,10,\n 10,-20,0,10,0],\n \n ])\n}\n\n# Số lượng hình ảnh được chọn để xoay\nnum_indices = X_train_pick2rotate['index'].shape[0]\nnum_offsets = X_train_pick2rotate['offset'].shape[0]\n\nif num_indices == num_offsets:\n print(\"Số lượng chỉ số và số lượng offset là bằng nhau.\")\nelse:\n print(\"Số lượng chỉ số và số lượng offset không bằng nhau.\")\n if num_indices - num_offsets > 0 :\n print(\"So luong index lon hon offset\")\n else: print(\"So luong index nho hon offset\")\n \nnumPickRotate = len(X_train_pick2rotate['index'])\n\nprint('numPickRotate = ', numPickRotate)\nprint('numOffsets = ', num_offsets)","metadata":{"execution":{"iopub.status.busy":"2024-06-14T06:01:58.579672Z","iopub.execute_input":"2024-06-14T06:01:58.580515Z","iopub.status.idle":"2024-06-14T06:01:58.788164Z","shell.execute_reply.started":"2024-06-14T06:01:58.580469Z","shell.execute_reply":"2024-06-14T06:01:58.787210Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# -------------------------------------------------- Xoay hình ảnh\n\n# Xác định góc quay chuẩn hóa\nstandard_offsets = np.mean(X_train_pick2rotate['offset'], axis=1)\n\n# Khởi tạo các mảng để lưu trữ hình ảnh sau khi xoay và nhãn của chúng\nX_sample_rotate = np.zeros((10,numRotate_prior, len(X_train_pick2rotate['index']), X_train.shape[1]))\ny_sample_rotate = np.zeros((10,numRotate_prior,len(X_train_pick2rotate['index'])))\n\nfor k0 in range(10):\n for k1 in range(numRotate_prior):\n for k2 in range(numPickRotate):\n \n idxPick_X_train = X_train_pick2rotate['index'][k2]\n \n idx = np.where(y_train == k0)[0][idxPick_X_train]\n \n y_sample_rotate[k0,k1,k2] = k0 * numRotate_prior + k1\n \n # Tính góc quay mới dựa trên offset chuẩn hóa\n angleOffset_adjusted = X_train_pick2rotate['offset'][k2][k0] - standard_offsets[k0]\n angleRotate = (k1/numRotate_prior * angleRange_prior) + angleOffset_prior + angleOffset_adjusted\n \n X_sample_rotate[k0,k1,k2,:] = func_array1d_rotate2d(X_train[idx][np.newaxis,:],angleRotate, 28)\n\n \nX_sample_rotate[X_sample_rotate < 10] = 0\n\n\nprint(\"Before applying MinMax Scaler:\")\nprint(\" - Minimum value:\", X_sample_rotate.min())\nprint(\" - Maximum value:\", X_sample_rotate.max())\n\n# Chuẩn hóa hình ảnh sau khi xoay\nminmax_scaler = MinMaxScaler()\n\nfor k0 in range(10):\n for k1 in range(numRotate_prior):\n X_sample_rotate[k0,k1,:,:] = minmax_scaler.fit_transform(X_sample_rotate[k0,k1,:,:]) * 255\n\nprint(\"\\nAfter applying MinMax Scaler:\")\nprint(\" - Minimum value:\", X_sample_rotate.min())\nprint(\" - Maximum value:\", X_sample_rotate.max())\n\n\n\n# Next, let's visualize one image from each class.\n# We will use the matplotlib 'imshow' function.\n\n# for k2 in range(numPickRotate):\nfor k2 in range(0, numPickRotate, 10):\n fig, axs = plt.subplots(nrows=2, ncols=5, figsize=(10, 5))\n\n for k0 in range(10):\n img = axs[int(k0 >= 5), k0 % 5].imshow(\n X_sample_rotate[k0, 0, k2, :].reshape(28, 28),\n cmap='Blues'\n )\n img.axes.get_yaxis().set_visible(False)\n img.axes.get_xaxis().set_visible(False)\n img.axes.set_title(f'Label: {k0}')\n img.axes.annotate(f'{X_train_pick2rotate[\"index\"][k2]}', xy=(1, 1), xycoords='axes fraction', fontsize=12, color='red', xytext=(-5, -5), textcoords='offset points', ha='right', va='top')\n plt.show()\n\nX_train_rotate = np.reshape(X_sample_rotate, (-1, 784))\ny_train_rotate = np.reshape(y_sample_rotate, (-1, 1)).flatten()","metadata":{"papermill":{"duration":15.925568,"end_time":"2024-06-11T03:05:09.488840","exception":false,"start_time":"2024-06-11T03:04:53.563272","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-14T06:01:58.789572Z","iopub.execute_input":"2024-06-14T06:01:58.789877Z","iopub.status.idle":"2024-06-14T06:02:32.071075Z","shell.execute_reply.started":"2024-06-14T06:01:58.789852Z","shell.execute_reply":"2024-06-14T06:02:32.070156Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# **Phần 3: Phân tích thành phần chính**\n","metadata":{"papermill":{"duration":0.021635,"end_time":"2024-06-11T03:05:09.533669","exception":false,"start_time":"2024-06-11T03:05:09.512034","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"Phân tích thành phần chính (PCA) là một kỹ thuật thống kê để giảm chiều dữ liệu của một tập dữ liệu. Điều này được thực hiện bằng cách biến đổi tuyến tính dữ liệu thành một hệ tọa độ mới, trong đó (phần lớn) sự biến đổi trong dữ liệu có thể được mô tả với ít chiều hơn so với dữ liệu ban đầu.\n\nThành phần chính đầu tiên có thể được định nghĩa tương đương như một hướng tối đa hóa phương sai của dữ liệu đã được chiếu. Thành phần chính thứ i có thể được xem là một hướng vuông góc với các thành phần chính đầu tiên i-1 mà tối đa hóa phương sai của dữ liệu đã được chiếu.\n\nsklearn.decomposition.PCA: Giảm chiều tuyến tính bằng cách sử dụng Phân tích Giá trị Kỳ dị (SVD) của dữ liệu để chiếu nó vào không gian có chiều thấp hơn.\n\nỞ đây chúng ta sẽ sử dụng 10 thành phần chính hàng đầu, giải thích gần 50% phương sai trong dữ liệu. Điều này có nghĩa là chúng ta có thể biểu diễn 50% phương sai của dữ liệu với một không gian tính năng nhỏ hơn >98% so với không gian ban đầu (10 tính năng so với 784 tính năng).
","metadata":{"papermill":{"duration":0.02143,"end_time":"2024-06-11T03:05:09.577136","exception":false,"start_time":"2024-06-11T03:05:09.555706","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# Chuẩn hóa giá trị pixel về khoảng [0,1]\n# Công thức: MinMaxscale(X) = (X - X_min) / (X_max - X_min)\n\n# In ra giá trị nhỏ nhất và lớn nhất của dữ liệu trước khi áp dụng MinMax Scaler\nprint(\"Trước khi áp dụng MinMax Scaler:\")\nprint(\" - Giá trị nhỏ nhất:\", X_train.min())\nprint(\" - Giá trị lớn nhất:\", X_train.max())\n\n# Tạo một đối tượng MinMaxScaler\nminmax_scaler = MinMaxScaler()\n\n# Áp dụng MinMaxScaler để chuẩn hóa dữ liệu X_train_rotate\nX_train_rotate_scaled = minmax_scaler.fit_transform(X_train_rotate)\n\n# In ra giá trị nhỏ nhất và lớn nhất của dữ liệu sau khi áp dụng MinMax Scaler\nprint(\"\\nSau khi áp dụng MinMax Scaler:\")\nprint(\" - Giá trị nhỏ nhất:\", X_train_rotate_scaled.min())\nprint(\" - Giá trị lớn nhất:\", X_train_rotate_scaled.max())\n","metadata":{"papermill":{"duration":0.312607,"end_time":"2024-06-11T03:05:09.911662","exception":false,"start_time":"2024-06-11T03:05:09.599055","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-14T06:02:32.072701Z","iopub.execute_input":"2024-06-14T06:02:32.073102Z","iopub.status.idle":"2024-06-14T06:02:32.414319Z","shell.execute_reply.started":"2024-06-14T06:02:32.073069Z","shell.execute_reply":"2024-06-14T06:02:32.413255Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Thực tế, chúng ta sẽ tích hợp các module MinMaxScaler và PCA\n# vào một Pipeline của Sklearn.\n\n# Tạo một Pipeline với các bước:\n# - 'scaler': Chuẩn hóa giá trị pixel sử dụng MinMaxScaler\n# - 'pca': Giảm chiều dữ liệu sử dụng PCA với số thành phần chính là 40\npca = Pipeline( steps = [\n ('scaler', MinMaxScaler()),\n ('pca', PCA(n_components=40,\n svd_solver ='auto', \n whiten =True))\n ])\n\n# Áp dụng Pipeline để chuẩn hóa và giảm chiều dữ liệu\nX_train_rotate_pca = pca.fit_transform(X=X_train_rotate)\n\n# In ra kích thước của dữ liệu sau khi áp dụng PCA\nprint('X_train_pca shape:', X_train_rotate_pca.shape)\n","metadata":{"papermill":{"duration":3.588746,"end_time":"2024-06-11T03:05:13.523628","exception":false,"start_time":"2024-06-11T03:05:09.934882","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-14T06:02:32.415838Z","iopub.execute_input":"2024-06-14T06:02:32.416220Z","iopub.status.idle":"2024-06-14T06:02:35.670575Z","shell.execute_reply.started":"2024-06-14T06:02:32.416169Z","shell.execute_reply":"2024-06-14T06:02:35.669597Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Trích xuất tỷ lệ phương sai được giải thích\n# Và tính tổng lũy kế của chúng\n\n# Lấy đối tượng PCA từ Pipeline\npca_object = pca.named_steps['pca']\n\n# Trích xuất tỷ lệ phương sai được giải thích cho từng thành phần chính\nvar_ratio = pd.Series(\n data=pca_object.explained_variance_ratio_, # Dữ liệu là tỷ lệ phương sai được giải thích\n index=range(1,41) # Đặt chỉ số từ 1 đến 10 tương ứng với các thành phần chính\n)\n\n# Tính tổng lũy kế của tỷ lệ phương sai được giải thích\ncumsum_var_ratio = var_ratio.cumsum()\n","metadata":{"papermill":{"duration":0.034304,"end_time":"2024-06-11T03:05:13.580793","exception":false,"start_time":"2024-06-11T03:05:13.546489","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-14T06:02:35.671760Z","iopub.execute_input":"2024-06-14T06:02:35.672046Z","iopub.status.idle":"2024-06-14T06:02:35.677478Z","shell.execute_reply.started":"2024-06-14T06:02:35.672022Z","shell.execute_reply":"2024-06-14T06:02:35.676565Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Vẽ biểu đồ tỷ lệ phương sai được giải thích cho từng thành phần chính (trái)\n# Và vẽ biểu đồ tổng lũy kế của chúng (phải)\n\n# Tạo một figure với hai subplots đặt cạnh nhau\nfig, (ax1, ax2) = plt.subplots(1, 2, figsize=(15,5))\n\n# Biểu đồ thanh cho tỷ lệ phương sai được giải thích của từng thành phần chính\nsns.barplot(\n x=var_ratio.index, # Trục x là chỉ số của các thành phần chính\n y=var_ratio.values, # Trục y là giá trị tỷ lệ phương sai được giải thích\n color='#4895EE', # Màu của thanh biểu đồ\n ax=ax1 # Vẽ lên subplot ax1\n)\n# Thiết lập tiêu đề và nhãn cho trục x và y của biểu đồ thứ nhất\nax1.set(\n title='Giải thích tỷ lệ phương sai PCA', # Tiêu đề\n xlabel='Component', # Nhãn trục x\n ylabel='Var Ratio', # Nhãn trục y\n)\n# Thiết lập giới hạn cho trục y của biểu đồ thứ nhất\nax1.set_ylim(0, 0.5)\n\n# Biểu đồ thanh cho tổng lũy kế của tỷ lệ phương sai được giải thích\nsns.barplot(\n x=cumsum_var_ratio.index, # Trục x là chỉ số của các thành phần chính\n y=cumsum_var_ratio.values, # Trục y là giá trị tổng lũy kế của tỷ lệ phương sai được giải thích\n color='#4895EE', # Màu của thanh biểu đồ\n ax=ax2 # Vẽ lên subplot ax2\n)\n# Thiết lập tiêu đề và nhãn cho trục x và y của biểu đồ thứ hai\nax2.set(\n title='Tỷ lệ phương sai giải thích tích lũy PCA', # Tiêu đề\n xlabel='Component', # Nhãn trục x\n ylabel='Cumulative Var Ratio', # Nhãn trục y\n)\n# Thiết lập giới hạn cho trục y của biểu đồ thứ hai\n_ = ax2.set_ylim(0, 0.5)\n","metadata":{"papermill":{"duration":1.519599,"end_time":"2024-06-11T03:05:15.122953","exception":false,"start_time":"2024-06-11T03:05:13.603354","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-14T06:02:35.678577Z","iopub.execute_input":"2024-06-14T06:02:35.678852Z","iopub.status.idle":"2024-06-14T06:02:37.325680Z","shell.execute_reply.started":"2024-06-14T06:02:35.678829Z","shell.execute_reply":"2024-06-14T06:02:37.324684Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Chuyển các thành phần chính thành định dạng DataFrame của Pandas\n# Bằng cách này chúng ta có thể sử dụng các thư viện seaborn và plotly\n# để trực quan hóa dữ liệu.\n# (các gói này hoạt động tốt hơn với Pandas DataFrames)\n\n# Tạo một DataFrame từ dữ liệu đã giảm chiều\ndf_train_pca = pd.DataFrame(\n data=X_train_rotate_pca[:,:3], # Lấy 3 thành phần chính đầu tiên\n columns=['x', 'y', 'z'] # Đặt tên cho các cột là 'x', 'y', 'z'\n)\n\n# Thêm cột 'target' để xác định màu sắc của các điểm trong trực quan hóa\ndf_train_pca['target'] = y_train_rotate\n\n# In ra kích thước của DataFrame\nprint(df_train_pca.shape)\n\n# Hiển thị 5 hàng đầu tiên của DataFrame\ndf_train_pca.head()\n","metadata":{"papermill":{"duration":0.046659,"end_time":"2024-06-11T03:05:15.192863","exception":false,"start_time":"2024-06-11T03:05:15.146204","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-14T06:02:37.327001Z","iopub.execute_input":"2024-06-14T06:02:37.327335Z","iopub.status.idle":"2024-06-14T06:02:37.342286Z","shell.execute_reply.started":"2024-06-14T06:02:37.327309Z","shell.execute_reply":"2024-06-14T06:02:37.341247Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Trực quan hóa dữ liệu hai chiều\n# Ở đây hai thành phần chính hàng đầu được vẽ biểu đồ\n\n# Đặt phong cách biểu đồ là 'dark'\nsns.set_style(\"dark\")\n\n# Tạo biểu đồ tán xạ với seaborn\nax = sns.scatterplot(\n data=df_train_pca, # Dữ liệu để vẽ biểu đồ là DataFrame df_train_pca\n x='x', # Trục x là cột 'x' (thành phần chính thứ nhất)\n y='y', # Trục y là cột 'y' (thành phần chính thứ hai)\n hue='target', # Màu sắc của các điểm dựa trên giá trị của cột 'target'\n palette='Blues', # Bảng màu sử dụng là 'Blues'\n)\n\n# Thiết lập tiêu đề và nhãn cho trục x và y của biểu đồ\nax.set(\n title='Biểu đồ không gian 2 chiều PCA', # Tiêu đề của biểu đồ\n xlabel='1st Comp.', # Nhãn trục x (thành phần chính thứ nhất)\n ylabel='2nd Comp.' # Nhãn trục y (thành phần chính thứ hai)\n)\n\n# Thêm đường ngang tại y=0 với đường nét đứt và màu '#FFBF00'\nax.axhline(y=0, linestyle='--', color='#FFBF00', zorder=-1)\n\n# Thêm đường dọc tại x=0 với đường nét đứt và màu '#FFBF00'\nax.axvline(x=0, linestyle='--', color='#FFBF00', zorder=-1)\n","metadata":{"papermill":{"duration":3.922873,"end_time":"2024-06-11T03:05:19.141434","exception":false,"start_time":"2024-06-11T03:05:15.218561","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-14T06:02:37.343395Z","iopub.execute_input":"2024-06-14T06:02:37.343681Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Trực quan hóa dữ liệu ba chiều\n# Ở đây ba thành phần chính hàng đầu được vẽ biểu đồ\n\n# Đặt phong cách biểu đồ là 'white'\nsns.set_style(\"white\")\n\n# Tạo biểu đồ tán xạ 3D với plotly\nfig = px.scatter_3d(\n data_frame=df_train_pca, # Dữ liệu để vẽ biểu đồ là DataFrame df_train_pca\n x='x', # Trục x là cột 'x' (thành phần chính thứ nhất)\n y='y', # Trục y là cột 'y' (thành phần chính thứ hai)\n z='z', # Trục z là cột 'z' (thành phần chính thứ ba)\n labels={ # Thiết lập nhãn cho các trục\n 'x': '1st Comp.', # Nhãn trục x\n 'y': '2nd Comp.', # Nhãn trục y\n 'z': '3rd Comp.' # Nhãn trục z\n },\n color='target', # Màu sắc của các điểm dựa trên giá trị của cột 'target'\n title='Biểu đồ không gian 3 chiều PCA', # Tiêu đề của biểu đồ\n color_continuous_scale='blues', # Bảng màu sử dụng là 'blues'\n template='ggplot2' # Template của biểu đồ là 'ggplot2'\n)\n\n# Hiển thị biểu đồ\nfig.show()\n","metadata":{"papermill":{"duration":2.483131,"end_time":"2024-06-11T03:05:21.657807","exception":false,"start_time":"2024-06-11T03:05:19.174676","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-14T06:02:41.370366Z","iopub.execute_input":"2024-06-14T06:02:41.370697Z","iopub.status.idle":"2024-06-14T06:02:41.469975Z","shell.execute_reply.started":"2024-06-14T06:02:41.370668Z","shell.execute_reply":"2024-06-14T06:02:41.468844Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Bây giờ, chúng ta sẽ áp dụng inverse_transform để tính\n# sự xấp xỉ của các hình ảnh dựa trên 10 thành phần chính.\n# Có thể thấy rằng chỉ với 10 thành phần, chúng ta đã truyền tải được\n# một phần lớn thông tin của mỗi hình ảnh.\n\npca_object = pca.named_steps['pca']\n\nfig, axs = plt.subplots(nrows=10, ncols=2, figsize=(6,20))\n\nfor i in range(10):\n idx = np.where(y_train==i)[0][0]\n original_img = axs[i, 0].imshow(\n X=X_train_rotate[idx].reshape(28, 28),\n cmap='Blues'\n )\n original_img.axes.get_yaxis().set_visible(False)\n original_img.axes.get_xaxis().set_visible(False)\n original_img.axes.set_title(f'Original [{i}]:')\n \n approx_img_ = pca_object.inverse_transform(X=X_train_rotate_pca[idx])\n approx_img = axs[i, 1].imshow(\n X=approx_img_.reshape(28, 28),\n cmap='Blues'\n )\n approx_img.axes.get_yaxis().set_visible(False)\n approx_img.axes.get_xaxis().set_visible(False)\n approx_img.axes.set_title(f'Approximation [{i}]:')\n","metadata":{"papermill":{"duration":1.383877,"end_time":"2024-06-11T03:05:23.110699","exception":false,"start_time":"2024-06-11T03:05:21.726822","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-14T06:02:41.471252Z","iopub.execute_input":"2024-06-14T06:02:41.471552Z","iopub.status.idle":"2024-06-14T06:02:42.437433Z","shell.execute_reply.started":"2024-06-14T06:02:41.471527Z","shell.execute_reply":"2024-06-14T06:02:42.435738Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Hiển thị rằng các thành phần PCA không tương quan\n# (Tuyên bố này là đúng theo định nghĩa và xây dựng\n# của thuật toán Phân tích thành phần chính)\n\ncorr = np.corrcoef(x=X_train_rotate_pca, rowvar=False)\ncorr = np.around(corr, decimals=2)\n\nax = sns.heatmap(\n data=corr, \n vmax=1, \n vmin=-1,\n square=True,\n linewidths=.5,\n annot=True,\n cmap='Blues',\n)\n\n_ = ax.set(\n title='Correlation PCA Components [Uncorrelated]',\n xlabel='Components',\n ylabel='Components'\n)\n","metadata":{"papermill":{"duration":5.006689,"end_time":"2024-06-11T03:05:28.183809","exception":false,"start_time":"2024-06-11T03:05:23.177120","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-14T06:02:42.439041Z","iopub.execute_input":"2024-06-14T06:02:42.439481Z","iopub.status.idle":"2024-06-14T06:02:46.251285Z","shell.execute_reply.started":"2024-06-14T06:02:42.439431Z","shell.execute_reply":"2024-06-14T06:02:46.250350Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# **Phần 4: Sử dụng K-Nearest Neighbors Classifier phân loại 10 thành phần chính hàng đầu từ PCA**","metadata":{"papermill":{"duration":0.063251,"end_time":"2024-06-11T03:05:28.315163","exception":false,"start_time":"2024-06-11T03:05:28.251912","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"Thuật toán K-Nearest Neighbors (KNN) là một bộ phân loại học có giám sát, phi tham số, sử dụng độ gần để thực hiện các phân loại hoặc dự đoán về sự nhóm của một điểm dữ liệu cá nhân. Dự đoán: Một nhãn lớp được gán dựa trên cơ sở của một cuộc bỏ phiếu đa số, nhãn được đại diện nhiều nhất xung quanh một điểm dữ liệu cho trước.\n\nTrong dự án này, đầu vào của Bộ phân loại KNN sẽ là 10 thành phần chính hàng đầu từ PCA.\n\nsklearn.neighbors.KNeighborsClassifier: Bộ phân loại thực hiện việc bỏ phiếu của k-nearest neighbors. Chúng ta sẽ chạy GridSearchCV để tối ưu hóa siêu tham số n_neighbors.
","metadata":{"papermill":{"duration":0.067821,"end_time":"2024-06-11T03:05:28.451762","exception":false,"start_time":"2024-06-11T03:05:28.383941","status":"completed"},"tags":[]}},{"cell_type":"code","source":"from sklearn.preprocessing import StandardScaler\nfrom sklearn.decomposition import PCA\nfrom sklearn.neighbors import KNeighborsClassifier\nfrom sklearn.model_selection import RandomizedSearchCV, RepeatedStratifiedKFold\nfrom sklearn.pipeline import Pipeline\n\nknn_clf = Pipeline(steps=[\n ('scaler', StandardScaler()),\n ('pca', PCA()),\n ('knn', KNeighborsClassifier())\n])\n\nknn_clf_params = {\n 'pca__n_components': [20, 30, 40, 50, 60],\n 'knn__n_neighbors': [2, 3, 5, 7, 10, 15, 20],\n 'knn__weights': ['uniform', 'distance'],\n 'knn__metric': ['euclidean', 'manhattan']\n}\nrepeated_cv = RepeatedStratifiedKFold(n_splits=5, n_repeats=3, random_state=42)\n\nknn_clf_gs = RandomizedSearchCV(\n estimator=knn_clf, \n param_distributions=knn_clf_params,\n n_iter=10, \n scoring='accuracy',\n n_jobs=-1,\n cv=repeated_cv, \n verbose=1,\n return_train_score=True,\n random_state=42\n)\n\nknn_clf_gs.fit(X=X_train_rotate_pca, y=y_train_rotate)\n\nprint('Best PCA+KNN params:', knn_clf_gs.best_params_)\nprint('Best PCA+KNN CV accuracy:', knn_clf_gs.best_score_)","metadata":{"papermill":{"duration":1410.266042,"end_time":"2024-06-11T03:28:58.786220","exception":false,"start_time":"2024-06-11T03:05:28.520178","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-14T06:02:46.252523Z","iopub.execute_input":"2024-06-14T06:02:46.252831Z","iopub.status.idle":"2024-06-14T06:11:59.845622Z","shell.execute_reply.started":"2024-06-14T06:02:46.252805Z","shell.execute_reply":"2024-06-14T06:11:59.844231Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Vẽ biểu đồ cho độ chính xác của tập huấn luyện và tập validation\n# cho mỗi giá trị siêu tham số được thử nghiệm trong quá trình\n# tìm kiếm trên lưới.\n# Biểu đồ này cho phép chúng ta kiểm tra sự quá mức (overfitting).\n\n# Số lượng kết quả thử nghiệm từ GridSearchCV\nnum_trials = len(knn_clf_gs.cv_results_['params'])\n\nfig, ax = plt.subplots()\n\n# Vẽ biểu đồ cho độ chính xác trung bình của tập huấn luyện\nax.plot(\n range(num_trials), \n knn_clf_gs.cv_results_['mean_train_score'],\n label='train',\n color='blue'\n)\n\n# Vẽ biểu đồ cho độ chính xác trung bình của tập kiểm tra (validation)\nax.plot(\n range(num_trials),\n knn_clf_gs.cv_results_['mean_test_score'],\n label='test',\n color='orange'\n)\n\n# Đặt nhãn cho các điểm trên trục x dựa trên số lượng hàng xóm\nax.set_xticks(\n ticks=range(num_trials),\n labels=[param['knn__n_neighbors'] for param in knn_clf_gs.cv_results_['params']]\n)\n\n# Tìm chỉ số của kết quả tốt nhất và vẽ đường thẳng đứng tại điểm đó\nbest_index = knn_clf_gs.best_index_\nplt.axvline(x=best_index, linestyle='--', color='red', label='best', alpha=0.3)\n\nplt.legend()\nplt.xlabel(\"'n_neighbors' values\")\nplt.ylabel('Accuracy')\nplt.title(\"PCA+KNN GridSearchCV Results\")\nplt.show()\n","metadata":{"papermill":{"duration":0.552951,"end_time":"2024-06-11T03:28:59.410502","exception":false,"start_time":"2024-06-11T03:28:58.857551","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-14T06:11:59.848342Z","iopub.execute_input":"2024-06-14T06:11:59.849172Z","iopub.status.idle":"2024-06-14T06:12:00.255273Z","shell.execute_reply.started":"2024-06-14T06:11:59.849121Z","shell.execute_reply":"2024-06-14T06:12:00.254343Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# **Phần 5: Tạo một tệp tin chứa các dự đoán của mô hình học máy trên tập dữ liệu Test**","metadata":{"papermill":{"duration":0.063226,"end_time":"2024-06-11T03:28:59.543749","exception":false,"start_time":"2024-06-11T03:28:59.480523","status":"completed"},"tags":[]}},{"cell_type":"code","source":"from sklearn.metrics import accuracy_score\n\n# Tiến hành dự đoán với dữ liệu validation\nX_val = X_train_rotate_pca\nY_val = y_train_rotate\n\nbest_cv_clf = knn_clf_gs.best_estimator_\ny_pred = best_cv_clf.predict(X=X_val)\n\n\n# Dự đoán các phần tử số và góc quay\ny_val_number = Y_val // numRotate_prior\ny_pred_number = y_pred // numRotate_prior\n\ny_val_angle = (Y_val % numRotate_prior)\ny_pred_angle = (y_pred % numRotate_prior)\n\nprint('KNN number + angle, Acc({:.2f}%)', accuracy_score(Y_val, y_pred))\nprint('KNN number, Acc({:.2f}%)', accuracy_score(y_val_number, y_pred_number))\nprint('KNN angle, Acc({:.2f}%)', accuracy_score(y_val_angle, y_pred_angle))\n\ny_val_angle_degree = y_val_angle/numRotate_prior * angleRange_prior + angleOffset_prior\ny_pred_angle_degree = y_pred_angle/numRotate_prior * angleRange_prior + angleOffset_prior\n\n# In ma trận chéo thông tin (Confusion Matrix)\nconf_matrix = confusion_matrix(y_val_number, y_pred_number)\nconf_matrix_disp = ConfusionMatrixDisplay(conf_matrix)\nconf_matrix_disp.plot(cmap='Blues')\n\nconf_matrix = confusion_matrix(y_val_angle, y_pred_angle)\nconf_matrix_disp = ConfusionMatrixDisplay(conf_matrix)\nconf_matrix_disp.plot(cmap='Blues')","metadata":{"papermill":{"duration":7.118028,"end_time":"2024-06-11T03:29:06.732549","exception":false,"start_time":"2024-06-11T03:28:59.614521","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-14T06:12:00.256817Z","iopub.execute_input":"2024-06-14T06:12:00.257494Z","iopub.status.idle":"2024-06-14T06:12:09.449875Z","shell.execute_reply.started":"2024-06-14T06:12:00.257458Z","shell.execute_reply":"2024-06-14T06:12:09.448914Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X_val = pca.transform(X_train)\nY_val = y_train\n\nbest_cv_clf = knn_clf_gs.best_estimator_\ny_pred = best_cv_clf.predict(X=X_val)\n\ny_pred_number = y_pred\ny_val_number = Y_val\n\nprint('KNN number, Acc({:.2f}%)', accuracy_score(y_val_number, y_pred_number))\n\nconf_matrix = confusion_matrix(y_val_number, y_pred_number)\nconf_matrix_disp = ConfusionMatrixDisplay(conf_matrix)\nconf_matrix_disp.plot(cmap='Blues')\n\ny_pred_angle = (y_pred % numRotate_prior)\n\ny_pred_angle_degree = y_pred_angle/numRotate_prior * angleRange_prior + angleOffset_prior\n\nprint('y_pred_angle_degree =', y_pred_angle_degree)","metadata":{"papermill":{"duration":33.710098,"end_time":"2024-06-11T03:29:40.518271","exception":false,"start_time":"2024-06-11T03:29:06.808173","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-14T06:12:09.451141Z","iopub.execute_input":"2024-06-14T06:12:09.451492Z","iopub.status.idle":"2024-06-14T06:12:36.631374Z","shell.execute_reply.started":"2024-06-14T06:12:09.451466Z","shell.execute_reply":"2024-06-14T06:12:36.630436Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Bây giờ, hãy áp dụng biến đổi nghịch đảo để tính toán\n# sự gần đúng của hình ảnh dựa trên nguyên tắc 10 thành phần.\n# Chúng ta có thể thấy rằng chỉ với 10 thành phần chúng ta đã truyền tải được một lượng lớn\n# một phần thông tin của mỗi hình ảnh.\npca_object = pca.named_steps['pca']\n\nfig, axs = plt.subplots(nrows=10, ncols=2, figsize=(6,20))\n\nfor i in range(10):\n idx = np.where(y_train == i )[0][0]\n original_img = axs[i, 0].imshow(\n X=X_train[idx].reshape(28, 28),\n cmap='Blues'\n )\n original_img.axes.get_yaxis().set_visible(False)\n original_img.axes.get_xaxis().set_visible(False)\n original_img.axes.set_title(f'Original [{i}]:')\n \n approx_img_ = pca_object.inverse_transform(X=X_val[idx])\n approx_img = axs[i, 1].imshow(\n X=approx_img_.reshape(28, 28),\n cmap='Blues'\n )\n approx_img.axes.get_yaxis().set_visible(False)\n approx_img.axes.get_xaxis().set_visible(False)\n approx_img.axes.set_title(f'Predicted angle [{y_pred_angle_degree[idx]}]:')","metadata":{"papermill":{"duration":1.682892,"end_time":"2024-06-11T03:29:42.271330","exception":false,"start_time":"2024-06-11T03:29:40.588438","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-14T06:12:36.632785Z","iopub.execute_input":"2024-06-14T06:12:36.633209Z","iopub.status.idle":"2024-06-14T06:12:37.524939Z","shell.execute_reply.started":"2024-06-14T06:12:36.633164Z","shell.execute_reply":"2024-06-14T06:12:37.524021Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import plotly.express as px\nfrom plotly.offline import init_notebook_mode, iplot\nimport plotly.graph_objects as go\nimport warnings\nwarnings.filterwarnings('ignore')","metadata":{"papermill":{"duration":0.086713,"end_time":"2024-06-11T03:29:42.439735","exception":false,"start_time":"2024-06-11T03:29:42.353022","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-14T06:12:37.530690Z","iopub.execute_input":"2024-06-14T06:12:37.530953Z","iopub.status.idle":"2024-06-14T06:12:37.535880Z","shell.execute_reply.started":"2024-06-14T06:12:37.530931Z","shell.execute_reply":"2024-06-14T06:12:37.534900Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def plot_confusion_matrix(cm, labels, title):\n # cm : confusion matrix list(list)\n # labels : name of the data list(str)\n # title : title for the heatmap\n data = go.Heatmap(z=cm, y=labels, x=labels)\n annotations = []\n for i, row in enumerate(cm):\n for j, value in enumerate(row):\n annotations.append(\n {\n \"x\": labels[i],\n \"y\": labels[j],\n \"font\": {\"color\": \"white\"},\n \"text\": str(value),\n \"xref\": \"x1\",\n \"yref\": \"y1\",\n \"showarrow\": False\n }\n )\n layout = {\n \"title\": title,\n \"xaxis\": {\"title\": \"Predicted value\"},\n \"yaxis\": {\"title\": \"Real value\"},\n \"annotations\": annotations\n }\n fig = go.Figure(data=data, layout=layout)\n return fig","metadata":{"papermill":{"duration":0.092695,"end_time":"2024-06-11T03:29:42.612886","exception":false,"start_time":"2024-06-11T03:29:42.520191","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-14T06:12:37.536927Z","iopub.execute_input":"2024-06-14T06:12:37.537170Z","iopub.status.idle":"2024-06-14T06:12:37.548682Z","shell.execute_reply.started":"2024-06-14T06:12:37.537149Z","shell.execute_reply":"2024-06-14T06:12:37.547830Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# from tensorflow.keras.utils import np_utils\nfrom tensorflow.keras import utils\nfrom tensorflow.keras.callbacks import ModelCheckpoint, EarlyStopping\nfrom tensorflow.keras import datasets, layers, models","metadata":{"papermill":{"duration":16.902472,"end_time":"2024-06-11T03:29:59.597554","exception":false,"start_time":"2024-06-11T03:29:42.695082","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-14T06:12:37.549771Z","iopub.execute_input":"2024-06-14T06:12:37.550108Z","iopub.status.idle":"2024-06-14T06:12:37.561897Z","shell.execute_reply.started":"2024-06-14T06:12:37.550077Z","shell.execute_reply":"2024-06-14T06:12:37.561047Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X_train_cnn = X_train_rotate.reshape(-1,28,28,1)\n# X_val = np.array(pca.transform(X_train)).reshape(-1,28,28,1)\ny_train_cnn = utils.to_categorical(np.array(y_train_rotate// numRotate_prior))\nX_val = X_train.reshape(-1,28,28,1)\nY_val = utils.to_categorical(np.array(y_train))","metadata":{"papermill":{"duration":0.094764,"end_time":"2024-06-11T03:29:59.775739","exception":false,"start_time":"2024-06-11T03:29:59.680975","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-14T06:12:37.562978Z","iopub.execute_input":"2024-06-14T06:12:37.563339Z","iopub.status.idle":"2024-06-14T06:12:37.591914Z","shell.execute_reply.started":"2024-06-14T06:12:37.563313Z","shell.execute_reply":"2024-06-14T06:12:37.590890Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X_train_cnn.shape\nX_train_cnn = X_train_cnn / 255.0\nX_val = X_val / 255.0\n","metadata":{"papermill":{"duration":0.091979,"end_time":"2024-06-11T03:29:59.948338","exception":false,"start_time":"2024-06-11T03:29:59.856359","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-14T06:12:37.593065Z","iopub.execute_input":"2024-06-14T06:12:37.593416Z","iopub.status.idle":"2024-06-14T06:12:37.771451Z","shell.execute_reply.started":"2024-06-14T06:12:37.593381Z","shell.execute_reply":"2024-06-14T06:12:37.770594Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(\"Giá trị pixel tối thiểu trong X_train_cnn:\", X_train_cnn.min())\nprint(\"Giá trị pixel tối đa trong X_train_cnn:\", X_train_cnn.max())\nprint(\"Giá trị pixel tối thiểu trong X_val:\", X_val.min())\nprint(\"Giá trị pixel tối đa trong X_val:\", X_val.max())\n","metadata":{"execution":{"iopub.status.busy":"2024-06-14T06:12:37.772628Z","iopub.execute_input":"2024-06-14T06:12:37.772915Z","iopub.status.idle":"2024-06-14T06:12:37.861188Z","shell.execute_reply.started":"2024-06-14T06:12:37.772891Z","shell.execute_reply":"2024-06-14T06:12:37.860265Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Số lượng tập dữ liệu train\nnum_train_samples = X_train_cnn.shape[0]\nprint(\"Số lượng tập dữ liệu train:\", num_train_samples)\n\n# Số lượng tập dữ liệu validation (được sử dụng như test trong trường hợp này)\nnum_val_samples = X_val.shape[0]\nprint(\"Số lượng tập dữ liệu validation:\", num_val_samples)\n","metadata":{"execution":{"iopub.status.busy":"2024-06-14T06:12:37.862153Z","iopub.execute_input":"2024-06-14T06:12:37.862428Z","iopub.status.idle":"2024-06-14T06:12:37.868102Z","shell.execute_reply.started":"2024-06-14T06:12:37.862406Z","shell.execute_reply":"2024-06-14T06:12:37.867216Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# from tensorflow.keras.preprocessing.image import ImageDataGenerator\n# # Định nghĩa ImageDataGenerator với các thông số tăng cường dữ liệu\n# datagen = ImageDataGenerator(\n# rotation_range=10,\n# width_shift_range=0.1,\n# height_shift_range=0.1,\n# zoom_range=0.1\n# )\n# datagen.fit(X_train_cnn)","metadata":{"execution":{"iopub.status.busy":"2024-06-14T06:12:37.869171Z","iopub.execute_input":"2024-06-14T06:12:37.869455Z","iopub.status.idle":"2024-06-14T06:12:37.880031Z","shell.execute_reply.started":"2024-06-14T06:12:37.869432Z","shell.execute_reply":"2024-06-14T06:12:37.879223Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# import numpy as np\n# from tensorflow.keras import utils\n# from tensorflow.keras.callbacks import ModelCheckpoint, EarlyStopping, ReduceLROnPlateau\n# from tensorflow.keras.preprocessing.image import ImageDataGenerator\n# from tensorflow.keras import models, layers\n# import matplotlib.pyplot as plt","metadata":{"execution":{"iopub.status.busy":"2024-06-14T06:12:37.881083Z","iopub.execute_input":"2024-06-14T06:12:37.881915Z","iopub.status.idle":"2024-06-14T06:12:37.889314Z","shell.execute_reply.started":"2024-06-14T06:12:37.881891Z","shell.execute_reply":"2024-06-14T06:12:37.888577Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# y_train_cnn = utils.to_categorical(np.array(y_train_rotate // numRotate_prior))\n# Y_val = utils.to_categorical(np.array(y_train))","metadata":{"papermill":{"duration":0.098171,"end_time":"2024-06-11T03:30:00.131501","exception":false,"start_time":"2024-06-11T03:30:00.033330","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-14T06:12:37.890426Z","iopub.execute_input":"2024-06-14T06:12:37.890725Z","iopub.status.idle":"2024-06-14T06:12:37.899634Z","shell.execute_reply.started":"2024-06-14T06:12:37.890702Z","shell.execute_reply":"2024-06-14T06:12:37.898856Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# X_train_rotate = np.array(X_train_rotate).reshape(-1,28,28,1)\n# # X_val = np.array(pca.transform(X_train)).reshape(-1,28,28,1)\n# X_val = np.array(X_test_crop).reshape(-1,28,28,1)\n# y_train_rotate = utils.to_categorical(np.array(y_train_rotate))\n# Y_val = utils.to_categorical(np.array(y_test_crop))","metadata":{"papermill":{"duration":0.105582,"end_time":"2024-06-11T03:30:00.326644","exception":false,"start_time":"2024-06-11T03:30:00.221062","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-14T06:12:37.900757Z","iopub.execute_input":"2024-06-14T06:12:37.901033Z","iopub.status.idle":"2024-06-14T06:12:37.910168Z","shell.execute_reply.started":"2024-06-14T06:12:37.901005Z","shell.execute_reply":"2024-06-14T06:12:37.909403Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# model = models.Sequential()\n# model.add(layers.Conv2D(filters = 32, kernel_size = (5,5),padding = 'Same', \n# activation ='relu', input_shape = (28,28,1)))\n# model.add(layers.Conv2D(filters = 32, kernel_size = (5,5),padding = 'Same', \n# activation ='relu'))\n# model.add(layers.MaxPool2D(pool_size=(2,2)))\n# model.add(layers.BatchNormalization())\n# model.add(layers.Dropout(0.25))\n# model.add(layers.Conv2D(filters = 64, kernel_size = (3,3),padding = 'Same', \n# activation ='relu'))\n# model.add(layers.Conv2D(filters = 64, kernel_size = (3,3),padding = 'Same', \n# activation ='relu'))\n# model.add(layers.MaxPool2D(pool_size=(2,2), strides=(2,2)))\n# model.add(layers.BatchNormalization())\n# model.add(layers.Dropout(0.25))\n# model.add(layers.Flatten())\n# model.add(layers.Dense(256, activation = \"relu\"))\n# model.add(layers.Dropout(0.5))\n# model.add(layers.BatchNormalization())\n# model.add(layers.Dense(10, activation = \"softmax\"))\n\n# model.summary()","metadata":{"papermill":{"duration":0.724239,"end_time":"2024-06-11T03:30:01.140241","exception":false,"start_time":"2024-06-11T03:30:00.416002","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-14T06:12:37.911023Z","iopub.execute_input":"2024-06-14T06:12:37.911309Z","iopub.status.idle":"2024-06-14T06:12:37.920091Z","shell.execute_reply.started":"2024-06-14T06:12:37.911286Z","shell.execute_reply":"2024-06-14T06:12:37.919368Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from tensorflow.keras import models, layers, regularizers\n\nmodel = models.Sequential()\n# First convolutional block\nmodel.add(layers.Conv2D(filters=32, kernel_size=(3, 3), padding='Same', activation='relu', input_shape=(28, 28, 1)))\nmodel.add(layers.BatchNormalization())\nmodel.add(layers.Conv2D(filters=32, kernel_size=(3, 3), padding='Same', activation='relu'))\nmodel.add(layers.BatchNormalization())\nmodel.add(layers.MaxPool2D(pool_size=(2, 2)))\nmodel.add(layers.Dropout(0.25))\n\n# Second convolutional block\nmodel.add(layers.Conv2D(filters=64, kernel_size=(3, 3), padding='Same', activation='relu'))\nmodel.add(layers.BatchNormalization())\nmodel.add(layers.Conv2D(filters=64, kernel_size=(3, 3), padding='Same', activation='relu'))\nmodel.add(layers.BatchNormalization())\nmodel.add(layers.MaxPool2D(pool_size=(2, 2), strides=(2, 2)))\nmodel.add(layers.Dropout(0.25))\n\n# Third convolutional block\nmodel.add(layers.Conv2D(filters=128, kernel_size=(3, 3), padding='Same', activation='relu'))\nmodel.add(layers.BatchNormalization())\nmodel.add(layers.Conv2D(filters=128, kernel_size=(3, 3), padding='Same', activation='relu'))\nmodel.add(layers.BatchNormalization())\nmodel.add(layers.MaxPool2D(pool_size=(2, 2), strides=(2, 2)))\nmodel.add(layers.Dropout(0.25))\n\n# Fully connected layers\nmodel.add(layers.Flatten())\nmodel.add(layers.Dense(256, activation='relu'))\nmodel.add(layers.BatchNormalization())\nmodel.add(layers.Dropout(0.5))\nmodel.add(layers.Dense(128, activation='relu'))\nmodel.add(layers.BatchNormalization())\nmodel.add(layers.Dropout(0.5))\nmodel.add(layers.Dense(10, activation='softmax'))\n\n# Print the model summary\nmodel.summary()\n","metadata":{"execution":{"iopub.status.busy":"2024-06-14T06:12:37.921323Z","iopub.execute_input":"2024-06-14T06:12:37.921652Z","iopub.status.idle":"2024-06-14T06:12:38.331102Z","shell.execute_reply.started":"2024-06-14T06:12:37.921623Z","shell.execute_reply":"2024-06-14T06:12:38.330225Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from tensorflow.keras.callbacks import ReduceLROnPlateau\nimport tensorflow as tf\n# Define ReduceLROnPlateau callback\nreduce_lr = ReduceLROnPlateau(\n monitor='val_loss', \n factor=0.2, \n patience=2, \n min_lr=0.001, \n verbose=1\n)\n\ninitial_lr = 0.01\nmodel_path = './best_model.keras'\n\n# early_stopping = EarlyStopping(\n# monitor='val_accuracy',\n# patience=10, \n# verbose=1,\n# mode='max'\n \n# )\n\ncheckpointer = ModelCheckpoint(\n filepath=model_path,\n monitor='val_accuracy',\n verbose=1,\n mode='max',\n save_best_only=True)\n\nmodel.compile(\n optimizer=tf.keras.optimizers.Adam(learning_rate=initial_lr),\n loss='categorical_crossentropy',\n metrics=['accuracy']\n)","metadata":{"papermill":{"duration":0.116465,"end_time":"2024-06-11T03:30:01.352142","exception":false,"start_time":"2024-06-11T03:30:01.235677","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-14T06:12:38.332151Z","iopub.execute_input":"2024-06-14T06:12:38.332455Z","iopub.status.idle":"2024-06-14T06:12:38.342457Z","shell.execute_reply.started":"2024-06-14T06:12:38.332430Z","shell.execute_reply":"2024-06-14T06:12:38.341604Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Fit the model\nhistory = model.fit(\n X_train_cnn, y_train_cnn,\n validation_data=(X_val, Y_val),\n epochs=100,\n batch_size=128,\n callbacks=[checkpointer, reduce_lr]\n)","metadata":{"papermill":{"duration":12791.3813,"end_time":"2024-06-11T07:03:12.826992","exception":false,"start_time":"2024-06-11T03:30:01.445692","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-14T06:12:38.343523Z","iopub.execute_input":"2024-06-14T06:12:38.343773Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def plot_training_history(history):\n fig = px.line(\n x=history.epoch, \n y=[history.history['accuracy'], history.history['val_accuracy']],\n labels={'x': 'Epoch', 'y': 'Accuracy'},\n title='Training and Validation Accuracy'\n )\n fig.data[0].name = 'Training Accuracy'\n fig.data[1].name = 'Validation Accuracy'\n iplot(fig)\n\n fig = px.line(\n x=history.epoch, \n y=[history.history['loss'], history.history['val_loss']],\n labels={'x': 'Epoch', 'y': 'Loss'},\n title='Training and Validation Loss'\n )\n fig.data[0].name = 'Training Loss'\n fig.data[1].name = 'Validation Loss'\n iplot(fig)\n\nplot_training_history(history)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import plotly.graph_objects as go\nfrom plotly.subplots import make_subplots\nimport numpy as np\n\nfig = go.Figure()\n\nlabels = ['Loss', 'Accuracy']\ncolors = ['rgb(67,67,67)', 'rgb(49,130,189)']\n\nmode_size = [8, 8]\nline_size = [2, 2]\n\nx_data = np.vstack((np.arange(1, 51),)*2)\n\ny_data = np.array([\n np.array(history.history['loss']), \n np.array(history.history['accuracy'])\n])\n\nfor i in range(0, 2):\n fig.add_trace(go.Scatter(x=x_data[i], y=y_data[i], mode='lines',\n name=labels[i],\n line=dict(color=colors[i], width=line_size[i]),\n connectgaps=True,\n ))\n\n # endpoints\n fig.add_trace(go.Scatter(\n x=[x_data[i][0], x_data[i][-1]],\n y=[y_data[i][0], y_data[i][-1]],\n mode='markers',\n marker=dict(color=colors[i], size=mode_size[i])\n ))\n\nfig.update_layout(\n xaxis=dict(\n showline=True,\n showgrid=False,\n showticklabels=True,\n linecolor='rgb(204, 204, 204)',\n linewidth=2,\n ticks='outside',\n tickfont=dict(\n family='Arial',\n size=12,\n color='rgb(82, 82, 82)',\n ),\n ),\n yaxis=dict(\n showgrid=False,\n zeroline=False,\n showline=False,\n showticklabels=False,\n ),\n autosize=False,\n margin=dict(\n autoexpand=False,\n l=100,\n r=20,\n t=110,\n ),\n showlegend=False,\n plot_bgcolor='white'\n)\n\nannotations = []\n\n# Adding labels\nfor y_trace, label, color in zip(y_data, labels, colors):\n # labeling the left_side of the plot\n annotations.append(dict(xref='paper', x=0.05, y=y_trace[0],\n xanchor='right', yanchor='middle',\n text=label + ' {:.2f}'.format(y_trace[0]),\n font=dict(family='Arial',\n size=16),\n showarrow=False))\n # labeling the right_side of the plot\n annotations.append(dict(xref='paper', x=0.95, y=y_trace[-1],\n xanchor='left', yanchor='middle',\n text='{:.4f}%'.format(y_trace[-1]),\n font=dict(family='Arial',\n size=16),\n showarrow=False))\n# Title\nannotations.append(dict(xref='paper', yref='paper', x=0.0, y=1.05,\n xanchor='left', yanchor='bottom',\n text='Training Loss & Accuracy',\n font=dict(family='Arial',\n size=30,\n color='rgb(37,37,37)'),\n showarrow=False))\n# Source\nannotations.append(dict(xref='paper', yref='paper', x=0.5, y=-0.1,\n xanchor='center', yanchor='top',\n text='Epochs',\n font=dict(family='Arial',\n size=12,\n color='rgb(150,150,150)'),\n showarrow=False))\n\nfig.update_layout(annotations=annotations,height=600, width=1000)\n\nfig.show()","metadata":{"papermill":{"duration":2.58904,"end_time":"2024-06-11T07:03:17.979073","exception":false,"start_time":"2024-06-11T07:03:15.390033","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from sklearn.metrics import confusion_matrix\nimport seaborn as sns\nimport matplotlib.pyplot as plt\n\n# Dự đoán nhãn trên tập dữ liệu validation\nY_val_pred = model.predict(X_val)\nY_val_pred_classes = np.argmax(Y_val_pred, axis=1) \nY_true = np.argmax(Y_val, axis=1) \n\n# Tính toán confusion matrix\nconf_matrix = confusion_matrix(Y_true, Y_val_pred_classes)\n\n# Hiển thị confusion matrix dưới dạng heatmap\nplt.figure(figsize=(10,8))\nsns.heatmap(conf_matrix, annot=True, fmt='d', cmap='Blues')\nplt.xlabel('Predicted Label')\nplt.ylabel('True Label')\nplt.title('Confusion Matrix')\nplt.show()\n","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from keras.models import load_model\nmodel = load_model('./best_model.keras')\ny_pred = model.predict(X_val)\nM = confusion_matrix(np.argmax(Y_val,1), np.argmax(y_pred,1))\nplot_confusion_matrix(M, sorted(np.unique(np.argmax(Y_val,1))), 'Simple CNN Confusion Matrix, Acc({:.2f}%)'.format(accuracy_score(np.argmax(Y_val,1), np.argmax(y_pred,1))*100))","metadata":{"papermill":{"duration":42.82929,"end_time":"2024-06-11T07:04:03.358528","exception":false,"start_time":"2024-06-11T07:03:20.529238","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"n = np.where(np.argmax(y_pred,1)!=np.argmax(Y_val,1))\nn = np.array(n).reshape(-1)\nn","metadata":{"papermill":{"duration":2.668185,"end_time":"2024-06-11T07:04:08.704133","exception":false,"start_time":"2024-06-11T07:04:06.035948","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X_val_vis = np.array(X_val).reshape(X_val.shape[0], 28, 28)\nX_val_vis = X_val_vis\nfig, axis = plt.subplots(int(len(n)/8), 8, figsize=(35, 15))\nfor i, ax in enumerate(axis.flat):\n ax.imshow(X_val_vis[n[i]], cmap='binary')\n digit = np.argmax(y_pred,1)[n[i]]\n digit2 = np.argmax(Y_val,1)[n[i]]\n ax.set(title = f\"Pred Number is {digit}\\n Real Number is {digit2}\");","metadata":{"papermill":{"duration":119.155453,"end_time":"2024-06-11T07:06:10.364411","exception":false,"start_time":"2024-06-11T07:04:11.208958","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X_test = np.array(df_test).reshape(-1,28,28,1)","metadata":{"papermill":{"duration":2.690873,"end_time":"2024-06-11T07:06:15.559946","exception":false,"start_time":"2024-06-11T07:06:12.869073","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = load_model('./best_model.keras')\ny_pred = model.predict(X_test)\ny_pred = np.argmax(y_pred,1)\npd_submit = pd.read_csv('../input/digit-recognizer/sample_submission.csv')\npd_submit['Label'] = y_pred\npd_submit.to_csv('submission.csv', header=True, index=False)","metadata":{"papermill":{"duration":28.359486,"end_time":"2024-06-11T07:06:46.545313","exception":false,"start_time":"2024-06-11T07:06:18.185827","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"papermill":{"duration":2.71789,"end_time":"2024-06-11T07:06:51.800899","exception":false,"start_time":"2024-06-11T07:06:49.083009","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]}]}