{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.7.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"nvidiaTeslaT4","dataSources":[{"sourceId":35725,"databundleVersionId":3597961,"sourceType":"competition"}],"dockerImageVersionId":30203,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"### Sidebar: kwargs and Delegates","metadata":{}},{"cell_type":"code","source":"from fastai.collab import *\nfrom fastai.tabular.all import *\nimport torch\n\n\nset_seed(42)\n\n\npath = Path('../input/fmi-su-recommender-system-hw-22022')\n\nratings = pd.read_csv(path/'ratings_train.csv')\nratings = ratings.drop(ratings.columns[0], axis=1)  # 删除不需要的列\nprint(ratings.columns)  # 检查列名\n\n\nbooks = pd.read_csv(path/'books.csv')\nprint(books.columns)  # 检查列名\n\n\nratings = ratings.merge(books, on='Book_Id')\n\n\nratings.rename(columns={'book_title': 'Title'}, inplace=True)\n\nprint(ratings.head()) \n\ndls = CollabDataLoaders.from_df(ratings, item_name='Title', bs=64)\ndls.show_batch()\n\n\nn_users = len(dls.classes['User_Id'])\nn_books = len(dls.classes['Title'])\nn_factors = 5\n\ndef create_params(size):\n    return nn.Parameter(torch.zeros(*size).normal_(0, 0.01))\n\n\n\n\n\n","metadata":{"execution":{"iopub.status.busy":"2024-08-10T13:57:47.779137Z","iopub.execute_input":"2024-08-10T13:57:47.779705Z","iopub.status.idle":"2024-08-10T13:57:56.482884Z","shell.execute_reply.started":"2024-08-10T13:57:47.779656Z","shell.execute_reply":"2024-08-10T13:57:56.481666Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\nclass DotProductBias(Module):\n    def __init__(self, n_users, n_books, n_factors, y_range=(0,5.5)):\n        self.user_factors = create_params([n_users, n_factors])\n        self.user_bias = create_params([n_users])\n        self.book_factors = create_params([n_books, n_factors])\n        self.book_bias = create_params([n_books])\n        self.y_range = y_range\n        \n    def forward(self, x):\n        users = self.user_factors[x[:,0]]\n        books = self.book_factors[x[:,1]]\n        res = (users*books).sum(dim=1)\n        res += self.user_bias[x[:,0]] + self.book_bias[x[:,1]]\n        return sigmoid_range(res, *self.y_range)\n\n\nn_factors = 50\nmodel = DotProductBias(n_users, n_books, n_factors=n_factors)\nlearn = Learner(dls, model, loss_func=MSELossFlat())\nlearn.fit_one_cycle(5, 5e-3, wd=0.1)","metadata":{"execution":{"iopub.status.busy":"2024-08-10T13:57:56.485380Z","iopub.execute_input":"2024-08-10T13:57:56.486077Z","iopub.status.idle":"2024-08-10T14:07:58.695546Z","shell.execute_reply.started":"2024-08-10T13:57:56.486034Z","shell.execute_reply":"2024-08-10T14:07:58.694453Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\nbook_bias = learn.model.book_bias.squeeze()\nidxs = book_bias.argsort()[:5]\n[dls.classes['Title'][i] for i in idxs]","metadata":{"execution":{"iopub.status.busy":"2024-08-10T14:07:58.696936Z","iopub.execute_input":"2024-08-10T14:07:58.697288Z","iopub.status.idle":"2024-08-10T14:07:58.728577Z","shell.execute_reply.started":"2024-08-10T14:07:58.697246Z","shell.execute_reply":"2024-08-10T14:07:58.727581Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\ng = ratings.groupby('Title')['Rating'].count()\ntop_books = g.sort_values(ascending=False).index.values[:1000]\ntop_idxs = tensor([learn.dls.classes['Title'].o2i[m] for m in top_books])\nbook_w = learn.model.book_factors[top_idxs].cpu().detach()\nbook_pca = book_w.pca(3)\nfac0,fac1,fac2 = book_pca.t()\nidxs = list(range(50))\nX = fac0[idxs]\nY = fac2[idxs]\nplt.figure(figsize=(12,12))\nplt.scatter(X, Y)\nfor i, x, y in zip(top_books[idxs], X, Y):\n    plt.text(x,y,i, color=np.random.rand(3)*0.7, fontsize=11)\nplt.show()\n","metadata":{"execution":{"iopub.status.busy":"2024-08-10T14:14:31.699598Z","iopub.execute_input":"2024-08-10T14:14:31.700711Z","iopub.status.idle":"2024-08-10T14:14:33.139155Z","shell.execute_reply.started":"2024-08-10T14:14:31.700659Z","shell.execute_reply":"2024-08-10T14:14:33.137978Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\n\nlearn = collab_learner(dls, n_factors=50, y_range=(0, 5.5))\nlearn.lr_find()\nlearn.fit_one_cycle(5, 5e-3, wd=0.1)","metadata":{"execution":{"iopub.status.busy":"2024-08-10T14:14:37.251958Z","iopub.execute_input":"2024-08-10T14:14:37.252754Z","iopub.status.idle":"2024-08-10T14:24:15.024663Z","shell.execute_reply.started":"2024-08-10T14:14:37.252714Z","shell.execute_reply":"2024-08-10T14:24:15.023740Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\n\nbook_bias = learn.model.i_bias.weight.squeeze()\nidxs = book_bias.argsort(descending=True)[:5]\n[dls.classes['Title'][i] for i in idxs]","metadata":{"execution":{"iopub.status.busy":"2024-08-10T14:25:08.728592Z","iopub.execute_input":"2024-08-10T14:25:08.729032Z","iopub.status.idle":"2024-08-10T14:25:08.738541Z","shell.execute_reply.started":"2024-08-10T14:25:08.728995Z","shell.execute_reply":"2024-08-10T14:25:08.737549Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\ntest_data = pd.read_csv('/kaggle/input/fmi-su-recommender-system-hw-22022/ratings_to_predict.csv')\n\ntest_data['User_Id'] = pd.to_numeric(test_data['User_Id'], errors='coerce').astype(int)\n\n\ntest_data = test_data.merge(books[['Book_Id', 'Title']], on='Book_Id', how='left')\n\n\ntest_dl = dls.test_dl(test_data[['User_Id', 'Book_Id', 'Title']], bs=64)\n\n\ntest_preds, _ = learn.get_preds(dl=test_dl)\n\n\nsubmission_df = test_data[['User_Id', 'Book_Id']].copy()\nsubmission_df.loc[:, 'Rating'] = test_preds.numpy()\nsubmission_df['Rating'] = submission_df['Rating'].clip(0, 10.5)\nsubmission_df.to_csv('submission.csv', index=False)\n\nprint(\"Submission file 'submission.csv' created successfully.\")\n","metadata":{"execution":{"iopub.status.busy":"2024-08-10T14:42:56.422455Z","iopub.execute_input":"2024-08-10T14:42:56.423039Z","iopub.status.idle":"2024-08-10T14:43:08.439398Z","shell.execute_reply.started":"2024-08-10T14:42:56.422996Z","shell.execute_reply":"2024-08-10T14:43:08.438164Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"`EmbeddingNN` includes `**kwargs` as a parameter to `__init__`. In Python `**kwargs` in a parameter list means \"put any additional keyword arguments into a dict called `kwargs`. And `**kwargs` in an argument list means \"insert all key/value pairs in the `kwargs` dict as named arguments here\". This approach is used in many popular libraries, such as `matplotlib`, in which the main `plot` function simply has the signature `plot(*args, **kwargs)`. The [`plot` documentation](https://matplotlib.org/api/pyplot_api.html#matplotlib.pyplot.plot) says \"The `kwargs` are `Line2D` properties\" and then lists those properties.\n\nWe're using `**kwargs` in `EmbeddingNN` to avoid having to write all the arguments to `TabularModel` a second time, and keep them in sync. However, this makes our API quite difficult to work with, because now Jupyter Notebook doesn't know what parameters are available. Consequently things like tab completion of parameter names and pop-up lists of signatures won't work.\n\nfastai resolves this by providing a special `@delegates` decorator, which automatically changes the signature of the class or function (`EmbeddingNN` in this case) to insert all of its keyword arguments into the signature.","metadata":{}},{"cell_type":"markdown","source":"### End sidebar","metadata":{}},{"cell_type":"markdown","source":"Although the results of `EmbeddingNN` are a bit worse than the dot product approach (which shows the power of carefully constructing an architecture for a domain), it does allow us to do something very important: we can now directly incorporate other user and movie information, date and time information, or any other information that may be relevant to the recommendation. That's exactly what `TabularModel` does. In fact, we've now seen that `EmbeddingNN` is just a `TabularModel`, with `n_cont=0` and `out_sz=1`. So, we'd better spend some time learning about `TabularModel`, and how to use it to get great results! We'll do that in the next chapter.","metadata":{}},{"cell_type":"markdown","source":"## Conclusion","metadata":{}},{"cell_type":"markdown","source":"For our first non-computer vision application, we looked at recommendation systems and saw how gradient descent can learn intrinsic factors or biases about items from a history of ratings. Those can then give us information about the data. \n\nWe also built our first model in PyTorch. We will do a lot more of this in the next section of the book, but first, let's finish our dive into the other general applications of deep learning, continuing with tabular data.","metadata":{}},{"cell_type":"markdown","source":"## Questionnaire","metadata":{}},{"cell_type":"markdown","source":"1. What problem does collaborative filtering solve?\n1. How does it solve it?\n1. Why might a collaborative filtering predictive model fail to be a very useful recommendation system?\n1. What does a crosstab representation of collaborative filtering data look like?\n1. Write the code to create a crosstab representation of the MovieLens data (you might need to do some web searching!).\n1. What is a latent factor? Why is it \"latent\"?\n1. What is a dot product? Calculate a dot product manually using pure Python with lists.\n1. What does `pandas.DataFrame.merge` do?\n1. What is an embedding matrix?\n1. What is the relationship between an embedding and a matrix of one-hot-encoded vectors?\n1. Why do we need `Embedding` if we could use one-hot-encoded vectors for the same thing?\n1. What does an embedding contain before we start training (assuming we're not using a pretained model)?\n1. Create a class (without peeking, if possible!) and use it.\n1. What does `x[:,0]` return?\n1. Rewrite the `DotProduct` class (without peeking, if possible!) and train a model with it.\n1. What is a good loss function to use for MovieLens? Why? \n1. What would happen if we used cross-entropy loss with MovieLens? How would we need to change the model?\n1. What is the use of bias in a dot product model?\n1. What is another name for weight decay?\n1. Write the equation for weight decay (without peeking!).\n1. Write the equation for the gradient of weight decay. Why does it help reduce weights?\n1. Why does reducing weights lead to better generalization?\n1. What does `argsort` do in PyTorch?\n1. Does sorting the movie biases give the same result as averaging overall movie ratings by movie? Why/why not?\n1. How do you print the names and details of the layers in a model?\n1. What is the \"bootstrapping problem\" in collaborative filtering?\n1. How could you deal with the bootstrapping problem for new users? For new movies?\n1. How can feedback loops impact collaborative filtering systems?\n1. When using a neural network in collaborative filtering, why can we have different numbers of factors for movies and users?\n1. Why is there an `nn.Sequential` in the `CollabNN` model?\n1. What kind of model should we use if we want to add metadata about users and items, or information such as date and time, to a collaborative filtering model?","metadata":{}},{"cell_type":"markdown","source":"### Further Research\n\n1. Take a look at all the differences between the `Embedding` version of `DotProductBias` and the `create_params` version, and try to understand why each of those changes is required. If you're not sure, try reverting each change to see what happens. (NB: even the type of brackets used in `forward` has changed!)\n1. Find three other areas where collaborative filtering is being used, and find out what the pros and cons of this approach are in those areas.\n1. Complete this notebook using the full MovieLens dataset, and compare your results to online benchmarks. See if you can improve your accuracy. Look on the book's website and the fast.ai forum for ideas. Note that there are more columns in the full dataset—see if you can use those too (the next chapter might give you ideas).\n1. Create a model for MovieLens that works with cross-entropy loss, and compare it to the model in this chapter.","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}