{"cells":[{"metadata":{},"cell_type":"markdown","source":"This code builds on these two improvements to Inversion's benchmark kernel:\n  - https://www.kaggle.com/seriousran/just-speed-up-calculate-distance-from-benchmark\n\n  - https://www.kaggle.com/rakibilly/faster-distance-calculation-from-benchmark\n\nNot that we need anything faster than 400ms, but I wanted to see what a GPU could do. I used [RAPIDS](https://rapids.ai) cuDF, which is a GPU DataFrame library. It's pretty cool from what I've seen so far. You can use an API similar to Pandas and get a 16x speedup over numpy functions!"},{"metadata":{},"cell_type":"markdown","source":"### Setup"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\ntrain = pd.read_csv('../input/train.csv', index_col='id')\ntest = pd.read_csv('../input/test.csv', index_col='id')\n\nstructures = pd.read_csv('../input/structures.csv')\n\ndef map_atom_info(df, atom_idx):\n    df = pd.merge(df, structures, how = 'left',\n                  left_on  = ['molecule_name', f'atom_index_{atom_idx}'],\n                  right_on = ['molecule_name',  'atom_index'])\n    \n    df = df.drop('atom_index', axis=1)\n    df = df.rename(columns={'atom': f'atom_{atom_idx}',\n                            'x': f'x_{atom_idx}',\n                            'y': f'y_{atom_idx}',\n                            'z': f'z_{atom_idx}'})\n    return df\n\ntrain = map_atom_info(train, 0)\ntrain = map_atom_info(train, 1)\n\ntest = map_atom_info(test, 0)\ntest = map_atom_info(test, 1)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Speed Tests\n\n#### 0. Original Benchmark code - pandas.DataFrame.apply()"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Engineer a single feature: distance vector between atoms\n#  (there's ways to speed this up!)\n\n# def dist(row):\n#     return ( (row['x_1'] - row['x_0'])**2 +\n#              (row['y_1'] - row['y_0'])**2 +\n#              (row['z_1'] - row['z_0'])**2 ) ** 0.5\n\n# train['dist'] = train.apply(lambda x: dist(x), axis=1)\n# test['dist'] = test.apply(lambda x: dist(x), axis=1)\n\n# takes 7+ minutes per run","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"#### 1. numpy.linalg()"},{"metadata":{"trusted":true},"cell_type":"code","source":"%%timeit\n# This block is SPED UP\n\ntrain_p_0 = train[['x_0', 'y_0', 'z_0']].values\ntrain_p_1 = train[['x_1', 'y_1', 'z_1']].values\ntest_p_0 = test[['x_0', 'y_0', 'z_0']].values\ntest_p_1 = test[['x_1', 'y_1', 'z_1']].values\n\ntr_a_min_b = train_p_0 - train_p_1\nte_a_min_b = test_p_0 - test_p_1\ntrain['dist_speedup'] = np.linalg.norm(tr_a_min_b, axis=1)\ntest['dist_speedup'] = np.linalg.norm(te_a_min_b, axis=1)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"   #### 2. numpy.einsum()"},{"metadata":{"trusted":true},"cell_type":"code","source":"%%timeit\n# This block is SPED UP a little more\n\ntrain_p_0 = train[['x_0', 'y_0', 'z_0']].values\ntrain_p_1 = train[['x_1', 'y_1', 'z_1']].values\ntest_p_0 = test[['x_0', 'y_0', 'z_0']].values\ntest_p_1 = test[['x_1', 'y_1', 'z_1']].values\n\ntr_a_min_b = train_p_0 - train_p_1\nte_a_min_b = test_p_0 - test_p_1\ntrain['dist_speedup_einsum'] = np.sqrt(np.einsum('ij,ij->i', tr_a_min_b, tr_a_min_b))\ntest['dist_speedup_einsum'] = np.sqrt(np.einsum('ij,ij->i', te_a_min_b, te_a_min_b))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"#### 3. cudf.DataFrame"},{"metadata":{"trusted":true,"_kg_hide-output":true},"cell_type":"code","source":"# Install rapids component, cuDF\n# Takes a few minutes...\n! conda update --force-reinstall conda -y\n\n!conda install -c nvidia -c rapidsai -c numba -c conda-forge -c defaults \\\n    cudf=0.7 python=3.6 cudatoolkit=10.0 -y\n\nimport cudf\n\n# Convert existing Pandas df since it exists already\ngputrain = cudf.DataFrame.from_pandas(train)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"%%timeit\n#This block is STRAIGHT UP ACCELERATED\n\ndef get_dist(df):\n    return np.sqrt((df.x_1-df.x_0)**2 +\n                   (df.y_1-df.y_0)**2 +\n                   (df.z_1-df.z_0)**2)\n\ngputrain['dist_rapids'] = get_dist(gputrain)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Verify the Results"},{"metadata":{"trusted":true},"cell_type":"code","source":"print(gputrain[['dist_speedup', 'dist_speedup_einsum','dist_rapids']].head())","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Good stuff! Anybody else using Rapids?"}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.4","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}