{"cells":[{"metadata":{},"cell_type":"markdown","source":"## Message Passing Neural Network\n\nSo, as many of you might have surmised by now the dataset for this challenge is essentially the QM9 dataset with some new values calculated for it. \n\nThe first thing I though of when seeing this challenge was the [Gilmer paper](https://arxiv.org/abs/1704.01212), as it uses the QM9 dataset. ([see this talk](https://vimeo.com/238221016))\n\nThe major difference in this challenge is that we are asked to calulate bond properties (thus edges in a graph) as opposed to bulk properties in the paper. \n\nHere the model is laid out in a modular way so the parts can easily be replaced\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Internet needs to be on\n!pip install tensorflow-gpu==2.0a0","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import warnings\nwarnings.simplefilter(action='ignore', category=FutureWarning)\nimport tensorflow as tf\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nfrom sklearn.utils import shuffle\nimport os\nprint(os.listdir(\"../input\"))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Make sure tf 2.0 alpha has been installed\nprint(tf.__version__)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#is it using the gpu?\ntf.test.is_gpu_available(\n    cuda_only=False,\n    min_cuda_compute_capability=None\n)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"tf.random.set_seed(42)\ndatadir = \"../input/\"","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Message passer\n\nThe message passer here is a MLP that takes $concat([node_i, edge_{ij}, node_j])$ as input and returns a message of the same dimension of the node"},{"metadata":{"trusted":true},"cell_type":"code","source":"class Message_Passer_1(tf.keras.layers.Layer):\n    def __init__(self, intermediate_dim, state_dim):\n        super(Message_Passer_1, self).__init__()\n        self.concat_layer = tf.keras.layers.Concatenate()\n        self.hidden_layer_1 = tf.keras.layers.Dense(units=intermediate_dim, activation=tf.nn.relu)\n        self.output_layer = tf.keras.layers.Dense(units=state_dim, activation=tf.nn.relu)\n\n\n        \n    def call(self, node_i, node_j, edge_ij):\n        concat = self.concat_layer([node_i, node_j, edge_ij])\n        activation = self.hidden_layer_1(concat)\n        return self.output_layer(activation)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Aggregator\n\nDefine the message aggregator (just sum)  \nProbably overkill to have it as its own layer, but good if you want to replace it with something more complex\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"class Message_Agg(tf.keras.layers.Layer):\n    def __init__(self):\n        super(Message_Agg, self).__init__()\n    \n    def call(self, messages):\n        return tf.math.reduce_sum(messages, 2)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Node Update function\n\nThe node update function is an MLP that takes $[old\\_node, agg\\_messages]$ as input and return the new node value"},{"metadata":{"trusted":true},"cell_type":"code","source":"class Update_Func_1(tf.keras.layers.Layer):\n    def __init__(self, intermediate_dim, state_dim):\n        super(Update_Func_1, self).__init__()\n        self.concat_layer = tf.keras.layers.Concatenate()\n        self.hidden_layer_1  = tf.keras.layers.Dense(units=intermediate_dim, activation=tf.nn.relu)\n        self.output_layer = tf.keras.layers.Dense(units=state_dim, activation =  tf.nn.relu)\n\n        \n    def call(self, old_state, agg_messages):\n        concat = self.concat_layer([old_state, agg_messages])\n        activation = self.hidden_layer_1(concat)\n        return self.output_layer(activation)\n\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Edge update \n\nThe edge update function is a MLP that takes $concat([node_i, edge_{ij}, node_j])$ as input and produces a new edge value"},{"metadata":{"trusted":true},"cell_type":"code","source":"class Adj_Updater_1(tf.keras.layers.Layer):\n    def __init__(self, intermediate_dim, state_dim):\n        super(Adj_Updater_1, self).__init__()\n        self.concat_layer = tf.keras.layers.Concatenate()\n        self.hidden_layer_1 = tf.keras.layers.Dense(units=intermediate_dim, activation=tf.nn.relu)\n        self.output_layer = tf.keras.layers.Dense(units=state_dim, activation = tf.nn.relu)\n\n    def call(self, node_i, node_j, edge_ij):\n        concat = self.concat_layer([node_i, node_j, edge_ij])\n        activation = self.hidden_layer_1(concat)\n        return self.output_layer(activation)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Output layer\n\nThis is where the model diverges with the paper.   \nAs the paper predicts bulk properties, but we are interested in edges, we need something different.   \n\nHere the each edge is passed through a MLP which is used to regress the scalar coupling for each edge"},{"metadata":{"trusted":true},"cell_type":"code","source":"class Edge_Regressor(tf.keras.layers.Layer):\n    def __init__(self, intermediate_dim):\n        super(Edge_Regressor, self).__init__()\n        self.concat_layer = tf.keras.layers.Concatenate()\n        self.hidden_layer_1 = tf.keras.layers.Dense(units=intermediate_dim, activation=tf.nn.relu)\n        self.hidden_layer_2 = tf.keras.layers.Dense(units=intermediate_dim, activation=tf.nn.relu)\n        self.hidden_layer_3 = tf.keras.layers.Dense(units=intermediate_dim, activation=tf.nn.relu)\n        self.output_layer = tf.keras.layers.Dense(units=1)#, activation=tf.nn.tanh)\n\n        \n    def call(self, edges):\n            \n        activation_1 = self.hidden_layer_1(edges)\n        activation_2 = self.hidden_layer_2(activation_1)\n        activation_3 = self.hidden_layer_3(activation_2)\n\n        return self.output_layer(activation_3)\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Message passing layer\n\nPut all of the above together to make a message passing layer which does one round of message passing and node updating"},{"metadata":{"trusted":true},"cell_type":"code","source":"class MP_Layer(tf.keras.layers.Layer):\n    def __init__(self, mp_int_dim, up_int_dim, out_int_dim, state_dim):\n        super(MP_Layer, self).__init__(self)\n        self.state_dim = state_dim  \n        self.message_passers  = Message_Passer_1(intermediate_dim = mp_int_dim, state_dim = state_dim) \n        self.update_functions = Update_Func_1(intermediate_dim = up_int_dim, state_dim = state_dim)\n        self.adj_updaters     = Adj_Updater_1(intermediate_dim = up_int_dim, state_dim = state_dim)\n        self.message_aggs    = Message_Agg()       \n        self.batch_norm_n = tf.keras.layers.BatchNormalization() \n        self.batch_norm_e = tf.keras.layers.BatchNormalization() \n\n        \n    def call(self, nodes, edges, mask):\n        \n        nodes_0          = nodes\n        edges_0          = edges\n        \n        n_nodes  = tf.shape(nodes_0)[1]\n        node_dim = tf.shape(nodes_0)[2]\n        \n        state_i = tf.tile(nodes_0, [1, n_nodes, 1])\n        state_j = tf.reshape(tf.tile(nodes_0, [1, 1, n_nodes]),[-1,n_nodes*n_nodes, node_dim ])\n\n        new_edges = self.adj_updaters(state_i, state_j, edges_0)\n        new_edges = tf.math.multiply(new_edges, mask)\n\n        \n        messages  = self.message_passers(state_i, state_j, new_edges)\n        #Do this to ignore messages from non-existant nodes\n        masked =  tf.math.multiply(messages, mask)\n        masked = tf.reshape(masked, [tf.shape(messages)[0], tf.shape(nodes_0)[1], tf.shape(nodes_0)[1], tf.shape(messages)[2]])\n        agg_m = self.message_aggs(masked)\n        \n        # Update states\n        state_1 = self.update_functions(nodes_0, agg_m)\n      \n        # Batch norm and output\n        nodes_out = self.batch_norm_n(state_1)\n        edges_out = self.batch_norm_e(new_edges)     \n\n        return nodes_out, edges_out","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Define an edge only version of the MPL to do a final edge update. "},{"metadata":{"trusted":true},"cell_type":"code","source":"class MP_Layer_edge_only(tf.keras.layers.Layer):\n    def __init__(self, mp_int_dim, up_int_dim, out_int_dim, state_dim):\n        super(MP_Layer_edge_only, self).__init__(self)\n        self.adj_updaters     = Adj_Updater_1(intermediate_dim = up_int_dim, state_dim = state_dim)\n        self.message_aggs    = Message_Agg()\n        self.state_dim = state_dim         \n\n        \n    def call(self, nodes, edges, mask):\n     \n        nodes_0          = nodes\n        edges_0          = edges\n        \n        n_nodes  = tf.shape(nodes_0)[1]\n        node_dim = tf.shape(nodes_0)[2]\n        \n        state_i = tf.tile(nodes_0, [1, n_nodes, 1])\n        state_j = tf.reshape(tf.tile(nodes_0, [1, 1, n_nodes]),[-1,n_nodes*n_nodes, node_dim ])\n\n        new_edges = self.adj_updaters(state_i, state_j, edges_0)\n        new_edges = tf.math.multiply(new_edges, mask)\n        \n        edges_out = new_edges\n\n        return edges_out","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Put it all together to form a MPNN\n\nDefines the full mpnn that does T message passing steps, where T is a hyperparameter.   \nHere each layer has it's own weights, but weights can be shared across layers. "},{"metadata":{"trusted":true},"cell_type":"code","source":"# Define the MPNN here using the parts defined earlier\nadj_input = tf.keras.Input(shape=(None,), name='adj_input')\nnod_input = tf.keras.Input(shape=(None,), name='nod_input')\nclass MPNN(tf.keras.Model):\n    def __init__(self, mp_int_dim, up_int_dim, out_int_dim, state_dim, T):\n        super(MPNN, self).__init__(self)        \n        self.MP = [MP_Layer(mp_int_dim, up_int_dim, out_int_dim, state_dim) for _ in range(T)]        \n        self.MP_edge = MP_Layer_edge_only(mp_int_dim, up_int_dim, out_int_dim, state_dim) \n        self.embed_node = tf.keras.layers.Dense(units=state_dim, activation=tf.nn.relu)\n        self.embed_edge = tf.keras.layers.Dense(units=state_dim, activation=tf.nn.relu)        \n        self.edge_regressor  = Edge_Regressor(mp_int_dim)\n        \n    def call(self, inputs =  [adj_input, nod_input]):\n      \n      \n        nodes            = inputs['nod_input']\n        edges            = inputs['adj_input']\n\n        \n        edges_0    = edges\n\n        len_edges = tf.shape(edges)[-1]\n        \n        _, x = tf.split(edges, [len_edges -1, 1], 2)\n        mask =  tf.where(tf.equal(x, 0), x, tf.ones_like(x))\n        \n\n        nodes = self.embed_node(nodes) \n        edges = self.embed_edge(edges)\n\n        nodes_ = nodes\n        edges_ = edges\n              \n        for i, mp in enumerate(self.MP):\n            index = i + 1\n            if index%2 == 0:\n                nodes, edges =  mp(nodes, edges, mask)\n                nodes = nodes - nodes_\n                edges = edges - edges_\n                nodes_ = nodes \n                edges_ = edges\n                \n            else:\n                nodes, edges =  mp(nodes, edges, mask)\n                \n        \n        edges = self.MP_edge(nodes, edges, mask)\n        \n        con_edges = self.edge_regressor(edges)\n    \n        return con_edges","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Define the loss functions. \n\n(**note**: that for LMAE, as the output values have been scaled down values will be much smaller than for unscaled values)"},{"metadata":{"trusted":true},"cell_type":"code","source":"def log_mae(orig , preds):\n \n    # Mask values for which no scalar coupling exists\n    mask  = tf.where(tf.equal(orig, 0), orig, tf.ones_like(orig))\n\n    nums  = tf.boolean_mask(orig,  mask)\n    preds = tf.boolean_mask(preds,  mask)\n\n    reconstruction_error = tf.math.log(tf.reduce_mean(tf.abs(tf.subtract(nums, preds))))\n\n    return reconstruction_error","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Define some callbacks, the initial learning rate and the optimizer"},{"metadata":{"trusted":true},"cell_type":"code","source":"learning_rate = 0.001\ndef warmup(epoch):\n    initial_lrate = learning_rate   \n    if epoch == 0:\n        lrate = 0.00001\n    if epoch == 1:\n        lrate = 0.0001\n    if epoch > 1:\n        lrate = 0.001   \n    if epoch > 20:\n        lrate = 0.0001\n    if epoch > 25:\n        lrate = 0.00001\n        \n    tf.print(\"Learning rate: \", lrate)\n    return lrate\n\nlrate = tf.keras.callbacks.LearningRateScheduler(warmup)\n\n\nopt = tf.optimizers.Adam(learning_rate=learning_rate)\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Finally create the model, and compile"},{"metadata":{"trusted":true},"cell_type":"code","source":"mpnn = MPNN(mp_int_dim = 512, up_int_dim = 1024, out_int_dim = 512, state_dim = 256, T = 7)\n#mpnn = MPNN(mp_int_dim = 128, up_int_dim = 128, out_int_dim = 256, state_dim = 64, T = 5)\n\nmpnn.compile(opt, log_mae)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Define some hyperparameters"},{"metadata":{"trusted":true},"cell_type":"code","source":"batch_size = 64\nepochs = 30\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Let the learning begin!"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Wrap in a function so that memory is freed after calling\ndef train():\n    nodes_train     = np.load(datadir + \"internalgraphdata/nodes_train.npz\" )['arr_0']\n    in_edges_train  = np.load(datadir + \"internalgraphdata/in_edges_train.npz\")['arr_0']\n    out_edges_train = np.load(datadir + \"internalgraphdata/out_edges_train.npz\" )['arr_0']\n\n    out_labels = out_edges_train.reshape(-1,out_edges_train.shape[1]*out_edges_train.shape[2],1)\n    in_edges_train = in_edges_train.reshape(-1,in_edges_train.shape[1]*in_edges_train.shape[2],in_edges_train.shape[3])\n\n\n    train_size = int(len(out_labels)*0.8)\n\n    mpnn.call({'adj_input' : in_edges_train[:10], 'nod_input': nodes_train[:10]})\n    \n#     mpnn.load_weights(datadir + \"/basicmodelweights/mymodel.h5\")\n\n    history = mpnn.fit({'adj_input' : in_edges_train[:train_size], 'nod_input': nodes_train[:train_size]}, y = out_labels[:train_size], batch_size = batch_size, epochs = epochs, \n             callbacks = [lrate], use_multiprocessing = True, initial_epoch = 0, verbose = 2, \n             validation_data = ({'adj_input' : in_edges_train[train_size:], 'nod_input': nodes_train[train_size:]},out_labels[train_size:]) )\n    \n    preds = mpnn.predict({'adj_input' : in_edges_train[train_size:], 'nod_input': nodes_train[train_size:]}, verbose = 0)\n    \n    return preds, train_size, history","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"preds, train_size, history = train()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"mpnn.save_weights(\"mymodel.h5\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Show the loss"},{"metadata":{"trusted":true},"cell_type":"code","source":"import matplotlib.pyplot as plt\n%matplotlib inline\n# list all data in history\nprint(history.history.keys())\n\n# plt.plot(history.history['lr'])\n# plt.title('learning rate')\n# plt.ylabel('accuracy')\n# plt.xlabel('epoch')\n# plt.legend(['train', 'test'], loc='upper left')\nplt.show()\n# summarize history for loss\nplt.plot(history.history['loss'])\nplt.plot(history.history['val_loss'])\nplt.title('model loss')\nplt.ylabel('loss')\nplt.xlabel('epoch')\nplt.legend(['train', 'test'], loc='upper left')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Save history of trainning"},{"metadata":{"trusted":true},"cell_type":"code","source":"with open('/trainHistoryDict.pkl', 'wb') as file_pi:\n        pickle.dump(history.history, file_pi)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Predict on val set"},{"metadata":{"trusted":true},"cell_type":"code","source":"train = pd.read_csv(datadir + \"champsscalarold/train.csv\")\ntest = pd.read_csv(datadir + \"champsscalarold/test.csv\")\n\ntrain_mol_names = train['molecule_name'].unique()\n\nval = train[train.molecule_name.isin(train_mol_names[train_size:])]\nval_group = val.groupby('molecule_name')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def make_outs(test_group, preds):\n    i = 0\n    x = np.array([])\n    for test_gp, preds in zip(test_group, preds):\n        if (not i%1000):\n            print(i)\n\n        gp = test_gp[1]\n        \n        x = np.append(x, (preds[gp['atom_index_0'].values, gp['atom_index_1'].values] + preds[gp['atom_index_1'].values, gp['atom_index_0'].values])/2.0)\n        \n        i = i+1\n    return x\n\ndef group_mean_log_mae(y_true, y_pred, types, floor=1e-9):\n    \"\"\"\n    Fast metric computation for this competition: https://www.kaggle.com/c/champs-scalar-coupling\n    Code is from this kernel: https://www.kaggle.com/uberkinder/efficient-metric\n    \"\"\"\n    maes = (y_true-y_pred).abs().groupby(types).mean()\n    return np.log(maes.map(lambda x: max(x, floor))).mean()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"max_size = 29\npreds = preds.reshape((-1,max_size, max_size))\nout_unscaled = make_outs(val_group, preds)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"val['pred_scalar_coupling_constant'] = out_unscaled\n\n\ncoups_to_isolate = ['1JHC', '1JHN', '2JHC', '2JHH', '2JHN', '3JHC', '3JHH', '3JHN']\nfor i, coup in enumerate(coups_to_isolate):\n    \n    \n    scale_min = train['scalar_coupling_constant'].loc[train.type == coup].min()\n    scale_max = train['scalar_coupling_constant'].loc[train.type == coup].max()\n    scale_mid = (scale_max + scale_min)/2\n    scale_norm = scale_max - scale_mid\n\n    val.loc[val.type == coup, 'pred_scalar_coupling_constant'] = val['pred_scalar_coupling_constant'].loc[val.type == coup]*scale_norm + scale_mid\n\n    \n    val.loc[val.type == coup, 'pred_scalar_coupling_constant'] = val['pred_scalar_coupling_constant'].loc[val.type == coup]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"for coup in coups_to_isolate:\n    log_mae = group_mean_log_mae(val['scalar_coupling_constant'], val['pred_scalar_coupling_constant'], val['type'][val.type == coup])\n    print(coup,\"\\t\", log_mae)\n    \ntotal = group_mean_log_mae(val['scalar_coupling_constant'], val['pred_scalar_coupling_constant'], val['type'])\nprint(\"\")\nprint(\"Total:\",\"\\t\", total)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Visualize the predict"},{"metadata":{"trusted":true},"cell_type":"code","source":"import seaborn as sns\nimport matplotlib.pyplot as plt\n%matplotlib inline\n\nplot_data = pd.DataFrame(val['scalar_coupling_constant'])\nplot_data.index.name = 'id'\nplot_data['yhat'] = val['pred_scalar_coupling_constant']\nplot_data['type'] = val['type']\n\ndef plot_oof_preds(ctype, llim, ulim):\n        plt.figure(figsize=(6,6))\n        sns.scatterplot(x='scalar_coupling_constant',y='yhat',\n                        data=plot_data.loc[plot_data['type']==ctype,\n                        ['scalar_coupling_constant', 'yhat']]);\n        plt.xlim((llim, ulim))\n        plt.ylim((llim, ulim))\n        plt.plot([llim, ulim], [llim, ulim])\n        plt.xlabel('scalar_coupling_constant')\n        plt.ylabel('predicted')\n        plt.title(f'{ctype}', fontsize=18)\n        plt.savefig(f'{ctype}.svg', format='svg', dpi=800)\n        plt.show()\n\nplot_oof_preds('1JHC', 0, 250)\nplot_oof_preds('1JHN', 0, 100)\nplot_oof_preds('2JHC', -50, 50)\nplot_oof_preds('2JHH', -50, 50)\nplot_oof_preds('2JHN', -25, 25)\nplot_oof_preds('3JHC', -25, 100)\nplot_oof_preds('3JHH', -20, 20)\nplot_oof_preds('3JHN', -15, 15)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Predict on the test set"},{"metadata":{"trusted":true},"cell_type":"code","source":"nodes_test     = np.load(datadir + \"internalgraphdata/nodes_test.npz\" )['arr_0']\nin_edges_test  = np.load(datadir + \"internalgraphdata/in_edges_test.npz\")['arr_0']\nin_edges_test  = in_edges_test.reshape(-1,in_edges_test.shape[1]*in_edges_test.shape[2],in_edges_test.shape[3])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"preds = mpnn.predict({'adj_input' : in_edges_test, 'nod_input': nodes_test}, verbose=1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"np.save(\"preds_kernel.npy\" , preds)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Prediction done!\n\nNow rescale outputs and create submission.csv"},{"metadata":{"trusted":true},"cell_type":"code","source":"test_group = test.groupby('molecule_name')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"preds = preds.reshape((-1,max_size, max_size))\nout_unscaled = make_outs(test_group, preds)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test['scalar_coupling_constant'] = out_unscaled\n\ncoups_to_isolate = ['1JHC', '1JHN', '2JHC', '2JHH', '2JHN', '3JHC', '3JHH', '3JHN']\nfor i, coup in enumerate(coups_to_isolate):\n    \n    \n    scale_min = train['scalar_coupling_constant'].loc[train.type == coup].min()\n    scale_max = train['scalar_coupling_constant'].loc[train.type == coup].max()\n    scale_mid = (scale_max + scale_min)/2\n    scale_norm = scale_max - scale_mid\n\n    test.loc[test.type == coup, 'scalar_coupling_constant'] = test['scalar_coupling_constant'].loc[test.type == coup]*scale_norm + scale_mid\n\n    \n    test.loc[test.type == coup, 'pred_scalar_coupling_constant'] = test['scalar_coupling_constant'].loc[test.type == coup]\n\n\n\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test[['id','scalar_coupling_constant']].to_csv('submission.csv', index=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test[['id','scalar_coupling_constant']].head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.4","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}