{"cells":[{"metadata":{},"cell_type":"markdown","source":"### VERSIONS\n* **V1**: One Net (m1) for all `delta` (2 epochs) with one-step neighbors\n* **V2**: One Net per (m2) `delta` (2 epochs) with one-step neighbors\n* **V3**: One Net per (m2) `delta` (10 epochs) with one-step neighbors\n* **V4**: One Net per (m2) `delta` (10 epochs) with **two**-step neighbors\n* **V5**: One Conv Net per (c1) `delta` (15 epochs) with **two**-step neighbors","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## Quick FE explanation\n\nHere we are given a $n \\times n$ grid, where the next state of cells is determined by its neighbors present state. So therefore reversely, the previous state of a cell can be infered by the present state of its neighbors. \n\nThat is for a given cell $(i, j)$, we can possibly deduce its past state by its one-step neighbors defined by:\n$$ \\mathcal{V}^{(1)}(i, j) = \\left \\lbrace (i', j') \\mid i-1 \\leq i' \\leq i+1 \\text{ and } j-1 \\leq j' \\leq j+1 \\right \\rbrace  $$\n\nSimilarly the state of cell $\\delta$ steps backward can be known from its $\\delta$-step neighbor defined as follow:\n\n$$ \\mathcal{V}^{(\\delta)}(i, j) = \\left \\lbrace (i', j') \\mid i-\\delta \\leq i' \\leq i+\\delta \\text{ and } j-\\delta \\leq j' \\leq j+\\delta \\right \\rbrace  $$\n\n## How to apply ?\n\nThat is:\n* for the grid with `delta==1`, we can use the one-step neighors state as features \n* for the grid with `delta==2`, we can use the two-step neighors state as features\n* ...\n* for the grid with `delta==5`, we can use the 5-step neighors state as features\n\n**NB**: In this notebook, we will use the one-step neighborhood for every `delta` for a quick probe of LB.\n","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import pandas as pd\nfrom tqdm import tqdm\nimport gc\nimport numpy as np\nimport matplotlib.pyplot as plt\n\n#TF\nimport tensorflow as tf\nimport tensorflow.keras.layers as L\nimport tensorflow.keras.models as M","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Useful functions","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def get_features(delta=1):\n    \"\"\"\n    Simple code to generate delta-neighbors states\n    \"\"\"\n    x_tr = []\n    x_te = []\n    nsteps = len(NEIGHBORS)\n    \n    #for idx in tqdm(range(nsteps)):\n    for idx in range(nsteps):\n        right, up = NEIGHBORS[idx]\n        cols = [f\"stop_{i + right + 25*up}\" if (i + right + 25*up >=0 and i+ right + 25*up <625)\\\n                else \"pad\" for i in range(625)]\n        new_cols = [f\"n{idx}_{i}\" for i in range(625)]\n        x_tr.append(tr.loc[tr.delta==delta, cols].values[:,:, np.newaxis])\n        x_te.append(te.loc[te.delta==delta, cols].values[:,:, np.newaxis])\n    #==================#\n    x_tr = np.concatenate(x_tr, axis=2)\n    x_te = np.concatenate(x_te, axis=2)\n    y_tr = tr.loc[tr.delta==delta, INIT_COLS].values\n    gc.collect()\n    return x_tr, x_te, y_tr\n#======================================\ndef make_model(nh=9):\n    \"\"\"\n    Simple MLP architecture\n    \"\"\"\n    \n    nn = 100\n    inp = L.Input(name=\"grid\", shape=(625,nh))\n    d1 = L.Dense(nn, name='d1', activation=\"relu\")(inp)\n    d2 = L.Dense(nn, name='d2', activation=\"relu\")(d1)\n    preds = L.Dense(1, name='preds', activation=\"sigmoid\")(d2)\n    \n    model = M.Model(inp, preds, name=\"ANN\")\n    model.compile(loss=\"binary_crossentropy\", optimizer=\"adam\", metrics=[\"accuracy\"])\n    return model\n#=======================\ndef make_conv(nh=1, h=2, nf=128, ks=5):\n    inp = L.Input(name=\"input\", shape=(25, 25, nh))\n    \n    for idx in range(h):\n        if idx==0:\n            x = L.Conv2D(nh, ks, padding=\"same\", activation=\"relu\", name=f\"conv{idx+1}\")(inp)\n        else:\n            x = L.Conv2D(nh, ks, padding=\"same\", activation=\"relu\", name=f\"conv{idx+1}\")(x)\n        x = L.BatchNormalization(name=f\"norm{idx+1}\")(x)\n    #\n    \n    x = L.Conv2D(1, ks, padding=\"same\", activation=\"sigmoid\", name=\"last_conv\")(x)\n    preds = L.Reshape(name=\"preds\", target_shape=(625, 1))(x)\n    \n    model = M.Model(inp, preds, name=\"CNN\")\n    model.compile(loss=\"binary_crossentropy\", optimizer=\"adam\", metrics=[\"accuracy\"])\n    return model\n#====================================================#\ndef line_search(y_true, y_prob, start, end, breaks):\n    \"\"\"\n    Line search of probability threshod for maximal accuracy\n    \"\"\"\n    space = np.linspace(start, end, breaks)\n    best_threshold = 0.\n    best_score = 0.\n    \n    for threshold in space:\n        acc = np.mean( y_true == (y_prob > threshold) )\n        if acc > best_score:\n            best_score = acc\n            best_threshold = threshold\n    print(f\"threshold: {best_threshold}, score: {best_score}\")\n    return best_threshold, best_score\n#===========================================#","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Learning Phase (One net per delta)","execution_count":null},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"%%time\ntr = pd.read_csv(\"../input/conways-reverse-game-of-life-2020/train.csv\")\nte = pd.read_csv(\"../input/conways-reverse-game-of-life-2020/test.csv\")\ntr[\"pad\"] = -1\nte[\"pad\"] = -1\nINIT_COLS = [f\"start_{i}\" for i in range(625)]\nSTOP_COLS = [f\"stop_{i}\" for i in range(625)]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"NF = 2\nNEIGHBORS = [(right, up) for up in range(-NF, NF+1) for right in range(-NF, NF+1)]\nnh = len(NEIGHBORS)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"grid = te[STOP_COLS].values\ndelta_te = te['delta'].values\nmiddle = (nh - 1) // 2","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\nCONV = True\nfor idx in range(5):\n    print(\"===================================\\n\")\n    print(f\"DELTA = {idx + 1}\")\n    x_tr, x_te, y_tr = get_features(delta=idx+1)\n    if CONV:\n        x_tr = x_tr.reshape((-1, 25, 25,nh ))\n        x_te = x_te.reshape((-1, 25, 25,nh ))\n        net = make_conv(nh=nh, h=2, nf=128, ks=5)\n    else:\n        net = make_model(nh)\n    net.fit(x_tr, y_tr, batch_size=100, epochs=15)\n    if not CONV:\n        print(\"naive accuracy\",(y_tr==x_tr[:,:,middle]).mean())\n    prob_tr = net.predict(x_tr, batch_size=50, verbose=1)[:,:,0]\n    bt, bs = line_search(y_tr, prob_tr, 0.3, 0.6, 7)\n    preds = net.predict(x_te, batch_size=50, verbose=1)[:,:,0]\n    grid[delta_te==idx+1] = (preds>bt).astype(int)\n    del x_tr, x_te, y_tr\n    gc.collect()\n#==========================","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Submission","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"sub = te[['id']].copy()\ntp = pd.DataFrame(grid, columns=INIT_COLS)\nsub = sub.join(tp)\ndel tp\ngc.collect()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sub.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sub.to_csv(\"submission.csv\", index=False)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}