{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Local API Emulator is now a module\n\nThis notebook shows you how to use the new `local_api` module. The module emulates the `gresearch_crypto` timeseries API locally. It gives you the flexibility to:\n+ feed different slices of train.csv into it\n+ create as many instances as you want within one session\n+ calculate your LB score (weighted correlation) locally.\n\nIt enforces similar constraints to the real API and produces realistic error messages. This code extends and updates a previous [notebook version](https://www.kaggle.com/jagofc/local-api-emulator).\n\nFor a quick introduction to importing utility scripts see [this intro video](https://www.youtube.com/watch?v=C4h88PfN5jA&ab_channel=Kaggle).\n","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2021-12-18T16:24:01.340788Z","iopub.execute_input":"2021-12-18T16:24:01.341213Z","iopub.status.idle":"2021-12-18T16:24:01.361741Z","shell.execute_reply.started":"2021-12-18T16:24:01.341111Z","shell.execute_reply":"2021-12-18T16:24:01.360507Z"}}},{"cell_type":"markdown","source":"# Demo\n\nTo use this in your notebook you need to:\n1. In the notebook menu select File > Add Utility Script\n2. Search for \"local_api\" and click Add.\n3. Import `local_api` as you would any module.","metadata":{}},{"cell_type":"code","source":"import time\nimport numpy as np\nimport pandas as pd\nimport gresearch_crypto\nfrom tqdm import tqdm\n\nimport local_api as la","metadata":{"_kg_hide-input":false,"execution":{"iopub.status.busy":"2022-01-08T10:21:17.653787Z","iopub.execute_input":"2022-01-08T10:21:17.654084Z","iopub.status.idle":"2022-01-08T10:21:17.663839Z","shell.execute_reply.started":"2022-01-08T10:21:17.654053Z","shell.execute_reply":"2022-01-08T10:21:17.66321Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"`local_api` has some utility functions, constants and the main `API` class. Check out its contents using tab completion. Here we use the utility function `read_csv_slice`.","metadata":{}},{"cell_type":"code","source":"train_df = la.read_csv_slice('../input/g-research-crypto-forecasting/train.csv')","metadata":{"execution":{"iopub.status.busy":"2022-01-08T10:02:27.015191Z","iopub.execute_input":"2022-01-08T10:02:27.015493Z","iopub.status.idle":"2022-01-08T10:03:26.73817Z","shell.execute_reply.started":"2022-01-08T10:02:27.015462Z","shell.execute_reply":"2022-01-08T10:03:26.736841Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The (approx.) public LB timestamp window is available as a constant:","metadata":{}},{"cell_type":"code","source":"la.LB_WINDOW","metadata":{"execution":{"iopub.status.busy":"2022-01-08T10:10:55.773527Z","iopub.execute_input":"2022-01-08T10:10:55.774126Z","iopub.status.idle":"2022-01-08T10:10:55.779867Z","shell.execute_reply.started":"2022-01-08T10:10:55.774093Z","shell.execute_reply":"2022-01-08T10:10:55.77903Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Create an example API instance using the public LB window:","metadata":{"execution":{"iopub.status.busy":"2021-12-19T20:17:41.332783Z","iopub.execute_input":"2021-12-19T20:17:41.333156Z","iopub.status.idle":"2021-12-19T20:17:41.362971Z","shell.execute_reply.started":"2021-12-19T20:17:41.333029Z","shell.execute_reply":"2021-12-19T20:17:41.361171Z"}}},{"cell_type":"code","source":"api = la.API(train_df, use_window=la.LB_WINDOW)","metadata":{"execution":{"iopub.status.busy":"2022-01-08T10:12:03.753859Z","iopub.execute_input":"2022-01-08T10:12:03.754817Z","iopub.status.idle":"2022-01-08T10:12:07.203784Z","shell.execute_reply.started":"2022-01-08T10:12:03.754777Z","shell.execute_reply":"2022-01-08T10:12:07.202925Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Get the first batch of data.","metadata":{}},{"cell_type":"code","source":"(data_df, pred_df) = next(api)\ndata_df.head()","metadata":{"execution":{"iopub.status.busy":"2022-01-08T10:12:23.856707Z","iopub.execute_input":"2022-01-08T10:12:23.857115Z","iopub.status.idle":"2022-01-08T10:12:23.884296Z","shell.execute_reply.started":"2022-01-08T10:12:23.857065Z","shell.execute_reply":"2022-01-08T10:12:23.882843Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We'll get an error if we try to continue on to the next batch without making our predictions for the current batch. *Commented out so that the notebook doesn't Fail.*","metadata":{}},{"cell_type":"code","source":"# next(api)","metadata":{"execution":{"iopub.status.busy":"2021-12-18T18:50:21.069998Z","iopub.execute_input":"2021-12-18T18:50:21.070356Z","iopub.status.idle":"2021-12-18T18:50:21.075421Z","shell.execute_reply.started":"2021-12-18T18:50:21.070313Z","shell.execute_reply":"2021-12-18T18:50:21.07455Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let's make a dummy prediction using `pred_df`.","metadata":{}},{"cell_type":"code","source":"api.predict(pred_df)","metadata":{"execution":{"iopub.status.busy":"2022-01-08T10:12:33.718373Z","iopub.execute_input":"2022-01-08T10:12:33.719017Z","iopub.status.idle":"2022-01-08T10:12:33.724375Z","shell.execute_reply.started":"2022-01-08T10:12:33.718973Z","shell.execute_reply":"2022-01-08T10:12:33.723425Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now you can continue to iterate. Lets get another slice of data and make another dummy prediction:","metadata":{}},{"cell_type":"code","source":"(data_df, pred_df) = next(api)\napi.predict(pred_df)","metadata":{"execution":{"iopub.status.busy":"2022-01-08T10:12:38.313689Z","iopub.execute_input":"2022-01-08T10:12:38.314506Z","iopub.status.idle":"2022-01-08T10:12:38.323744Z","shell.execute_reply.started":"2022-01-08T10:12:38.314464Z","shell.execute_reply":"2022-01-08T10:12:38.32279Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Your predictions are stored by the API. Let's just look at the first two prediction batches we made:","metadata":{}},{"cell_type":"code","source":"api.predictions","metadata":{"execution":{"iopub.status.busy":"2022-01-08T10:12:41.460815Z","iopub.execute_input":"2022-01-08T10:12:41.461109Z","iopub.status.idle":"2022-01-08T10:12:41.473263Z","shell.execute_reply.started":"2022-01-08T10:12:41.461078Z","shell.execute_reply":"2022-01-08T10:12:41.472299Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The API also has a length method, which tracks the number of timestamps still to be served:","metadata":{}},{"cell_type":"code","source":"len(api)","metadata":{"execution":{"iopub.status.busy":"2022-01-08T10:12:49.881165Z","iopub.execute_input":"2022-01-08T10:12:49.882078Z","iopub.status.idle":"2022-01-08T10:12:49.888185Z","shell.execute_reply.started":"2022-01-08T10:12:49.882012Z","shell.execute_reply":"2022-01-08T10:12:49.887084Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Note that you don't need to to restart the notebook kernel in order to make a new emulator (or to refresh the current one), in contrast with the `gresearch_crypto` env. Here is an instance on a different date window:","metadata":{}},{"cell_type":"code","source":"example_window = (la.datestring_to_timestamp(\"2020-05-01T00:00\"),\n                  la.datestring_to_timestamp(\"2020-05-14T00:00\"))\n\napi2 = la.API(train_df, use_window=example_window)","metadata":{"execution":{"iopub.status.busy":"2022-01-08T10:16:08.507496Z","iopub.execute_input":"2022-01-08T10:16:08.507888Z","iopub.status.idle":"2022-01-08T10:16:10.642089Z","shell.execute_reply.started":"2022-01-08T10:16:08.507838Z","shell.execute_reply":"2022-01-08T10:16:10.641115Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Example main loop\n\nAn example loop on `example_window` (making dummy predictions of Target=0) with a timing estimate for 100 days worth of data.","metadata":{}},{"cell_type":"code","source":"api = la.API(train_df, use_window=example_window)\n\nstart_time = time.time()\n\nfor (data_df, pred_df) in tqdm(api):\n    pred_df['Target'] = 0.\n    api.predict(pred_df)\n    \nfinish_time = time.time()\n\ntotal_time = finish_time - start_time\niter_speed = api.init_num_times/total_time\n\nprint(f\"Iterations/s = {round(iter_speed, 2)}.\")\ntest_iters = 60 * 24 * 100\nprint(f\"Expected number of iterations in test set is approx. {test_iters}\",\n      f\"which will take {round(test_iters / (iter_speed * 3600), 2)} hours\",\n      \"using this API emulator while making dummy predictions.\")","metadata":{"execution":{"iopub.status.busy":"2022-01-08T10:23:42.110391Z","iopub.execute_input":"2022-01-08T10:23:42.110663Z","iopub.status.idle":"2022-01-08T10:24:56.76856Z","shell.execute_reply.started":"2022-01-08T10:23:42.110634Z","shell.execute_reply":"2022-01-08T10:24:56.767018Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The API has a `score` method. This returns:\n+ a dataframe containing your predictions, the targets, and weights,\n+ the LB score: weighted correlation between predictions and targets.","metadata":{}},{"cell_type":"code","source":"df, score = api.score()\nprint(f\"Your LB score is {round(score, 4)}\")","metadata":{"execution":{"iopub.status.busy":"2022-01-08T10:25:19.291351Z","iopub.execute_input":"2022-01-08T10:25:19.291655Z","iopub.status.idle":"2022-01-08T10:25:26.599352Z","shell.execute_reply.started":"2022-01-08T10:25:19.291625Z","shell.execute_reply":"2022-01-08T10:25:26.598425Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## TL;DR example loop with random predictions","metadata":{}},{"cell_type":"code","source":"api = la.API(train_df, use_window=la.LB_WINDOW)\n\nfor (data_df, pred_df) in tqdm(api):\n    pred_df['Target'] = np.random.randn(len(pred_df), 1)\n    api.predict(pred_df)\n    \ndf, score = api.score()\n\nprint(f\"Your LB score is {round(score, 4)}\")","metadata":{"execution":{"iopub.status.busy":"2021-12-19T22:04:29.34672Z","iopub.execute_input":"2021-12-19T22:04:29.347071Z","iopub.status.idle":"2021-12-19T22:12:36.1312Z","shell.execute_reply.started":"2021-12-19T22:04:29.347038Z","shell.execute_reply":"2021-12-19T22:12:36.130282Z"},"trusted":true},"execution_count":null,"outputs":[]}]}