{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Optuna tutorial for begginers: hyperparameter optimization framework\n\nWhen I try building a model (XGBoost, LightGBM, CatBoost, Neural Network etc...), I always face an issue of how to tune these hyperparameters?
\nSome people may be trying to set parameters manually to see if the score improves or not.\n\n**In this tutorial, I will introduce [optuna](https://optuna.org/), *Define-by-Run Hyperparameter Optimization Framework* for automated hyperparameter tuning.**\n\n**

UPDATE: added 6. Visualize study history to analayze the hyperparams-performance relationship section, please check!!!

**"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"from IPython.display import HTML\n\nHTML('')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Table of Contents:\n\nBelow code forked and a bit modified from [ASHRAE: Training LGBM by meter type](https://www.kaggle.com/corochann/ashrae-training-lgbm-by-meter-type) kernel (taken from version 25) until data preprocessing.
\nPlease jump to **[1. Define objective function](#id1)** to start learning optuna usage!\n\n**[ASHRAE - Great Energy Predictor III](#title)**
\n**[Data Preprocessing](#id0)**
\n**[1. Define \"objective\" function](#id1)**
\n**[2. Use \"trial\" module to define hyperparameters dynamically!](#id2)**
\n**[3. Make \"study\" and let optimize!](#id3)**
\n**[4. [Advanced] Pruning unpromising trials for more faster search](#id4)**
\n**[5. Check study history to get best hyperparameters](#id5)**
\n**[6. Visualize study history to analayze the hyperparams-performance relationship](#id6)**
\n**[More to go](#id10)**
"},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"\"\"\"\nSome visualization methods used in this tutorial is supported in optuna from v0.18.0 released recently!\nHowever this kaggle kernel pre-installs version 0.16.0\n\"\"\"\n!pip install optuna==0.18.1","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n# ASHRAE - Great Energy Predictor III\n\nOur aim in this competition is to predict energy consumption of buildings.\n\nThere are 4 types of energy to predict:\n\n - 0: electricity\n - 1: chilledwater\n - 2: steam\n - 3: hotwater\n\nElectricity and water consumption may have different behavior!\nSo I tried to separately train & predict the model.\n\nI moved previous [ASHRAE: Simple LGBM submission](https://www.kaggle.com/corochann/ashrae-simple-lgbm-submission) kernel."},{"metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_kg_hide-input":true,"_kg_hide-output":true,"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","trusted":true},"cell_type":"code","source":"import gc\nimport os\nfrom pathlib import Path\nimport random\nimport sys\n\nfrom tqdm import tqdm_notebook as tqdm\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n\nfrom IPython.core.display import display, HTML\n\n# --- plotly ---\nfrom plotly import tools, subplots\nimport plotly.offline as py\npy.init_notebook_mode(connected=True)\nimport plotly.graph_objs as go\nimport plotly.express as px\nimport plotly.figure_factory as ff\n\n# --- models ---\nfrom sklearn import preprocessing\nfrom sklearn.model_selection import KFold\nimport lightgbm as lgb\nimport xgboost as xgb\nimport catboost as cb\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"# Original code from https://www.kaggle.com/gemartin/load-data-reduce-memory-usage by @gemartin\n# Modified to support timestamp type, categorical type\n# Modified to add option to use float16 or not. feather format does not support float16.\nfrom pandas.api.types import is_datetime64_any_dtype as is_datetime\nfrom pandas.api.types import is_categorical_dtype\n\ndef reduce_mem_usage(df, use_float16=False):\n \"\"\" iterate through all the columns of a dataframe and modify the data type\n to reduce memory usage. \n \"\"\"\n start_mem = df.memory_usage().sum() / 1024**2\n print('Memory usage of dataframe is {:.2f} MB'.format(start_mem))\n \n for col in df.columns:\n if is_datetime(df[col]) or is_categorical_dtype(df[col]):\n # skip datetime type or categorical type\n continue\n col_type = df[col].dtype\n \n if col_type != object:\n c_min = df[col].min()\n c_max = df[col].max()\n if str(col_type)[:3] == 'int':\n if c_min > np.iinfo(np.int8).min and c_max < np.iinfo(np.int8).max:\n df[col] = df[col].astype(np.int8)\n elif c_min > np.iinfo(np.int16).min and c_max < np.iinfo(np.int16).max:\n df[col] = df[col].astype(np.int16)\n elif c_min > np.iinfo(np.int32).min and c_max < np.iinfo(np.int32).max:\n df[col] = df[col].astype(np.int32)\n elif c_min > np.iinfo(np.int64).min and c_max < np.iinfo(np.int64).max:\n df[col] = df[col].astype(np.int64) \n else:\n if use_float16 and c_min > np.finfo(np.float16).min and c_max < np.finfo(np.float16).max:\n df[col] = df[col].astype(np.float16)\n elif c_min > np.finfo(np.float32).min and c_max < np.finfo(np.float32).max:\n df[col] = df[col].astype(np.float32)\n else:\n df[col] = df[col].astype(np.float64)\n else:\n df[col] = df[col].astype('category')\n\n end_mem = df.memory_usage().sum() / 1024**2\n print('Memory usage after optimization is: {:.2f} MB'.format(end_mem))\n print('Decreased by {:.1f}%'.format(100 * (start_mem - end_mem) / start_mem))\n \n return df","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"!ls ../input","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Fast data loading\n\nThis kernel uses the preprocessed data from my previous kernel, [\nASHRAE: feather format for fast loading](https://www.kaggle.com/corochann/ashrae-feather-format-for-fast-loading), to accelerate data loading!"},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"%%time\nroot = Path('../input/ashrae-feather-format-for-fast-loading')\n\ntrain_df = pd.read_feather(root/'train.feather')\nweather_train_df = pd.read_feather(root/'weather_train.feather')\nbuilding_meta_df = pd.read_feather(root/'building_metadata.feather')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"train_df['date'] = train_df['timestamp'].dt.date\ntrain_df['meter_reading_log1p'] = np.log1p(train_df['meter_reading'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"np.sum(train_df['meter_reading_log1p'].values < 0)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"def plot_date_usage(train_df, meter=0, building_id=0):\n train_temp_df = train_df[train_df['meter'] == meter]\n train_temp_df = train_temp_df[train_temp_df['building_id'] == building_id] \n train_temp_df_meter = train_temp_df.groupby('date')['meter_reading_log1p'].sum()\n train_temp_df_meter = train_temp_df_meter.to_frame().reset_index()\n fig = px.line(train_temp_df_meter, x='date', y='meter_reading_log1p')\n fig.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"plot_date_usage(train_df, meter=0, building_id=0)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Removing weired data on site_id 0\n\nAs you can see above, this data looks weired until May 20. It is reported in [this discussion](https://www.kaggle.com/c/ashrae-energy-prediction/discussion/113054#656588) by @barnwellguy that **All electricity meter is 0 until May 20 for site_id == 0**. I will remove these data from training data.\n\nIt corresponds to `building_id <= 104`."},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"building_meta_df[building_meta_df.site_id == 0]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"train_df = train_df.query('not (building_id <= 104 & meter == 0 & timestamp <= \"2016-05-20\")')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n# Data preprocessing\n\nNow, Let's try building GBDT (Gradient Boost Decision Tree) model to predict `meter_reading_log1p`. I will try using LightGBM in this notebook.\n\n\n[UPDATE]\n - Processing of 'weekend'\n - Take building stats by building_id **and meter type**\n - Align timestamp in weather data, by@nz0722 https://www.kaggle.com/nz0722/aligned-timestamp-lgbm-by-meter-type"},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"debug = False","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Add time feature"},{"metadata":{},"cell_type":"markdown","source":"Some features introduced in https://www.kaggle.com/ryches/simple-lgbm-solution by @ryches\n\nFeatures that are likely predictive:\n\n#### Weather\n\n- time of day\n- holiday\n- weekend\n- cloud_coverage + lags\n- dew_temperature + lags\n- precip_depth + lags\n- sea_level_pressure + lags\n- wind_direction + lags\n- wind_speed + lags\n\n#### Train\n\n- max, mean, min, std of the specific building historically\n\n\n\nHowever we should be careful of putting time feature, since we have only 1 year data in training,\nincluding `date` makes overfiting to training data.\n\nHow about `month`? It may be better to check performance by cross validation.\nI go not using this data in this kernel for robust modeling."},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"def preprocess(df):\n df[\"hour\"] = df[\"timestamp\"].dt.hour\n# df[\"day\"] = df[\"timestamp\"].dt.day\n df[\"month\"] = df[\"timestamp\"].dt.month\n df[\"dayofweek\"] = df[\"timestamp\"].dt.dayofweek\n df[\"weekend\"] = df[\"dayofweek\"] >= 5\n\n# hour_rad = df[\"hour\"].values / 24. * 2 * np.pi\n# df[\"hour_sin\"] = np.sin(hour_rad)\n# df[\"hour_cos\"] = np.cos(hour_rad)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":false,"_kg_hide-output":true},"cell_type":"code","source":"preprocess(train_df)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"# take stats by ONLY building_id. meter type is merged in this script...\n\n# df_group = train_df.groupby('building_id')['meter_reading_log1p']\n# building_mean = df_group.mean().astype(np.float16)\n# building_median = df_group.median().astype(np.float16)\n# building_min = df_group.min().astype(np.float16)\n# building_max = df_group.max().astype(np.float16)\n# building_std = df_group.std().astype(np.float16)\n\n# train_df['building_mean'] = train_df['building_id'].map(building_mean)\n# train_df['building_median'] = train_df['building_id'].map(building_median)\n# train_df['building_min'] = train_df['building_id'].map(building_min)\n# train_df['building_max'] = train_df['building_id'].map(building_max)\n# train_df['building_std'] = train_df['building_id'].map(building_std)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_group = train_df.groupby(['building_id', 'meter'])['meter_reading_log1p']\nbuilding_mean = df_group.mean().astype(np.float16)\nbuilding_median = df_group.median().astype(np.float16)\nbuilding_min = df_group.min().astype(np.float16)\nbuilding_max = df_group.max().astype(np.float16)\nbuilding_std = df_group.std().astype(np.float16)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"building_stats_df = pd.concat([building_mean, building_median, building_min, building_max, building_std], axis=1,\n keys=['building_mean', 'building_median', 'building_min', 'building_max', 'building_std']).reset_index()\ntrain_df = pd.merge(train_df, building_stats_df, on=['building_id', 'meter'], how='left', copy=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Fill Nan value in weather dataframe by interpolation\n\n\nweather data has a lot of NaNs!!\n\n![](http://)I tried to fill these values by **interpolating** data."},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"weather_train_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"weather_train_df.isna().sum()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"weather_test_df = pd.read_feather(root/'weather_test.feather')\nweather = pd.concat([weather_train_df, weather_test_df],ignore_index=True)\ndel weather_test_df\nweather_key = ['site_id', 'timestamp']","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"# https://www.kaggle.com/nz0722/aligned-timestamp-lgbm-by-meter-type\n\ntemp_skeleton = weather[weather_key + ['air_temperature']].drop_duplicates(subset=weather_key).sort_values(by=weather_key).copy()\n\n# calculate ranks of hourly temperatures within date/site_id chunks\ntemp_skeleton['temp_rank'] = temp_skeleton.groupby(['site_id', temp_skeleton.timestamp.dt.date])['air_temperature'].rank('average')\n\n# create a dataframe of site_ids (0-16) x mean hour rank of temperature within day (0-23)\ndf_2d = temp_skeleton.groupby(['site_id', temp_skeleton.timestamp.dt.hour])['temp_rank'].mean().unstack(level=1)\n\n# Subtract the columnID of temperature peak by 14, getting the timestamp alignment gap.\nsite_ids_offsets = pd.Series(df_2d.values.argmax(axis=1) - 14)\nsite_ids_offsets.index.name = 'site_id'\n\ndef timestamp_align(df):\n df['offset'] = df.site_id.map(site_ids_offsets)\n df['timestamp_aligned'] = (df.timestamp - pd.to_timedelta(df.offset, unit='H'))\n df['timestamp'] = df['timestamp_aligned']\n del df['timestamp_aligned']\n return df\n\ndel weather\ndel temp_skeleton\ngc.collect()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":false,"_kg_hide-output":false},"cell_type":"code","source":"weather_train_df = timestamp_align(weather_train_df)\nweather_train_df = weather_train_df.groupby('site_id').apply(lambda group: group.interpolate(limit_direction='both'))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Seems number of nan has reduced by `interpolate` but some property has never appear in specific `site_id`, and nan remains for these features."},{"metadata":{},"cell_type":"markdown","source":"## lags\n\nAdding some lag feature"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"def add_lag_feature(weather_df, window=3):\n group_df = weather_df.groupby('site_id')\n cols = ['air_temperature', 'cloud_coverage', 'dew_temperature', 'precip_depth_1_hr', 'sea_level_pressure', 'wind_direction', 'wind_speed']\n rolled = group_df[cols].rolling(window=window, min_periods=0)\n lag_mean = rolled.mean().reset_index().astype(np.float16)\n lag_max = rolled.max().reset_index().astype(np.float16)\n lag_min = rolled.min().reset_index().astype(np.float16)\n lag_std = rolled.std().reset_index().astype(np.float16)\n for col in cols:\n weather_df[f'{col}_mean_lag{window}'] = lag_mean[col]\n weather_df[f'{col}_max_lag{window}'] = lag_max[col]\n weather_df[f'{col}_min_lag{window}'] = lag_min[col]\n weather_df[f'{col}_std_lag{window}'] = lag_std[col]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-output":true},"cell_type":"code","source":"add_lag_feature(weather_train_df, window=3)\nadd_lag_feature(weather_train_df, window=72)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"weather_train_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":false,"_kg_hide-output":false},"cell_type":"code","source":"weather_train_df.columns","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"# categorize primary_use column to reduce memory on merge...\n\nprimary_use_list = building_meta_df['primary_use'].unique()\nprimary_use_dict = {key: value for value, key in enumerate(primary_use_list)} \nprint('primary_use_dict: ', primary_use_dict)\nbuilding_meta_df['primary_use'] = building_meta_df['primary_use'].map(primary_use_dict)\n\ngc.collect()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"reduce_mem_usage(train_df, use_float16=True)\nreduce_mem_usage(building_meta_df, use_float16=True)\nreduce_mem_usage(weather_train_df, use_float16=True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"building_meta_df.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Train model\n\nTo win in kaggle competition, how to evaluate your model is important.\nWhat kind of cross validation strategy is suitable for this competition? This is time series data, so it is better to consider time-splitting.\n\nHowever this notebook is for simple tutorial, so I will proceed with KFold splitting without shuffling, so that at least near-term data is not included in validation."},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"category_cols = ['building_id', 'site_id', 'primary_use'] # , 'meter'\nweather_cols = [\n 'air_temperature', 'cloud_coverage',\n 'dew_temperature', 'precip_depth_1_hr', 'sea_level_pressure',\n 'wind_direction', 'wind_speed', 'air_temperature_mean_lag72',\n 'air_temperature_max_lag72', 'air_temperature_min_lag72',\n 'air_temperature_std_lag72', 'cloud_coverage_mean_lag72',\n 'dew_temperature_mean_lag72', 'precip_depth_1_hr_mean_lag72',\n 'sea_level_pressure_mean_lag72', 'wind_direction_mean_lag72',\n 'wind_speed_mean_lag72', 'air_temperature_mean_lag3',\n 'air_temperature_max_lag3',\n 'air_temperature_min_lag3', 'cloud_coverage_mean_lag3',\n 'dew_temperature_mean_lag3',\n 'precip_depth_1_hr_mean_lag3', 'sea_level_pressure_mean_lag3',\n 'wind_direction_mean_lag3', 'wind_speed_mean_lag3']\nfeature_cols = ['square_feet', 'year_built'] + [\n 'hour', 'weekend', 'dayofweek', # 'month'\n 'building_median'] + weather_cols","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"def create_X_y(train_df, target_meter):\n target_train_df = train_df[train_df['meter'] == target_meter]\n target_train_df = target_train_df.merge(building_meta_df, on='building_id', how='left')\n target_train_df = target_train_df.merge(weather_train_df, on=['site_id', 'timestamp'], how='left')\n X_train = target_train_df[feature_cols + category_cols]\n y_train = target_train_df['meter_reading_log1p'].values\n\n del target_train_df\n return X_train, y_train","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n# 1. Define \"objective\" function\n\nTo start hyperparameter tuning, we need an objective function to optimize the score.
\n**`objective` method needs to receive `trial` object as args, and return \"score\" to be optimized.**
\n(What is trial module? it is explained in next section) \n\n```\ndef objective(trial, ...):\n # calculate score...\n return score\n```\n\nBelow example, I train LightGBM model (only for electricity meter), get best validation score, and return this validation score as the final score.\n\n**The `objective` function is called many times** by `optuna` framework to try different hyperparameters, to search best hyperparameters."},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"import optuna\nfrom optuna import Trial\n\noptuna.__version__","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"debug = False\n\ntrain_df_original = train_df\n# Only use 10000 data,,, for fast computation for debugging.\ntrain_df = train_df.sample(10000)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def objective(trial: Trial, fast_check=True, target_meter=0, return_info=False):\n folds = 5\n seed = 666\n shuffle = False\n kf = KFold(n_splits=folds, shuffle=shuffle, random_state=seed)\n\n X_train, y_train = create_X_y(train_df, target_meter=target_meter)\n y_valid_pred_total = np.zeros(X_train.shape[0])\n gc.collect()\n print('target_meter', target_meter, X_train.shape)\n\n cat_features = [X_train.columns.get_loc(cat_col) for cat_col in category_cols]\n print('cat_features', cat_features)\n\n models = []\n valid_score = 0\n for train_idx, valid_idx in kf.split(X_train, y_train):\n train_data = X_train.iloc[train_idx,:], y_train[train_idx]\n valid_data = X_train.iloc[valid_idx,:], y_train[valid_idx]\n\n print('train', len(train_idx), 'valid', len(valid_idx))\n # model, y_pred_valid, log = fit_cb(train_data, valid_data, cat_features=cat_features, devices=[0,])\n model, y_pred_valid, log = fit_lgbm(trial, train_data, valid_data, cat_features=category_cols,\n num_rounds=1000)\n y_valid_pred_total[valid_idx] = y_pred_valid\n models.append(model)\n gc.collect()\n valid_score += log[\"valid/l2\"]\n if fast_check:\n break\n valid_score /= len(models)\n if return_info:\n return valid_score, models, y_pred_valid, y_train\n else:\n return valid_score","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Here I passed `trial` module to `fit_lgbm` function, which is the core training code and defines hyperparameters.
\nLet's see inside of `fit_lgbm` module next."},{"metadata":{},"cell_type":"markdown","source":"\n# 2. Use \"trial\" module to define hyperparameters dynamically!\n\ntrial module can be used to get hyperparameters.\nAs shown in below figure, we just need to get hyperparameters from `trial` module where we want to use hyper parameters!\n\n\nfrom [https://optuna.org/](https://optuna.org/)\n\nThis scheme is called \"Define-by-Run\" which makes user to intuitively write code to get hyperparameters, instead of defining whole search space in advance.\n\nYou can call these methods to get hyperparametes (ref: [Defining Parameter Spaces](https://optuna.readthedocs.io/en/latest/tutorial/configurations.html#defining-parameter-spaces)):\n\n # Categorical parameter\n optimizer = trial.suggest_categorical('optimizer', ['MomentumSGD', 'Adam'])\n\n # Int parameter\n num_layers = trial.suggest_int('num_layers', 1, 3)\n\n # Uniform parameter\n dropout_rate = trial.suggest_uniform('dropout_rate', 0.0, 1.0)\n\n # Loguniform parameter\n learning_rate = trial.suggest_loguniform('learning_rate', 1e-5, 1e-2)\n\n # Discrete-uniform parameter\n drop_path_rate = trial.suggest_discrete_uniform('drop_path_rate', 0.0, 1.0, 0.1)\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Referred https://github.com/pfnet/optuna/blob/master/examples/lightgbm_simple.py\n\ndef fit_lgbm(trial, train, val, devices=(-1,), seed=None, cat_features=None, num_rounds=1500):\n \"\"\"Train Light GBM model\"\"\"\n X_train, y_train = train\n X_valid, y_valid = val\n metric = 'l2'\n params = {\n 'num_leaves': trial.suggest_int('num_leaves', 2, 256),\n 'objective': 'regression',\n# 'max_depth': -1,\n 'learning_rate': 0.1,\n \"boosting\": \"gbdt\",\n 'lambda_l1': trial.suggest_loguniform('lambda_l1', 1e-8, 10.0),\n 'lambda_l2': trial.suggest_loguniform('lambda_l2', 1e-8, 10.0),\n \"bagging_freq\": 5,\n \"bagging_fraction\": trial.suggest_uniform('bagging_fraction', 0.1, 1.0),\n \"feature_fraction\": trial.suggest_uniform('feature_fraction', 0.4, 1.0),\n \"metric\": metric,\n \"verbosity\": -1,\n }\n device = devices[0]\n if device == -1:\n # use cpu\n pass\n else:\n # use gpu\n print(f'using gpu device_id {device}...')\n params.update({'device': 'gpu', 'gpu_device_id': device})\n\n params['seed'] = seed\n\n early_stop = 20\n verbose_eval = 20\n\n d_train = lgb.Dataset(X_train, label=y_train, categorical_feature=cat_features)\n d_valid = lgb.Dataset(X_valid, label=y_valid, categorical_feature=cat_features)\n watchlist = [d_train, d_valid]\n\n print('training LGB:')\n model = lgb.train(params,\n train_set=d_train,\n num_boost_round=num_rounds,\n valid_sets=watchlist,\n verbose_eval=verbose_eval,\n early_stopping_rounds=early_stop)\n\n # predictions\n y_pred_valid = model.predict(X_valid, num_iteration=model.best_iteration)\n \n print('best_score', model.best_score)\n log = {'train/l2': model.best_score['training']['l2'],\n 'valid/l2': model.best_score['valid_1']['l2']}\n return model, y_pred_valid, log","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n# 3. Make \"study\" and let optimize!\n\nAfter you define `objective` function and write a code to use hyperparamers by `trial` module, we are ready to go!\n\nJust 2 lines of code do the all troublesome hyperparameter tuning for you. That's all!!"},{"metadata":{"trusted":true,"_kg_hide-output":true},"cell_type":"code","source":"study = optuna.create_study()\nstudy.optimize(objective, n_trials=10)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Summary of trial & study\n\n - **`trial` manages each execution of model training, evaluation, getting a score by specifying one trial of hyperparameter**\n - **`study` manages all history of `trial`. So that we can know best hyperparamers, and suggest next hyperparameters to search etc**\n\n\n## Sampler\n\n`optuna` provides many types of `Sampler` class to suggest next hyperparameters. (Ref: [sampler](https://optuna.readthedocs.io/en/stable/tutorial/sampler.html))\n\nDefault behavior is to use TPE algorithm, which is same with famous `hyperopt` library. This is kind of a bayesian optimization based sampling.\n\n\n## FAQ: How to define objective functions that have own arguments?\n\n`study.optimize` method receives `objective` method with `trial` args, how to pass other 2nd, 3rd argument?
\nWe can use class or lambda expression.
\nFor example, `study.optimize(lambda trial: objective(trial, arg0=1, arg1=2), n_trials=100)`
\n\nRefer [How to define objective functions that have own arguments?](https://optuna.readthedocs.io/en/latest/faq.html#how-to-define-objective-functions-that-have-own-arguments) for details."},{"metadata":{},"cell_type":"markdown","source":"\n# 4. [Advanced] Pruning unpromising trials for more faster search\n\nAnother important functionality is **\"pruning\"**. When human try hyperparameter tuning, we usually stop training earlier when the learning curve was much worse than the best known result.\n\n`optuna` provides pruning feature, and also provides integration modules for famous ML framework so that user can easily try pruning during hyperparameter optimization.
\nSupported modules are following\n\n - XGBoost: `optuna.integration.XGBoostPruningCallback`\n - LightGBM: `optuna.integration.LightGBMPruningCallback`\n - Chainer: `optuna.integration.ChainerPruningExtension`\n - Keras: `optuna.integration.KerasPruningCallback`\n - TensorFlow `optuna.integration.TensorFlowPruningHook`\n - tf.keras `optuna.integration.TFKerasPruningCallback`\n - MXNet `optuna.integration.MXNetPruningCallback`\n\nSee [pruning](https://optuna.readthedocs.io/en/stable/tutorial/pruning.html) section for details.\n\n"},{"metadata":{},"cell_type":"markdown","source":"I just added `LightGBMPruningCallback` for LightGBM training.
\nThe difference is only 2 line. Define `pruning_callback` and set it in `lgb.train` method.\n\nThe example below will check validation score each iteration for pruning."},{"metadata":{"trusted":true},"cell_type":"code","source":"# Referred https://github.com/pfnet/optuna/blob/master/examples/pruning/lightgbm_integration.py\n\ndef fit_lgbm_with_pruning(trial, train, val, devices=(-1,), seed=None, cat_features=None, num_rounds=1500):\n \"\"\"Train Light GBM model\"\"\"\n X_train, y_train = train\n X_valid, y_valid = val\n metric = 'l2'\n params = {\n 'num_leaves': trial.suggest_int('num_leaves', 2, 256),\n 'objective': 'regression',\n# 'max_depth': -1,\n 'learning_rate': 0.1,\n \"boosting\": \"gbdt\",\n 'lambda_l1': trial.suggest_loguniform('lambda_l1', 1e-8, 10.0),\n 'lambda_l2': trial.suggest_loguniform('lambda_l2', 1e-8, 10.0),\n \"bagging_freq\": 5,\n \"bagging_fraction\": trial.suggest_uniform('bagging_fraction', 0.1, 1.0),\n \"feature_fraction\": trial.suggest_uniform('feature_fraction', 0.4, 1.0),\n \"metric\": metric,\n \"verbosity\": -1,\n }\n device = devices[0]\n if device == -1:\n # use cpu\n pass\n else:\n # use gpu\n print(f'using gpu device_id {device}...')\n params.update({'device': 'gpu', 'gpu_device_id': device})\n\n params['seed'] = seed\n\n early_stop = 20\n verbose_eval = 20\n\n d_train = lgb.Dataset(X_train, label=y_train, categorical_feature=cat_features)\n d_valid = lgb.Dataset(X_valid, label=y_valid, categorical_feature=cat_features)\n watchlist = [d_train, d_valid]\n\n # Add a callback for pruning.\n pruning_callback = optuna.integration.LightGBMPruningCallback(trial, 'l2', valid_name='valid_1') \n print('training LGB:')\n model = lgb.train(params,\n train_set=d_train,\n num_boost_round=num_rounds,\n valid_sets=watchlist,\n verbose_eval=verbose_eval,\n early_stopping_rounds=early_stop,\n callbacks=[pruning_callback])\n\n # predictions\n y_pred_valid = model.predict(X_valid, num_iteration=model.best_iteration)\n \n print('best_score', model.best_score)\n log = {'train/l2': model.best_score['training']['l2'],\n 'valid/l2': model.best_score['valid_1']['l2']}\n return model, y_pred_valid, log","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"def objective_with_prune(trial: Trial, fast_check=True, target_meter=0):\n \"\"\"This method is completely same with previous `objective` method except calling `fit_lgbm_with_pruning`\"\"\"\n folds = 5\n seed = 666\n shuffle = False\n kf = KFold(n_splits=folds, shuffle=shuffle, random_state=seed)\n\n X_train, y_train = create_X_y(train_df, target_meter=target_meter)\n y_valid_pred_total = np.zeros(X_train.shape[0])\n gc.collect()\n print('target_meter', target_meter, X_train.shape)\n\n cat_features = [X_train.columns.get_loc(cat_col) for cat_col in category_cols]\n print('cat_features', cat_features)\n\n models0 = []\n valid_score = 0\n for train_idx, valid_idx in kf.split(X_train, y_train):\n train_data = X_train.iloc[train_idx,:], y_train[train_idx]\n valid_data = X_train.iloc[valid_idx,:], y_train[valid_idx]\n\n print('train', len(train_idx), 'valid', len(valid_idx))\n model, y_pred_valid, log = fit_lgbm_with_pruning(trial, train_data, valid_data, cat_features=category_cols,\n num_rounds=1000)\n y_valid_pred_total[valid_idx] = y_pred_valid\n models0.append(model)\n gc.collect()\n valid_score += log[\"valid/l2\"]\n if fast_check:\n break\n valid_score /= len(models0)\n return valid_score","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Then you can again create `study` object with **setting `pruner` object**, and let optimize!\n\n`pruner` class defines strategy how to prune in each intermediate step.
\n`MedianPruner` used in this example just check reported value each iteration, and if its score is higher than the median value of whole study history in this iteration, prunes."},{"metadata":{"trusted":true,"_kg_hide-output":true},"cell_type":"code","source":"study = optuna.create_study(pruner=optuna.pruners.MedianPruner(n_warmup_steps=5))\nstudy.optimize(objective_with_prune, n_trials=50)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now you can check this kind of message:\n\n> [I 2019-10-28 13:03:33,408] Setting status of trial#15 as TrialState.PRUNED. Trial was pruned at iteration 6.\n\nSome unpromised trial is pruned before totally taining the model, so that we can check more trial!"},{"metadata":{},"cell_type":"markdown","source":"\n# 5. Check study history to get best hyperparameters\n\nNow we've done hyperparameter optimization. So we want to know the best hyperparameters.
\nWhole `trial` history is managed in `study` object, we can get best trial by **`study.best_trial`**"},{"metadata":{"trusted":true},"cell_type":"code","source":"print('Best trial: score {}, params {}'.format(study.best_trial.value, study.best_trial.params))\n\n# It has alias, below is also same.\n# print('Best trial: score {}, params {}'.format(study.best_value, study.best_params))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Another tips: If you want to see all the history of trials, you can get `pandas.DataFrame` format by calling `study.trials_dataframe()`."},{"metadata":{"trusted":true},"cell_type":"code","source":"trials_df = study.trials_dataframe()\ntrials_df","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n# 6. Visualize study history to analayze the hyperparams-performance relationship\n\nSince we have history of trials, we can analyze how each hyperparams affects to the performance.
\nAll the visualization below can be done in just one line!!!\n\n(Acknowledgement: thank you @shotaro for let me know this feature)"},{"metadata":{},"cell_type":"markdown","source":"## Optimization history\n\nIt is a best score's history, blue dot is the score of this trial and orange line show the best score.
\nNote that blue dot is not in the all trial, because we turned on pruning thus many of the trials are stopped before getting final objective value."},{"metadata":{"trusted":true},"cell_type":"code","source":"optuna.visualization.plot_optimization_history(study)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Visualize pruning history\n\nYou can visualize how the pruning is executed!
\nEach color shows the loss curve of each trial."},{"metadata":{"trusted":true},"cell_type":"code","source":"optuna.visualization.plot_intermediate_values(study)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Visualize plot slice\n\nWe can see how the optuna's sampler searches hyperparameter space.
\nThe white dots are searched in the early stage, while blue dots are searched in the later stage.\n\nWe can see following behavior (This is for the debugging small train_df training, actual dataset behavior maybe different!)\n\n - `feature_fraction`: Low value (0.4~0.8) got very but score at the begging of trial, and thus high value (0.9~1.0) are extensively searched in the later stage.\n - `num_leaves` : Intermediate value around 32~64 are extensively searched in the later stage, which may be \"reasonable\"."},{"metadata":{"trusted":true},"cell_type":"code","source":"optuna.visualization.plot_slice(study)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Visualize plot contour\n\nWe can see 2 parametes pair plot with objective value as contour."},{"metadata":{"trusted":true},"cell_type":"code","source":"optuna.visualization.plot_contour(study)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Visualize parallel_coordinate\n\nWhen I check the best valued line, which is most blue color line starts from left-bottom, we can see following behavior:\n\n - higher high feature_fraction seems better\n - lower lambda_l1 seems better\n - lower lambda_l2 seems better\n - not too big num_leaves seems better\n \nfor debugging data. This is very small data, so it is reasonable that it should use all the feature and less regularization..."},{"metadata":{"trusted":true},"cell_type":"code","source":"optuna.visualization.plot_parallel_coordinate(study)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## More examples?\n\nSee [https://github.com/pfnet/optuna/tree/master/examples/visualization](https://github.com/pfnet/optuna/tree/master/examples/visualization) !!"},{"metadata":{},"cell_type":"markdown","source":"# Let's start actual hyperparameter optimization\n\nReharsal is done! Now let's proceed with the actual tuning.
\n`study.optimize` can be configured **by time limitation using `timeout`**, instead of number of trials `n_trials`. It is convinient for kaggle kernel."},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"if not debug:\n # Restore to original data size for actual computation\n train_df = train_df_original\n timeout = 60 * 60 * 2 # For 2 hours...\nelse:\n timeout = 60 * 1 # Debug mode, only for 1 mins\n\nprint('train_df.shape', train_df.shape)\nprint(f'timeout {timeout /60} min')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-output":true},"cell_type":"code","source":"study = optuna.create_study(pruner=optuna.pruners.SuccessiveHalvingPruner(min_resource=2, reduction_factor=4, min_early_stopping_rate=1))\nstudy.optimize(objective_with_prune, timeout=timeout)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(f'Executed {len(study.trials)} trials, best score {study.best_value} with best_params {study.best_params}')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"trials_df = study.trials_dataframe()\ntrials_df.to_csv('trials_history.csv')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"# optuna.visualization.plot_optimization_history(study)\nfig = optuna.visualization._get_optimization_history_plot(study)\npy.plot(fig, filename='optimization_history.html')\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# optuna.visualization.plot_intermediate_values(study)\nfig = optuna.visualization._get_intermediate_plot(study)\npy.plot(fig, filename='intermediate_values.html')\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# optuna.visualization.plot_slice(study)\nfig = optuna.visualization._get_slice_plot(study)\npy.plot(fig, filename='slice.html')\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# optuna.visualization.plot_contour(study)\nfig = optuna.visualization._get_contour_plot(study)\npy.plot(fig, filename='contour.html')\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# optuna.visualization.plot_parallel_coordinate(study)\nfig = optuna.visualization._get_parallel_coordinate_plot(study)\npy.plot(fig, filename='parallel_coordinate.html')\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Train model and make submission with best params\n\nI just tuned parameter for meter type 0 and only for 1st fold above.
\nBut I use this parameter for all training for simplicity."},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"print(\"Training model with best_params {}\".format(study.best_params))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"When running the training again with specific hyperparameters, we can use `FixedTrial` object to reuse `objective` function which we defined before.
\n`FixedTrial` object always returns fixed value, so we can run with `best_params`."},{"metadata":{"trusted":true},"cell_type":"code","source":"# For meter 0\nvalid_score, models0, y_pred_valid, y_train = objective(optuna.trial.FixedTrial(study.best_params), fast_check=False, target_meter=0, return_info=True)\n\nsns.distplot(y_pred_valid, label='pred')\nsns.distplot(y_train, label='ground truth')\nplt.legend()\nplt.show()\n\ndel y_pred_valid, y_train","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# For meter 1\nvalid_score, models1, y_pred_valid, y_train = objective(optuna.trial.FixedTrial(study.best_params), fast_check=False, target_meter=1, return_info=True)\n\nsns.distplot(y_pred_valid, label='pred')\nsns.distplot(y_train, label='ground truth')\nplt.legend()\nplt.show()\n\ndel y_pred_valid, y_train","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# For meter 2\nvalid_score, models2, y_pred_valid, y_train = objective(optuna.trial.FixedTrial(study.best_params), fast_check=False, target_meter=2, return_info=True)\n\nsns.distplot(y_pred_valid, label='pred')\nsns.distplot(y_train, label='ground truth')\nplt.legend()\nplt.show()\n\ndel y_pred_valid, y_train","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# For meter 3\nvalid_score, models3, y_pred_valid, y_train = objective(optuna.trial.FixedTrial(study.best_params), fast_check=False, target_meter=3, return_info=True)\n\nsns.distplot(y_pred_valid, label='pred')\nsns.distplot(y_train, label='ground truth')\nplt.legend()\nplt.show()\n\ndel y_pred_valid, y_train","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"try:\n del train_df\n del train_df_original\n del weather_train_df\nexcept:\n pass\ngc.collect()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Create test data"},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"print('loading...')\ntest_df = pd.read_feather(root/'test.feather')\nweather_test_df = pd.read_feather(root/'weather_test.feather')\n\nprint('preprocessing building...')\ntest_df['date'] = test_df['timestamp'].dt.date\npreprocess(test_df)\n\nprint('preprocessing weather...')\nweather_test_df = timestamp_align(weather_test_df)\nweather_test_df = weather_test_df.groupby('site_id').apply(lambda group: group.interpolate(limit_direction='both'))\nweather_test_df.groupby('site_id').apply(lambda group: group.isna().sum())\n\nadd_lag_feature(weather_test_df, window=3)\nadd_lag_feature(weather_test_df, window=72)\n\nprint('reduce mem usage...')\nweather_test_df = weather_test_df[['site_id', 'timestamp'] + weather_cols]\nreduce_mem_usage(test_df, use_float16=True)\nreduce_mem_usage(weather_test_df, use_float16=True)\n\ngc.collect()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"del df_2d\ndel site_ids_offsets\ngc.collect()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true,"_kg_hide-output":true},"cell_type":"code","source":"def create_X(test_df, target_meter):\n target_test_df = test_df[test_df['meter'] == target_meter]\n target_test_df = target_test_df.merge(building_meta_df, on='building_id', how='left', copy=False)\n target_test_df = pd.merge(target_test_df, building_stats_df, on=['building_id', 'meter'], how='left', copy=False)\n target_test_df = target_test_df.merge(weather_test_df, on=['site_id', 'timestamp'], how='left', copy=False)\n X_test = target_test_df[feature_cols + category_cols]\n return X_test\n\ndef pred(X_test, models, batch_size=1000000):\n iterations = (X_test.shape[0] + batch_size -1) // batch_size\n print('iterations', iterations)\n\n y_test_pred_total = np.zeros(X_test.shape[0])\n for i, model in enumerate(models):\n print(f'predicting {i}-th model')\n for k in tqdm(range(iterations)):\n y_pred_test = model.predict(X_test[k*batch_size:(k+1)*batch_size], num_iteration=model.best_iteration)\n y_test_pred_total[k*batch_size:(k+1)*batch_size] += y_pred_test\n\n y_test_pred_total /= len(models)\n return y_test_pred_total","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"%%time\nX_test = create_X(test_df, target_meter=0)\ngc.collect()\n\ny_test0 = pred(X_test, models0)\n\ndel X_test\ngc.collect()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"%%time\nX_test = create_X(test_df, target_meter=1)\ngc.collect()\ny_test1 = pred(X_test, models1)\n\ndel X_test\ngc.collect()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"%%time\nX_test = create_X(test_df, target_meter=2)\ngc.collect()\ny_test2 = pred(X_test, models2)\n\ndel X_test\ngc.collect()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"%%time\nX_test = create_X(test_df, target_meter=3)\ngc.collect()\ny_test3 = pred(X_test, models3)\n\ndel X_test\ngc.collect()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"sns.distplot(y_test0)\nplt.title('test prediction for meter type 0')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"sns.distplot(y_test1)\nplt.title('test prediction for meter type 1')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"sns.distplot(y_test2)\nplt.title('test prediction for meter type 2')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"sns.distplot(y_test3)\nplt.title('test prediction for meter type 3')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"sample_submission = pd.read_feather(os.path.join(root, 'sample_submission.feather'))\nreduce_mem_usage(sample_submission)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(np.sum(y_test0 < 0))\nprint(np.sum(y_test1 < 0))\nprint(np.sum(y_test2 < 0))\nprint(np.sum(y_test3 < 0))\n\ny_test0 = np.where(y_test0 < 0, 0, y_test0)\ny_test1 = np.where(y_test1 < 0, 0, y_test1)\ny_test2 = np.where(y_test2 < 0, 0, y_test2)\ny_test3 = np.where(y_test3 < 0, 0, y_test3)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"sample_submission.loc[test_df['meter'] == 0, 'meter_reading'] = np.expm1(y_test0)\nsample_submission.loc[test_df['meter'] == 1, 'meter_reading'] = np.expm1(y_test1)\nsample_submission.loc[test_df['meter'] == 2, 'meter_reading'] = np.expm1(y_test2)\nsample_submission.loc[test_df['meter'] == 3, 'meter_reading'] = np.expm1(y_test3)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sample_submission.to_csv('submission.csv', index=False, float_format='%.4f')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sample_submission.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"np.log1p(sample_submission['meter_reading']).hist()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"def plot_feature_importance(model):\n importance_df = pd.DataFrame(model.feature_importance(),\n index=feature_cols + category_cols,\n columns=['importance']).sort_values('importance')\n fig, ax = plt.subplots(figsize=(8, 8))\n importance_df.plot.barh(ax=ax)\n fig.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_feature_importance(models0[1])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_feature_importance(models1[1])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_feature_importance(models2[1])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_feature_importance(models3[1])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n# More to go\n\n## Other features...\n\nThat's all for this tutorial. However `optuna` further provides other useful functionality.\n\n - Store trial information in Relational Database (SQLite, PostgreSQL, MySQL) rather than on memory, to save hyperparameter search history.\n - Parallel distributed optimization: you can run several process at the same time to asyncronously search hyperparameters with many CPUs in parallel.\n\nSee Official document, github for details!\n\n - [web site](https://optuna.org/)\n - [github: pfnet/optuna](https://github.com/pfnet/optuna)\n - [document](https://optuna.readthedocs.io/en/latest/)\n\n**Recently LightGBMTuner is introduced (See [this PR#549](https://github.com/pfnet/optuna/pull/549)), maybe it is quite interesting to try!**\n\nIf this kernel helps you, please upvote to keep me motivated 😁
Thanks!"}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.6.6"},"widgets":{"application/vnd.jupyter.widget-state+json":{"state":{"006c75a4329843dd9703abb1b4575146":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"01b92038a045425cbf630a52d0b691b4":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"037ca52a68d1404081c8bf944b1bf0fb":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_e5b34e8871b84b83b3d14cc396387303","IPY_MODEL_4e46d3755e7b41de917dc46ee7495ec5"],"layout":"IPY_MODEL_43689b5df3ba4f9bb94f86c982f2e479"}},"0582b098e54042b384cbba227680aa13":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_97a8d389de8c42d08f290c79598955b4","placeholder":"​","style":"IPY_MODEL_80e78290f2d14bd9b0126cf92c6828ac","value":" 25/25 [05:02<00:00, 12.11s/it]"}},"05e6f87a412746ac88522959a153a2c0":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":"initial"}},"06c80f2ae46f48ef819174bc21f578ae":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":"initial"}},"088d23b9f7ce4dca9b42de9c86075404":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_7c4f4ac674de4a69b7fcf653558c0874","IPY_MODEL_934190aaa02d4fe782cb8fe0f780d988"],"layout":"IPY_MODEL_006c75a4329843dd9703abb1b4575146"}},"09040e6f6ea74793825747dcac04a1be":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"IntProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"IntProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"100%","description_tooltip":null,"layout":"IPY_MODEL_aa72cc8cec64446f96c77a6ea3459759","max":6,"min":0,"orientation":"horizontal","style":"IPY_MODEL_06c80f2ae46f48ef819174bc21f578ae","value":6}},"0a2ba8d2ae2145808f2a32719250759e":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":"initial"}},"0d2cb02a151841c1b0da91ebbd6bc9bb":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_454744019b644d3e9eb2fe5d8ae87812","IPY_MODEL_92fe259f89d4409d968a9e4491d6b33d"],"layout":"IPY_MODEL_aa503e3eaf274a0c8bc53394c52450dd"}},"0d665c83d0dc4d9e85d93657b9f5af9e":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"IntProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"IntProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"100%","description_tooltip":null,"layout":"IPY_MODEL_e75399ef40fa4a06a8ef0acd78b26ad2","max":9,"min":0,"orientation":"horizontal","style":"IPY_MODEL_cd9a9669ad684906982bb561e785d097","value":9}},"0da4ca25180c4d2f96e89b39eefbbef7":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"0fb6f69d1032426e89eca6fb5a77f40a":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"1259302fd16547c3a41dc1586a3c67f7":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"142ce684c78b4116bd378522d9c6b6c5":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_160d01f884a34630aa89f4cd1cc9264a","IPY_MODEL_3532dc5705b04ff3b2643daae39e5971"],"layout":"IPY_MODEL_99ae32f6da414b4ba66f41014c9cda22"}},"154b7346ce4c4a85a4823b0a91137409":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_5008d4e28fb846479eee7b6174f02a3c","placeholder":"​","style":"IPY_MODEL_fc9fab1f26754474bf05f68b00823e13","value":" 9/9 [03:57<00:00, 26.36s/it]"}},"15b30532424b4648b8d47b0ef839de25":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"15de1ef3e1fe47b2918fb3774c8a6635":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_36d2c2ab989046c99291be82699dfcb4","IPY_MODEL_958904ee17f348ba8501bf5a62340785"],"layout":"IPY_MODEL_5bf23ddbd3c94d72b53ee2caf0293995"}},"160d01f884a34630aa89f4cd1cc9264a":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"IntProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"IntProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"100%","description_tooltip":null,"layout":"IPY_MODEL_c016441a11d94840b2c80706a1547995","max":3,"min":0,"orientation":"horizontal","style":"IPY_MODEL_9bc6f268f27c4ac194e84afe2cae6cf9","value":3}},"1baab22d27e449fbb47cbebd681a0b3b":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"1d6038ca630c4f758903ae358b0d8982":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_72acc81e2e4e415395320a101522fd50","IPY_MODEL_376ae1e50edc4613be16ef85105a83dd"],"layout":"IPY_MODEL_777e392ac6f34b24b9c921c0f68d38bf"}},"1e6e4c862f3e46e38332d7decfaec432":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"IntProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"IntProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"100%","description_tooltip":null,"layout":"IPY_MODEL_cf256dc7e3064af3b5f0ef1281a1eb8d","max":6,"min":0,"orientation":"horizontal","style":"IPY_MODEL_05e6f87a412746ac88522959a153a2c0","value":6}},"204bfe2221064322977e9e147196f853":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":"initial"}},"2075bc360e694946a533eab57d43c3e7":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"233923af0c2f40018b473c831d3fb5ae":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_5ee0985483c1491792fee8e6745518d6","IPY_MODEL_7a53180cb2fd4409b33866f0d4e1154d"],"layout":"IPY_MODEL_9654e7a8d1494646b8b7b6e4c8f08807"}},"27b81e63c8524d87b3fffe177f48d95f":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"IntProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"IntProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"100%","description_tooltip":null,"layout":"IPY_MODEL_839b39cb385144bd9273db624e92f9e6","max":3,"min":0,"orientation":"horizontal","style":"IPY_MODEL_42ae286ce1a34c71aa4dd373a9b532d7","value":3}},"28bd098c75224fd993f358cbf02c9a3f":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"IntProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"IntProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"100%","description_tooltip":null,"layout":"IPY_MODEL_cff3feeae0904afbab40c1a3322d38b1","max":3,"min":0,"orientation":"horizontal","style":"IPY_MODEL_945a88282f4d4da5a62732c2f9871c56","value":3}},"2b366620cd3d40fda1217db4de43baa1":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"2c95fa01e43145eaaa57abffda9425e3":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"33cd383522644eb58d53b1fa28bda361":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_0d665c83d0dc4d9e85d93657b9f5af9e","IPY_MODEL_154b7346ce4c4a85a4823b0a91137409"],"layout":"IPY_MODEL_01b92038a045425cbf630a52d0b691b4"}},"3532dc5705b04ff3b2643daae39e5971":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_94bd46f6d22f4c2f871b687da996fff2","placeholder":"​","style":"IPY_MODEL_7445f4394880489db3ae63f453b93605","value":" 3/3 [00:10<00:00, 3.34s/it]"}},"36047d5b677342b98d6bbd2829cfcf14":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"36631529aa164c14a55a7c73478d6d20":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_587a7df246e747e7a95a0938c1dfa608","IPY_MODEL_0582b098e54042b384cbba227680aa13"],"layout":"IPY_MODEL_dd648cd10aec4dcebd60877a5b1bb711"}},"367de9f22f6e421c9458d92762549876":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"36d2c2ab989046c99291be82699dfcb4":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"IntProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"IntProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"100%","description_tooltip":null,"layout":"IPY_MODEL_15b30532424b4648b8d47b0ef839de25","max":6,"min":0,"orientation":"horizontal","style":"IPY_MODEL_39600f40527a495282b5842040e13dcf","value":6}},"376ae1e50edc4613be16ef85105a83dd":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_ca0a876c61544d08bd9cffeb3f57a244","placeholder":"​","style":"IPY_MODEL_558f79f616a249029d8494abd5389c5a","value":" 9/9 [02:21<00:00, 15.77s/it]"}},"378576e3c83a44dea48952ff584cd9d9":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_09040e6f6ea74793825747dcac04a1be","IPY_MODEL_949ef8aba0014b9c8b41c97c19268ba3"],"layout":"IPY_MODEL_b193bf1d162f49f0b793048d546a3594"}},"384d1f935ac54614ac9c47e85e0d8417":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"39600f40527a495282b5842040e13dcf":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":"initial"}},"39731261c7d54040b9453158462471cd":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_1e6e4c862f3e46e38332d7decfaec432","IPY_MODEL_a1ac7e4169e240cea3331db2ca949b5e"],"layout":"IPY_MODEL_8bfcd92a531441c69e4c5bd1234010aa"}},"41df3d03817147779df9c0f83b41a030":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"42ae286ce1a34c71aa4dd373a9b532d7":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":"initial"}},"43435674371048268ab97bddca4369b3":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"43689b5df3ba4f9bb94f86c982f2e479":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"43a315ccca45440eb3ac80f8ffe5babd":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_c8c718ac7cb0426aa5147e54597652c1","IPY_MODEL_64945e4ac0064c51aa492c957850c22a"],"layout":"IPY_MODEL_461655411e8d47cf8dd43b3961902f71"}},"454744019b644d3e9eb2fe5d8ae87812":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"IntProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"IntProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"100%","description_tooltip":null,"layout":"IPY_MODEL_697427b6af7d4162aad9bb2634360039","max":25,"min":0,"orientation":"horizontal","style":"IPY_MODEL_467e77c2b35b4d31836c2dcca256a400","value":25}},"461655411e8d47cf8dd43b3961902f71":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"467e77c2b35b4d31836c2dcca256a400":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":"initial"}},"4871dced8d1d4e6faea0ee7baad6697a":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"IntProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"IntProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"100%","description_tooltip":null,"layout":"IPY_MODEL_51d63cbc1c3b44b78c25749036313137","max":25,"min":0,"orientation":"horizontal","style":"IPY_MODEL_4af74597659748b3a66af3026fecb656","value":25}},"4abbda63ceb941b8878c1382af9d424c":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"IntProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"IntProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"100%","description_tooltip":null,"layout":"IPY_MODEL_b7c747522f5c44f799c39721abac7321","max":9,"min":0,"orientation":"horizontal","style":"IPY_MODEL_5bc30488b1d64871b222335e2a9c2c39","value":9}},"4af74597659748b3a66af3026fecb656":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":"initial"}},"4b82b320770146f7ac2dad8ff1817244":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"IntProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"IntProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"100%","description_tooltip":null,"layout":"IPY_MODEL_813090c11b7643e0881d7716cc114ce0","max":25,"min":0,"orientation":"horizontal","style":"IPY_MODEL_0a2ba8d2ae2145808f2a32719250759e","value":25}},"4e17af18f7f74481b2884a601ef1de03":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"4e46d3755e7b41de917dc46ee7495ec5":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_de6cd811d64349728faf5b561c615a86","placeholder":"​","style":"IPY_MODEL_aa38bc22c47d40a7b6dec0fac5fb254b","value":" 9/9 [01:18<00:00, 8.71s/it]"}},"4fa2304b4bdc42c1be2133c2a051161e":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":"initial"}},"5008d4e28fb846479eee7b6174f02a3c":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"50992c00c98042f8965a3abf60bfdbfa":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_27b81e63c8524d87b3fffe177f48d95f","IPY_MODEL_aa95ad82d4bd41b790f6f7edf799e9ba"],"layout":"IPY_MODEL_b59c080c02514337a891fab113a4c1b2"}},"50ecfdc00e7a4b5cb0617393365bdde3":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_7fb900c4fe03412f823fa8f0c9c22356","placeholder":"​","style":"IPY_MODEL_c4bb72313feb4226b4764dc1e4fbc19f","value":" 25/25 [01:57<00:00, 4.72s/it]"}},"51d63cbc1c3b44b78c25749036313137":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"5302ba3dbd1b404c82cda74db6e843f6":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"543c45ad7a344b08bede9d4539dcfd6e":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"547f5e4127894bfcbad45a4c456e5026":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":"initial"}},"558f79f616a249029d8494abd5389c5a":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"587a7df246e747e7a95a0938c1dfa608":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"IntProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"IntProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"100%","description_tooltip":null,"layout":"IPY_MODEL_ef632ee4108941b48dc7030e8c5d3dd6","max":25,"min":0,"orientation":"horizontal","style":"IPY_MODEL_8586fe286122407da46714b7f21ba519","value":25}},"58d29201a4da4d1b940c19b1a3be33af":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"5a1276c9f4324764a6b2b6fb81ca7816":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_b6db665e2b4c4f9281eb440730f7587b","placeholder":"​","style":"IPY_MODEL_80d144ae1e8748e2b4027014c9d45d80","value":" 25/25 [03:14<00:00, 7.79s/it]"}},"5bc30488b1d64871b222335e2a9c2c39":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":"initial"}},"5bf23ddbd3c94d72b53ee2caf0293995":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"5ce113da74164fe3b1e0a78b5248b75a":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":"initial"}},"5ee0985483c1491792fee8e6745518d6":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"IntProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"IntProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"100%","description_tooltip":null,"layout":"IPY_MODEL_f27eedf8140549788e34c76b636d734a","max":3,"min":0,"orientation":"horizontal","style":"IPY_MODEL_8efc439bd3794543841979d9baf2c906","value":3}},"64945e4ac0064c51aa492c957850c22a":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_36047d5b677342b98d6bbd2829cfcf14","placeholder":"​","style":"IPY_MODEL_8ea9f0964d454cf0925293f9f617c7cc","value":" 6/6 [02:14<00:00, 22.42s/it]"}},"676796d3e6a344baa932fc2bd4dca1bc":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":"initial"}},"697427b6af7d4162aad9bb2634360039":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"6db8934fb70441fca1a0ee08aaa3aec5":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"IntProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"IntProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"100%","description_tooltip":null,"layout":"IPY_MODEL_87e54e0e366b4235b49b98675d9fea1b","max":25,"min":0,"orientation":"horizontal","style":"IPY_MODEL_e0c62d3fff0945179d165251f0219245","value":25}},"6e374ae03592408e91779401d65c964b":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"6e7c763a37a749f5a3c04d070369ee4c":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"72acc81e2e4e415395320a101522fd50":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"IntProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"IntProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"100%","description_tooltip":null,"layout":"IPY_MODEL_b643e385d00e4ffea2bd8a9c63ad1fa6","max":9,"min":0,"orientation":"horizontal","style":"IPY_MODEL_5ce113da74164fe3b1e0a78b5248b75a","value":9}},"72fae5d65199495295c6ef8550a119a9":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"7445f4394880489db3ae63f453b93605":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"77218b34fbb24a85a5f7adc7739c9efc":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"777e392ac6f34b24b9c921c0f68d38bf":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"780e00dcf1ee4b68b5325f7732ca4ba4":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"78313317e2864057b0f45fff684d467b":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_28bd098c75224fd993f358cbf02c9a3f","IPY_MODEL_a1077a17bccc48d6a1888c1ececfde39"],"layout":"IPY_MODEL_41df3d03817147779df9c0f83b41a030"}},"7a53180cb2fd4409b33866f0d4e1154d":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_780e00dcf1ee4b68b5325f7732ca4ba4","placeholder":"​","style":"IPY_MODEL_e70d5901cf6d4a8db016e59b9eb51a43","value":" 3/3 [01:37<00:00, 32.64s/it]"}},"7c4f4ac674de4a69b7fcf653558c0874":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"IntProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"IntProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"100%","description_tooltip":null,"layout":"IPY_MODEL_ed190a01a3284b1bb4744f485205074d","max":9,"min":0,"orientation":"horizontal","style":"IPY_MODEL_676796d3e6a344baa932fc2bd4dca1bc","value":9}},"7e0e4d9eaf0244c887c941727df436b9":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"7e904af7812d431888014df0f688f953":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_d2ce7bb5402647f99d01a96b0ab7de46","IPY_MODEL_8859e324ab3e4079a53356f7aab5881d"],"layout":"IPY_MODEL_9dc8f9f5e13b444abac495b94fc50389"}},"7fb900c4fe03412f823fa8f0c9c22356":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"80d144ae1e8748e2b4027014c9d45d80":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"80e78290f2d14bd9b0126cf92c6828ac":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"813090c11b7643e0881d7716cc114ce0":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"839b39cb385144bd9273db624e92f9e6":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"8586fe286122407da46714b7f21ba519":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":"initial"}},"87e54e0e366b4235b49b98675d9fea1b":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"8859e324ab3e4079a53356f7aab5881d":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_7e0e4d9eaf0244c887c941727df436b9","placeholder":"​","style":"IPY_MODEL_6e374ae03592408e91779401d65c964b","value":" 3/3 [00:26<00:00, 8.75s/it]"}},"8bfcd92a531441c69e4c5bd1234010aa":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"8c70855847b34aa0a0ee4899ad29dccb":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"8ea9f0964d454cf0925293f9f617c7cc":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"8efc439bd3794543841979d9baf2c906":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":"initial"}},"92fe259f89d4409d968a9e4491d6b33d":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_2b366620cd3d40fda1217db4de43baa1","placeholder":"​","style":"IPY_MODEL_543c45ad7a344b08bede9d4539dcfd6e","value":" 25/25 [04:02<00:00, 9.72s/it]"}},"934190aaa02d4fe782cb8fe0f780d988":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_9caf6860f8eb4f7c9f75d73cf0ca04e4","placeholder":"​","style":"IPY_MODEL_be02ee282a964382ae095807d6b5318b","value":" 9/9 [00:43<00:00, 4.79s/it]"}},"945a88282f4d4da5a62732c2f9871c56":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":"initial"}},"949ef8aba0014b9c8b41c97c19268ba3":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_0da4ca25180c4d2f96e89b39eefbbef7","placeholder":"​","style":"IPY_MODEL_c167e9f19bec4912931dcf24eafa977f","value":" 6/6 [00:16<00:00, 2.69s/it]"}},"94bd46f6d22f4c2f871b687da996fff2":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"958904ee17f348ba8501bf5a62340785":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_dd53be7f27b14a9795fb7bac29d3f541","placeholder":"​","style":"IPY_MODEL_384d1f935ac54614ac9c47e85e0d8417","value":" 6/6 [00:42<00:00, 7.13s/it]"}},"9654e7a8d1494646b8b7b6e4c8f08807":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"97a8d389de8c42d08f290c79598955b4":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"97dc362d93ee441e9d854db8b802aa18":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"99ae32f6da414b4ba66f41014c9cda22":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"9bc6f268f27c4ac194e84afe2cae6cf9":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":"initial"}},"9caf6860f8eb4f7c9f75d73cf0ca04e4":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"9dc8f9f5e13b444abac495b94fc50389":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"a1077a17bccc48d6a1888c1ececfde39":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_72fae5d65199495295c6ef8550a119a9","placeholder":"​","style":"IPY_MODEL_1baab22d27e449fbb47cbebd681a0b3b","value":" 3/3 [00:23<00:00, 7.69s/it]"}},"a1ac7e4169e240cea3331db2ca949b5e":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_6e7c763a37a749f5a3c04d070369ee4c","placeholder":"​","style":"IPY_MODEL_43435674371048268ab97bddca4369b3","value":" 6/6 [00:37<00:00, 6.22s/it]"}},"a343f87424124063a87ede3c4e110ff7":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"a6fd109fa30b4b15b365611bdbda200f":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"aa38bc22c47d40a7b6dec0fac5fb254b":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"aa503e3eaf274a0c8bc53394c52450dd":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"aa72cc8cec64446f96c77a6ea3459759":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"aa95ad82d4bd41b790f6f7edf799e9ba":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_ef64e21bbac643b9ae6f8d62a6bfae33","placeholder":"​","style":"IPY_MODEL_b394d9fdb76243768d32290280ce1e04","value":" 3/3 [01:10<00:00, 23.57s/it]"}},"ac3284da0a9948e9a98b28cb9b3ec7d8":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_4abbda63ceb941b8878c1382af9d424c","IPY_MODEL_f4bf333da2c148189029917b2316aea4"],"layout":"IPY_MODEL_0fb6f69d1032426e89eca6fb5a77f40a"}},"ac5748f982cc4bc0a03fd550d3506622":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"IntProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"IntProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"100%","description_tooltip":null,"layout":"IPY_MODEL_c51182fba5ad4cf1a378106c16495b9e","max":6,"min":0,"orientation":"horizontal","style":"IPY_MODEL_547f5e4127894bfcbad45a4c456e5026","value":6}},"b193bf1d162f49f0b793048d546a3594":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"b394d9fdb76243768d32290280ce1e04":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"b59c080c02514337a891fab113a4c1b2":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"b643e385d00e4ffea2bd8a9c63ad1fa6":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"b67eb922bc594c5ba86bef93b1fda5b3":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_5302ba3dbd1b404c82cda74db6e843f6","placeholder":"​","style":"IPY_MODEL_8c70855847b34aa0a0ee4899ad29dccb","value":" 25/25 [00:36<00:00, 1.46s/it]"}},"b6db665e2b4c4f9281eb440730f7587b":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"b7c747522f5c44f799c39721abac7321":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"be02ee282a964382ae095807d6b5318b":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"becb9606ecb245bb831c436b053a2cdf":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_eb098acdf1aa44de8085555b88aaf505","placeholder":"​","style":"IPY_MODEL_a6fd109fa30b4b15b365611bdbda200f","value":" 6/6 [02:32<00:00, 25.48s/it]"}},"c016441a11d94840b2c80706a1547995":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c167e9f19bec4912931dcf24eafa977f":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"c3cc3a78c687440ba6b9a1f6c53d2b15":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_4b82b320770146f7ac2dad8ff1817244","IPY_MODEL_5a1276c9f4324764a6b2b6fb81ca7816"],"layout":"IPY_MODEL_a343f87424124063a87ede3c4e110ff7"}},"c4bb72313feb4226b4764dc1e4fbc19f":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"c51182fba5ad4cf1a378106c16495b9e":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c8c718ac7cb0426aa5147e54597652c1":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"IntProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"IntProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"100%","description_tooltip":null,"layout":"IPY_MODEL_4e17af18f7f74481b2884a601ef1de03","max":6,"min":0,"orientation":"horizontal","style":"IPY_MODEL_fa2daa326b9a4d799f27e28aa70d3a69","value":6}},"ca0a876c61544d08bd9cffeb3f57a244":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"cd9a9669ad684906982bb561e785d097":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":"initial"}},"cf256dc7e3064af3b5f0ef1281a1eb8d":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"cff3feeae0904afbab40c1a3322d38b1":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"d2ce7bb5402647f99d01a96b0ab7de46":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"IntProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"IntProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"100%","description_tooltip":null,"layout":"IPY_MODEL_58d29201a4da4d1b940c19b1a3be33af","max":3,"min":0,"orientation":"horizontal","style":"IPY_MODEL_204bfe2221064322977e9e147196f853","value":3}},"d536dd03222e49eda65f88d45b264d6e":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_ac5748f982cc4bc0a03fd550d3506622","IPY_MODEL_becb9606ecb245bb831c436b053a2cdf"],"layout":"IPY_MODEL_2075bc360e694946a533eab57d43c3e7"}},"dd53be7f27b14a9795fb7bac29d3f541":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"dd648cd10aec4dcebd60877a5b1bb711":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"de6cd811d64349728faf5b561c615a86":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"e0c62d3fff0945179d165251f0219245":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":"initial"}},"e0f5f695546e4b4db8a01ab3f9b18426":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_6db8934fb70441fca1a0ee08aaa3aec5","IPY_MODEL_b67eb922bc594c5ba86bef93b1fda5b3"],"layout":"IPY_MODEL_97dc362d93ee441e9d854db8b802aa18"}},"e1b4b84fbfe242208296ed3c0f27df24":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_4871dced8d1d4e6faea0ee7baad6697a","IPY_MODEL_50ecfdc00e7a4b5cb0617393365bdde3"],"layout":"IPY_MODEL_1259302fd16547c3a41dc1586a3c67f7"}},"e5b34e8871b84b83b3d14cc396387303":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"IntProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"IntProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"100%","description_tooltip":null,"layout":"IPY_MODEL_2c95fa01e43145eaaa57abffda9425e3","max":9,"min":0,"orientation":"horizontal","style":"IPY_MODEL_4fa2304b4bdc42c1be2133c2a051161e","value":9}},"e70d5901cf6d4a8db016e59b9eb51a43":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"e75399ef40fa4a06a8ef0acd78b26ad2":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"eb098acdf1aa44de8085555b88aaf505":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"ed190a01a3284b1bb4744f485205074d":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"ef632ee4108941b48dc7030e8c5d3dd6":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"ef64e21bbac643b9ae6f8d62a6bfae33":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"f27eedf8140549788e34c76b636d734a":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"f4bf333da2c148189029917b2316aea4":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_367de9f22f6e421c9458d92762549876","placeholder":"​","style":"IPY_MODEL_77218b34fbb24a85a5f7adc7739c9efc","value":" 9/9 [04:48<00:00, 32.09s/it]"}},"fa2daa326b9a4d799f27e28aa70d3a69":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":"initial"}},"fc9fab1f26754474bf05f68b00823e13":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}}},"version_major":2,"version_minor":0}}},"nbformat":4,"nbformat_minor":1}