{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"\n\n\n\n","metadata":{}},{"cell_type":"markdown","source":"# <center>SEE-SLMHM-SPEM Dataset Test Notebook</center>\n\nThis notebook is an annex to the main analytic notebook: https://www.kaggle.com/code/dmitriygakh/kaggle-ai-report-maturity-at-a-glance\n***\n\n\nThe purpose of current notebook is to test SEE-SLMHM-SPEM Dataset. The main notebook contains all necessary documentation and practical examples of evaluation of scholarly papers. This notebook will not be sent to the competition for ecaluation.\n***\n\n\nYou can use this notebook to evaluate your texts [here](#5) (after running preparatory cells).\n***\n\n<a id='toc'></a>\n# Table of contents\n<div style='border-width: 2px;\n              border-bottom-width:4px;\n              border-bottom-color:#00FF00;\n              border-bottom-style: solid;'></div>\n\n\n- [1 | EDA](#1)\n- [2 | AI Model](#2)\n- [3 | Tests](#3)\n- [4 | Drawbacks](#4)\n- [5 | Free Text](#5)\n\n\n<a id='1'></a>\n# 1 | EDA\n<div style='border-width: 2px;\n              border-bottom-width:4px;\n              border-bottom-color:#00FF00;\n              border-bottom-style: solid;'></div>","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2023-07-02T14:28:41.776219Z","iopub.execute_input":"2023-07-02T14:28:41.776685Z","iopub.status.idle":"2023-07-02T14:28:41.809550Z","shell.execute_reply.started":"2023-07-02T14:28:41.776625Z","shell.execute_reply":"2023-07-02T14:28:41.808449Z"}}},{"cell_type":"code","source":"# Import necessary libraries\nimport pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport os\nfrom tqdm import tqdm\nimport random\n\nfrom keras.preprocessing.text import Tokenizer\nfrom keras.utils import pad_sequences\nimport tensorflow as tf\nfrom tensorflow.keras import Model\nfrom tensorflow.keras.layers import Input, Embedding, Bidirectional, LSTM, Conv1D, GlobalMaxPool1D, BatchNormalization, Dense, Dropout, SpatialDropout1D\nfrom tensorflow.keras.optimizers import RMSprop\nfrom tensorflow.keras.optimizers import Adam\nfrom keras.callbacks import EarlyStopping\nfrom tensorflow.keras.callbacks import ReduceLROnPlateau\n\nfrom nltk.corpus import stopwords\n\n# We will use English stopwords\nen_stopwords = set(stopwords.words('english'))\n\n# Set random seed to have stable results for runnings\nrseed = 55\nrandom.seed(rseed)\nnp.random.seed(rseed)\ntf.random.set_seed(rseed)\nos.environ['PYTHONHASHSEED'] = str(rseed)","metadata":{"execution":{"iopub.status.busy":"2023-07-15T03:17:11.915132Z","iopub.execute_input":"2023-07-15T03:17:11.916576Z","iopub.status.idle":"2023-07-15T03:17:11.929576Z","shell.execute_reply.started":"2023-07-15T03:17:11.916497Z","shell.execute_reply":"2023-07-15T03:17:11.928266Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Levels of SLMHM\nllabels = ['0','1','2','3','4','5','6','7','8','9','10','11','12','13','14','15','16']\nllabels_str = [\n    '0.0. Chaos',\n    '1.1. Desire','1.2. Intention','1.3. Feasibility','1.4. Action',\n    '2.1. Evaluation','2.2. Improvement','2.3. Alternatives','2.4. Innovations',\n    '3.1. Reward','3.2. Optimization','3.3. Synergy','3.4. Extraordinaries',\n    '4.1. Information','4.2. Streamlining','4.3. Expansion','4.4. Totality'\n]\n\n# Levels of SPEM\nllabels_spem = ['0','1','2','3','4']\nllabels_spem_str = ['IFP','PSP','ASP','ISP','CSP']\n# IFP - Initial Formation Pattern (chaos)\n# PSP - Primary Society Pattern (structure, discipline)\n# ASP - Adaptive Society Pattern (benefits, logic)\n# ISP - Information Society Pattern (connectivity, feelings, emotions)\n# CSP - Creative Society Pattern (creativity)\n\n# Loading the keyphrase classification data\nkpdf = pd.read_csv('../input/see-slmhm-spem/see_key_phrases.csv', delimiter = '\\t')\n\nkpdf.head()","metadata":{"execution":{"iopub.status.busy":"2023-07-15T03:12:06.281134Z","iopub.execute_input":"2023-07-15T03:12:06.281935Z","iopub.status.idle":"2023-07-15T03:12:06.351645Z","shell.execute_reply.started":"2023-07-15T03:12:06.281900Z","shell.execute_reply":"2023-07-15T03:12:06.350418Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[↑↑↑ Table of Contents ↑↑↑](#toc)","metadata":{}},{"cell_type":"code","source":"# Feature engineering.\n# Encode training dataset into 2-columns Class/Keywords structure.\n\n# Code Class values are:\n# ndndnd[X] - not determined, SLMHM level X\n# ndndsc[X] - society, SLMHM level X\n# ndndec[X] - economy, SLMHM level X\n# ndndev[X] - environment, SLMHM level X\n# ndscec[X] - society-economy, SLMHM level X\n# ndscev[X] - society-environment, SLMHM level X\n# ndecev[X] - economy-environment, SLMHM level X\n# scecev[X] - society-economy-environment, SLMHM level X\n\nkpdf.loc[kpdf['SLMHMlvsND'] >= 0, 'Class'] = 'ndndnd' + kpdf['SLMHMlvsND'].apply(lambda x: f\"{x:.0f}\")\nkpdf.loc[kpdf['SLMHMlvsSc'] >= 0, 'Class'] = 'ndndsc' + kpdf['SLMHMlvsSc'].apply(lambda x: f\"{x:.0f}\")\nkpdf.loc[kpdf['SLMHMlvsEc'] >= 0, 'Class'] = 'ndndec' + kpdf['SLMHMlvsEc'].apply(lambda x: f\"{x:.0f}\")\nkpdf.loc[kpdf['SLMHMlvsEv'] >= 0, 'Class'] = 'ndndev' + kpdf['SLMHMlvsEv'].apply(lambda x: f\"{x:.0f}\")\nkpdf.loc[kpdf['SLMHMlvsScEc'] >= 0, 'Class'] = 'ndscec' + kpdf['SLMHMlvsScEc'].apply(lambda x: f\"{x:.0f}\")\nkpdf.loc[kpdf['SLMHMlvsScEv'] >= 0, 'Class'] = 'ndscev' + kpdf['SLMHMlvsScEv'].apply(lambda x: f\"{x:.0f}\")\nkpdf.loc[kpdf['SLMHMlvsEcEv'] >= 0, 'Class'] = 'ndecev' + kpdf['SLMHMlvsEcEv'].apply(lambda x: f\"{x:.0f}\")\nkpdf.loc[kpdf['SLMHMlvsScEcEv'] >= 0, 'Class'] = 'scecev' + kpdf['SLMHMlvsScEcEv'].apply(lambda x: f\"{x:.0f}\")\n\n# Simplify the DataFrame. GC could free old unnecessary data structures.\nkpdf = kpdf[['Class','Keywords']].copy()\n\n# Lets see the training dataset\nkpdf","metadata":{"execution":{"iopub.status.busy":"2023-07-15T03:12:06.359475Z","iopub.execute_input":"2023-07-15T03:12:06.360269Z","iopub.status.idle":"2023-07-15T03:12:06.409039Z","shell.execute_reply.started":"2023-07-15T03:12:06.360228Z","shell.execute_reply":"2023-07-15T03:12:06.407784Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[↑↑↑ Table of Contents ↑↑↑](#toc)","metadata":{}},{"cell_type":"code","source":"# Training dataset contains 136 unique categories.\n# It can be later dicomposed into 17 levels * 8 categories\npd.DataFrame(kpdf.Class.unique())","metadata":{"execution":{"iopub.status.busy":"2023-07-15T03:12:06.410962Z","iopub.execute_input":"2023-07-15T03:12:06.411734Z","iopub.status.idle":"2023-07-15T03:12:06.428923Z","shell.execute_reply.started":"2023-07-15T03:12:06.411689Z","shell.execute_reply":"2023-07-15T03:12:06.426434Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[↑↑↑ Table of Contents ↑↑↑](#toc)","metadata":{}},{"cell_type":"code","source":"# Distribution of values\n# !!! ToDo: Training set should be complimented to be balanced !!!\n# Such disbalanced Training Dataset cannot be used to assess papers according to SEE\n# But as one can see below Training Dataset can be used to asses general maturity level\n# There is firm confidence that there are no SEE-matured papers in the dataset\n# So, SEE could be considered as a future extension\n\nvc = kpdf.Class.value_counts()\n\nplt.figure(figsize = (12,4) )\nplt.bar(vc.index, vc.values)\nplt.title(\"Training Data Distribution\")\nplt.xticks(rotation = 90) # Rotates X-Axis Ticks by 45-degrees\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-07-15T03:12:06.430361Z","iopub.execute_input":"2023-07-15T03:12:06.430801Z","iopub.status.idle":"2023-07-15T03:12:06.915909Z","shell.execute_reply.started":"2023-07-15T03:12:06.430756Z","shell.execute_reply":"2023-07-15T03:12:06.914742Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[↑↑↑ Table of Contents ↑↑↑](#toc)","metadata":{}},{"cell_type":"code","source":"# Ideally it should include all 136 categories taking into account that \n# a text can relate to more than one category.\n# But lets use only 16 + 1 development levels\n\nkpdf['ClassD'] = kpdf['Class'].apply(lambda x: x[6:])\n\n# it is better to transform column ClassD to numeric later\n# kpdf['ClassD'] = pd.to_numeric(kpdf['ClassD'])\n\nkpdf","metadata":{"execution":{"iopub.status.busy":"2023-07-15T03:12:06.917750Z","iopub.execute_input":"2023-07-15T03:12:06.918952Z","iopub.status.idle":"2023-07-15T03:12:06.939673Z","shell.execute_reply.started":"2023-07-15T03:12:06.918899Z","shell.execute_reply":"2023-07-15T03:12:06.938434Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[↑↑↑ Table of Contents ↑↑↑](#toc)","metadata":{}},{"cell_type":"code","source":"# Distribution shows that now the training dataset is balanced better\n# Notes: Disbalance is smaller for this training data version.\n\nvc = kpdf.ClassD.value_counts()\n\nplt.figure(figsize = (12,4))\nplt.bar(vc.index, vc.values)\nplt.title(\"Training Data Distribution\")","metadata":{"execution":{"iopub.status.busy":"2023-07-15T03:12:06.941086Z","iopub.execute_input":"2023-07-15T03:12:06.941729Z","iopub.status.idle":"2023-07-15T03:12:07.258304Z","shell.execute_reply.started":"2023-07-15T03:12:06.941624Z","shell.execute_reply":"2023-07-15T03:12:07.256785Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Clears the text removing different special symbols/formatting and stopwords\n# Taken from: https://www.kaggle.com/code/christofhenkel/how-to-preprocessing-when-using-embeddings\n# Returns list of words\ndef clean_text_words(x):\n    stopwords_count = 0\n    x = str(x)\n    for punct in \"/-'\":\n        x = x.replace(punct, ' ')\n    for punct in '&':\n        x = x.replace(punct, f' {punct} ')\n    for punct in '?!.,\"#$%\\'()*+-/:;<=>@[\\\\]^_`{|}~' + '“”’':\n        x = x.replace(punct, '')\n        \n    x = x.replace('\\n', ' ').strip().lower()\n    \n    words = x.split()\n    words2 = []\n    \n    for wrd in words: \n        if wrd not in en_stopwords:\n            words2.append(wrd)\n        else:\n            stopwords_count += 1\n            \n    return words2, stopwords_count\n\n# Clears the text removing different special symbols/formatting and stopwords\n# Returns text\ndef clean_text(x):\n    words, stopwords_count = clean_text_words(x)\n    if(len(words) == 0):\n        print('Attention! Stop word(s) only in training dataset:', x)\n        return x\n    else:\n        phrase = ''\n        for wrd in words:\n            if(phrase != ''):\n                phrase += ' '\n            phrase += wrd\n\n        return phrase, stopwords_count","metadata":{"execution":{"iopub.status.busy":"2023-07-15T03:12:07.260500Z","iopub.execute_input":"2023-07-15T03:12:07.260938Z","iopub.status.idle":"2023-07-15T03:12:07.273556Z","shell.execute_reply.started":"2023-07-15T03:12:07.260898Z","shell.execute_reply":"2023-07-15T03:12:07.272193Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[↑↑↑ Table of Contents ↑↑↑](#toc)","metadata":{}},{"cell_type":"code","source":"# Create train_x data\n# Clear train_x data by removing stop words\n\ntrain_x = kpdf['Keywords']\n\nstopwords_cnt = 0\n\nfor i in range(len(train_x)):\n    ph_w_stop_words = train_x[i]\n    train_x[i], swrds_cnt = clean_text(train_x[i])\n    \n    #if(swrds_cnt > 0):\n    #    print('Stopwords found:', ph_w_stop_words, 'Record(',i,')')\n        \n    stopwords_cnt += swrds_cnt\n\nprint('Stopwords removed:', stopwords_cnt)\ntrain_x","metadata":{"execution":{"iopub.status.busy":"2023-07-15T03:12:07.277964Z","iopub.execute_input":"2023-07-15T03:12:07.278295Z","iopub.status.idle":"2023-07-15T03:12:07.567910Z","shell.execute_reply.started":"2023-07-15T03:12:07.278267Z","shell.execute_reply":"2023-07-15T03:12:07.566661Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Create 16 + 1 columns accordingly\ntrain_y = pd.get_dummies(kpdf.ClassD) \ntrain_y = train_y[llabels]\n\ntrain_y","metadata":{"execution":{"iopub.status.busy":"2023-07-15T03:12:07.570005Z","iopub.execute_input":"2023-07-15T03:12:07.570439Z","iopub.status.idle":"2023-07-15T03:12:07.594481Z","shell.execute_reply.started":"2023-07-15T03:12:07.570400Z","shell.execute_reply":"2023-07-15T03:12:07.593308Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[↑↑↑ Table of Contents ↑↑↑](#toc)","metadata":{}},{"cell_type":"markdown","source":"2.2 Tokenizing and Word Emdedding¶","metadata":{}},{"cell_type":"code","source":"# Need calculate max_chunk_size because it affect performance\nmax_chunk_size = 0\nwrdsmax = None\n\nfor phrase in train_x:\n    sc_size = len(phrase.split())\n    if(max_chunk_size < sc_size):\n        max_chunk_size = sc_size\n        wrdsmax = phrase\n        \nprint(max_chunk_size)\nwrdsmax","metadata":{"execution":{"iopub.status.busy":"2023-07-15T03:12:07.596497Z","iopub.execute_input":"2023-07-15T03:12:07.596915Z","iopub.status.idle":"2023-07-15T03:12:07.608637Z","shell.execute_reply.started":"2023-07-15T03:12:07.596879Z","shell.execute_reply":"2023-07-15T03:12:07.607276Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Tokenize train_x texts\ntokenizer = Tokenizer()\ntokenizer.fit_on_texts(train_x)\n\nword_index = tokenizer.word_index\nvlen = len(word_index) + 1\n\nvlen","metadata":{"execution":{"iopub.status.busy":"2023-07-15T03:12:07.610645Z","iopub.execute_input":"2023-07-15T03:12:07.611061Z","iopub.status.idle":"2023-07-15T03:12:07.635968Z","shell.execute_reply.started":"2023-07-15T03:12:07.611024Z","shell.execute_reply":"2023-07-15T03:12:07.634662Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[↑↑↑ Table of Contents ↑↑↑](#toc)","metadata":{}},{"cell_type":"code","source":"# Load GloVe vectors\n\nemb_dict = {}\nf = open('/kaggle/input/glove840b300dtxt/glove.840B.300d.txt','r',encoding='utf-8')\nfor line in tqdm(f):\n    values = line.split(' ')\n    word = values[0]\n    coefs = np.asarray([float(val) for val in values[1:]])\n    emb_dict[word] = coefs\nf.close()\n\nlen(emb_dict)","metadata":{"execution":{"iopub.status.busy":"2023-07-15T03:17:23.256302Z","iopub.execute_input":"2023-07-15T03:17:23.256773Z","iopub.status.idle":"2023-07-15T03:21:55.926405Z","shell.execute_reply.started":"2023-07-15T03:17:23.256741Z","shell.execute_reply":"2023-07-15T03:21:55.924934Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Create embedding matrix\nEMB_DIM = 300\n\nemb_mtrx = np.zeros((vlen, EMB_DIM))\nfor word, i in word_index.items():\n    emb_v = emb_dict.get(word)\n    if emb_v is not None:\n        emb_mtrx[i] = emb_v\n    \nemb_mtrx.shape","metadata":{"execution":{"iopub.status.busy":"2023-07-15T03:22:38.655512Z","iopub.execute_input":"2023-07-15T03:22:38.655979Z","iopub.status.idle":"2023-07-15T03:22:38.668059Z","shell.execute_reply.started":"2023-07-15T03:22:38.655946Z","shell.execute_reply":"2023-07-15T03:22:38.667181Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[↑↑↑ Table of Contents ↑↑↑](#toc)","metadata":{}},{"cell_type":"markdown","source":"<a id='2'></a>\n# 2 | AI Model\n<div style='border-width: 2px;\n              border-bottom-width:4px;\n              border-bottom-color:#00FF00;\n              border-bottom-style: solid;'></div>","metadata":{}},{"cell_type":"code","source":"# \"Chunkinize\" train textes\ntrain_x_seq = pad_sequences(tokenizer.texts_to_sequences(train_x), maxlen = max_chunk_size)","metadata":{"execution":{"iopub.status.busy":"2023-07-15T03:23:13.988848Z","iopub.execute_input":"2023-07-15T03:23:13.989329Z","iopub.status.idle":"2023-07-15T03:23:14.011092Z","shell.execute_reply.started":"2023-07-15T03:23:13.989298Z","shell.execute_reply":"2023-07-15T03:23:14.009827Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Create Artificial Neural Network model\n# Model (modified) from https://www.kaggle.com/code/andreshg/nlp-glove-bert-tf-idf-lstm-explained\n\nembedding_layer = Embedding(\n    input_dim = emb_mtrx.shape[0], \n    output_dim = emb_mtrx.shape[1], \n    weights = [emb_mtrx], \n    input_length = max_chunk_size,\n    trainable = False\n)\n\nsequence_input = Input(shape = (max_chunk_size), dtype = 'int32')\nembedding_sequences = embedding_layer(sequence_input)\nlr = SpatialDropout1D(0.2)(embedding_sequences)\nlr = Conv1D(64, 5, activation = 'relu')(lr)\nlr = Bidirectional(LSTM(32, dropout=0.2, recurrent_dropout = 0.2))(lr)\nlr = Dense(512, activation = 'relu')(lr)\nlr = Dropout(0.5)(lr)\nlr = Dense(512, activation = 'relu')(lr)\nlr = Dropout(0.5)(lr)\noutputs = Dense(17, activation = 'sigmoid')(lr)\nmodel = Model(sequence_input, outputs)\n\nmodel.compile(\n    optimizer = Adam(learning_rate = 0.001),\n    loss = 'binary_crossentropy',\n    metrics = ['accuracy']\n)\n\nmodel.summary()","metadata":{"execution":{"iopub.status.busy":"2023-07-15T03:23:30.523402Z","iopub.execute_input":"2023-07-15T03:23:30.523833Z","iopub.status.idle":"2023-07-15T03:23:31.296814Z","shell.execute_reply.started":"2023-07-15T03:23:30.523802Z","shell.execute_reply":"2023-07-15T03:23:31.295471Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[↑↑↑ Table of Contents ↑↑↑](#toc)","metadata":{}},{"cell_type":"code","source":"# Train the model\n# Ovefitting is Ok for this model (prototype)\n\nRLROnPl = ReduceLROnPlateau(\n    factor = 0.1,\n    min_lr = 0.01,\n    monitor = 'val_loss',\n    verbose = 1\n)\n\nhist = model.fit(\n    train_x_seq,\n    train_y,\n    batch_size = 1024,\n    validation_data = (train_x_seq, train_y), # Ovefitting is Ok for this model (prototype)\n    epochs = 500,\n    callbacks = [RLROnPl],\n    verbose = 0\n)\n\n# Overfitting is Ok for this model (prototype)\nhistory_df = pd.DataFrame(hist.history)\nhistory_df.loc[:, ['loss', 'val_loss']].plot();\nprint(\"Minimum validation loss: {}\".format(history_df['val_loss'].min()))","metadata":{"execution":{"iopub.status.busy":"2023-07-15T03:23:40.516510Z","iopub.execute_input":"2023-07-15T03:23:40.516941Z","iopub.status.idle":"2023-07-15T03:25:10.623239Z","shell.execute_reply.started":"2023-07-15T03:23:40.516910Z","shell.execute_reply":"2023-07-15T03:25:10.621816Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[↑↑↑ Table of Contents ↑↑↑](#toc)","metadata":{}},{"cell_type":"markdown","source":"## Helpful Functions","metadata":{}},{"cell_type":"code","source":"# Returns chunks of text with no longer than chunk_size words\ndef prep_str_chunks(words, chunk_size):\n    res = []\n    i = 0\n    while True:\n        ss = ''\n        iup = i + chunk_size\n        if (iup > len(words)):\n            iup = len(words)\n        for ii in range(i, iup):\n            if(len(ss) > 0):\n                ss = ss + ' '\n            ss = ss + words[ii]\n        res.append(ss)\n        i += 1\n        if (i >= len(words)):\n            break\n    return res\n\n# Returns chunks of text with no longer than chunk_size words\n# and chunks with lower size up to words\ndef prep_str_with_sub_chunks(words, chunk_size):\n    res = []\n    for i in range(1, chunk_size + 1):\n        res.extend(prep_str_chunks(words, i))\n    return res\n\n# Make predictions for the specific text\ndef lr_predict(txt):\n    txt_cleaned, stopwords_count = clean_text_words(txt)\n    wrds = prep_str_with_sub_chunks(txt_cleaned, max_chunk_size)\n    \n    tmt_mtrx = pad_sequences(\n        tokenizer.texts_to_sequences(wrds),\n        maxlen = max_chunk_size\n    )\n    \n    preds = model.predict(tmt_mtrx, verbose = 0)\n    return np.amax(preds, axis = 0)\n\n# Make predictions for the list of texts\n# maxcnt - maximum number of texts to be predicted\n# If maxcnt < 0, whole list will be predicted\ndef lr_predict_lst(txtar, maxcnt):\n    #arresmax = [0] * 17\n    #arressum = [0] * 17\n    predictions = []\n    \n    i = 0\n    sz = len(txtar)\n    \n    if((0 < maxcnt) and (maxcnt < sz)):\n        sz = maxcnt\n    \n    for t in txtar:\n        if(i == maxcnt):\n            break\n        \n        ar_predict = lr_predict(t)\n        #arresmax = np.maximum(arresmax, ar_predict)\n        #arressum = np.add(arressum, ar_predict)\n        predictions.append(ar_predict)\n        \n        print ('Text:', i, 'of', sz, 'done       ', end=\"\\r\")\n        i += 1\n        \n    print ('Text:', sz, 'of', sz, 'done (total:',len(txtar),')')\n    return predictions\n\n\n# Plot bars according to SLMHM levels\ndef plot_SLMHM_L(preds):\n    plt.figure(figsize = (12,4))\n    df = pd.DataFrame({'levels':preds, 'labels':llabels}, index = llabels_str)\n    ax = df.plot.barh(y = 'levels')    \n    plt.ylabel('SLMHM Level')\n    plt.xlabel('Value')\n    plt.title('Maturity')\n    plt.show()\n\n# Plot bars according to SPEM patterns\ndef plot_SPEM_L(preds):\n    pspem = [0.0] * 5\n    pspem[0] = preds[0]\n    for i in range(1,5):\n        pspem[i] = max(preds[i * 4 - 3: i * 4 + 1])\n    df = pd.DataFrame({'levels':pspem, 'labels':llabels_spem}, index = llabels_spem_str)\n    ax = df.plot.barh(y = 'levels')    \n    plt.ylabel('SPEM Pattern')\n    plt.xlabel('Value')\n    plt.title('Maturity')\n    plt.show()\n    \n# Plot bars according to SLMHM levels\ndef plot_SLMHM_L2(preds1, preds2, name1, name2):\n    plt.figure(figsize = (12,4))\n    df = pd.DataFrame({name1:preds1, name2:preds2, 'labels':llabels}, index = llabels_str)\n    ax = df.plot.barh()\n    plt.ylabel('SLMHM Level')\n    plt.xlabel('Value')\n    plt.title('Maturity')\n    plt.show()\n    \n# Plot bars according to SPEM patterns\ndef plot_SPEM_L2(preds1, preds2, name1, name2):\n    pspem1 = [0.0] * 5\n    pspem1[0] = preds1[0]\n    for i in range(1,5):\n        pspem1[i] = max(preds1[i * 4 - 3: i * 4 + 1])\n    pspem2 = [0.0] * 5\n    pspem2[0] = preds2[0]\n    for i in range(1,5):\n        pspem2[i] = max(preds2[i * 4 - 3: i * 4 + 1])\n    df = pd.DataFrame({name1:pspem1, name2:pspem2, 'labels':llabels_spem}, index = llabels_spem_str)\n    ax = df.plot.barh()\n    plt.ylabel('SPEM Pattern')\n    plt.xlabel('Value')\n    plt.title('Maturity')\n    plt.show()","metadata":{"execution":{"iopub.status.busy":"2023-07-15T03:26:45.745785Z","iopub.execute_input":"2023-07-15T03:26:45.746191Z","iopub.status.idle":"2023-07-15T03:26:45.774677Z","shell.execute_reply.started":"2023-07-15T03:26:45.746161Z","shell.execute_reply":"2023-07-15T03:26:45.773311Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[↑↑↑ Table of Contents ↑↑↑](#toc)","metadata":{}},{"cell_type":"markdown","source":"<a id='3'></a>\n# 3 | Tests\n<div style='border-width: 2px;\n              border-bottom-width:4px;\n              border-bottom-color:#00FF00;\n              border-bottom-style: solid;'></div>","metadata":{}},{"cell_type":"code","source":"# All bars for this text shall be of length 1.0\n# To-Do: Underrepresenting of levels 2.2 and levels 3.2 are due to flaw\n# in training dataset or mature text.\ntest_mature_text = \"Chaos was a primary characteristics of universe. The research should consider desire as the starting point. Desire leads to intention. Intention is a readiness to plan and spend resources. Availability of resources is a feasibility. Feasibility should be achieved before the project implementation. Active project relates to structure. Structure should be evaluated to produce benefits on base of the evaluation. Continous improvement and upgrading allows to maximize the benefit. All alternatives are helpful to appearence of innovation. Innovations hould provide reward for society, economy, and natural environment. It creates a ground to maximize benefit for all and to minimize loss for all. Synergy in interaction between society, economy, and environment will appear and lead to the extraordinary innotations. People will have opportunity to express their feelings and emotions. Monitoring will generate real time information and produce new knowledge. Expansion based on new knowledge will lead to The complete fusion of all society-economy-environment interactions into something infinite and overall.\"\nplot_SLMHM_L(lr_predict(test_mature_text))","metadata":{"execution":{"iopub.status.busy":"2023-07-15T03:28:07.183523Z","iopub.execute_input":"2023-07-15T03:28:07.184002Z","iopub.status.idle":"2023-07-15T03:28:07.649420Z","shell.execute_reply.started":"2023-07-15T03:28:07.183968Z","shell.execute_reply":"2023-07-15T03:28:07.648320Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[↑↑↑ Table of Contents ↑↑↑](#toc)","metadata":{}},{"cell_type":"code","source":"# Ok: SLMHM Level 1.1\nplot_SLMHM_L(lr_predict(\"disorder\"))","metadata":{"execution":{"iopub.status.busy":"2023-07-15T03:28:13.366538Z","iopub.execute_input":"2023-07-15T03:28:13.366944Z","iopub.status.idle":"2023-07-15T03:28:14.125106Z","shell.execute_reply.started":"2023-07-15T03:28:13.366913Z","shell.execute_reply":"2023-07-15T03:28:14.123856Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[↑↑↑ Table of Contents ↑↑↑](#toc)","metadata":{}},{"cell_type":"code","source":"# Ok: SLMHM Level 1.1\nplot_SLMHM_L(lr_predict(\"idea\"))","metadata":{"execution":{"iopub.status.busy":"2023-07-15T03:28:18.045979Z","iopub.execute_input":"2023-07-15T03:28:18.046434Z","iopub.status.idle":"2023-07-15T03:28:18.439947Z","shell.execute_reply.started":"2023-07-15T03:28:18.046363Z","shell.execute_reply":"2023-07-15T03:28:18.438525Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[↑↑↑ Table of Contents ↑↑↑](#toc)","metadata":{}},{"cell_type":"code","source":"# Ok: SLMHM Level 1.2\nplot_SLMHM_L(lr_predict(\"aspiration\"))","metadata":{"execution":{"iopub.status.busy":"2023-07-15T03:28:21.843148Z","iopub.execute_input":"2023-07-15T03:28:21.843915Z","iopub.status.idle":"2023-07-15T03:28:22.249073Z","shell.execute_reply.started":"2023-07-15T03:28:21.843876Z","shell.execute_reply":"2023-07-15T03:28:22.247778Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[↑↑↑ Table of Contents ↑↑↑](#toc)","metadata":{}},{"cell_type":"code","source":"# Ok: SLMHM Level 1.3\nplot_SLMHM_L(lr_predict(\"resource\"))","metadata":{"execution":{"iopub.status.busy":"2023-07-15T03:28:25.348327Z","iopub.execute_input":"2023-07-15T03:28:25.348765Z","iopub.status.idle":"2023-07-15T03:28:25.760084Z","shell.execute_reply.started":"2023-07-15T03:28:25.348732Z","shell.execute_reply":"2023-07-15T03:28:25.759053Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[↑↑↑ Table of Contents ↑↑↑](#toc)","metadata":{}},{"cell_type":"code","source":"# Ok: SLMHM Level 1.4\nplot_SLMHM_L(lr_predict(\"structure\"))","metadata":{"execution":{"iopub.status.busy":"2023-07-15T03:28:29.520791Z","iopub.execute_input":"2023-07-15T03:28:29.521203Z","iopub.status.idle":"2023-07-15T03:28:29.974273Z","shell.execute_reply.started":"2023-07-15T03:28:29.521174Z","shell.execute_reply":"2023-07-15T03:28:29.973059Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[↑↑↑ Table of Contents ↑↑↑](#toc)","metadata":{}},{"cell_type":"code","source":"# Ok: SLMHM Level 2.1\nplot_SLMHM_L(lr_predict(\"assessment\"))","metadata":{"execution":{"iopub.status.busy":"2023-07-15T03:28:33.545540Z","iopub.execute_input":"2023-07-15T03:28:33.545950Z","iopub.status.idle":"2023-07-15T03:28:34.007686Z","shell.execute_reply.started":"2023-07-15T03:28:33.545918Z","shell.execute_reply":"2023-07-15T03:28:34.006144Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[↑↑↑ Table of Contents ↑↑↑](#toc)","metadata":{}},{"cell_type":"code","source":"# Ok: SLMHM Level 2.2\nplot_SLMHM_L(lr_predict(\"upgrade\"))","metadata":{"execution":{"iopub.status.busy":"2023-07-15T03:28:37.863413Z","iopub.execute_input":"2023-07-15T03:28:37.863860Z","iopub.status.idle":"2023-07-15T03:28:38.267308Z","shell.execute_reply.started":"2023-07-15T03:28:37.863826Z","shell.execute_reply":"2023-07-15T03:28:38.265900Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[↑↑↑ Table of Contents ↑↑↑](#toc)","metadata":{}},{"cell_type":"code","source":"# Ok: SLMHM Level 2.3\nplot_SLMHM_L(lr_predict(\"variety\"))","metadata":{"execution":{"iopub.status.busy":"2023-07-15T03:28:43.072773Z","iopub.execute_input":"2023-07-15T03:28:43.073170Z","iopub.status.idle":"2023-07-15T03:28:43.478555Z","shell.execute_reply.started":"2023-07-15T03:28:43.073139Z","shell.execute_reply":"2023-07-15T03:28:43.477430Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[↑↑↑ Table of Contents ↑↑↑](#toc)","metadata":{}},{"cell_type":"code","source":"# Ok: SLMHM Level 2.4\nplot_SLMHM_L(lr_predict(\"modernization\"))","metadata":{"execution":{"iopub.status.busy":"2023-07-15T03:28:48.118296Z","iopub.execute_input":"2023-07-15T03:28:48.118753Z","iopub.status.idle":"2023-07-15T03:28:48.527544Z","shell.execute_reply.started":"2023-07-15T03:28:48.118719Z","shell.execute_reply":"2023-07-15T03:28:48.526175Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[↑↑↑ Table of Contents ↑↑↑](#toc)[↑↑↑ Table of Contents ↑↑↑](#toc)","metadata":{}},{"cell_type":"code","source":"# Not Ok: SLMHM Level 3.1\n# To Do: Need check\nplot_SLMHM_L(lr_predict(\"mutual reward\"))","metadata":{"execution":{"iopub.status.busy":"2023-07-15T03:28:52.443899Z","iopub.execute_input":"2023-07-15T03:28:52.444382Z","iopub.status.idle":"2023-07-15T03:28:52.860669Z","shell.execute_reply.started":"2023-07-15T03:28:52.444325Z","shell.execute_reply":"2023-07-15T03:28:52.859426Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[↑↑↑ Table of Contents ↑↑↑](#toc)","metadata":{}},{"cell_type":"code","source":"# Not Ok: SLMHM Level 3.2\n# To Do: Need check\nplot_SLMHM_L(lr_predict(\"mutual optimization\"))","metadata":{"execution":{"iopub.status.busy":"2023-07-15T03:29:05.344127Z","iopub.execute_input":"2023-07-15T03:29:05.344617Z","iopub.status.idle":"2023-07-15T03:29:05.761733Z","shell.execute_reply.started":"2023-07-15T03:29:05.344581Z","shell.execute_reply":"2023-07-15T03:29:05.760439Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[↑↑↑ Table of Contents ↑↑↑](#toc)","metadata":{}},{"cell_type":"code","source":"# Ok: SLMHM Level 3.3\nplot_SLMHM_L(lr_predict(\"harmony\"))","metadata":{"execution":{"iopub.status.busy":"2023-07-15T03:29:17.507548Z","iopub.execute_input":"2023-07-15T03:29:17.507971Z","iopub.status.idle":"2023-07-15T03:29:17.920206Z","shell.execute_reply.started":"2023-07-15T03:29:17.507940Z","shell.execute_reply":"2023-07-15T03:29:17.918506Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[↑↑↑ Table of Contents ↑↑↑](#toc)","metadata":{}},{"cell_type":"code","source":"# Not Ok: SLMHM Level 3.4\nplot_SLMHM_L(lr_predict(\"fascinating\"))","metadata":{"execution":{"iopub.status.busy":"2023-07-15T03:29:21.862878Z","iopub.execute_input":"2023-07-15T03:29:21.864142Z","iopub.status.idle":"2023-07-15T03:29:22.293382Z","shell.execute_reply.started":"2023-07-15T03:29:21.864086Z","shell.execute_reply":"2023-07-15T03:29:22.292187Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[↑↑↑ Table of Contents ↑↑↑](#toc)","metadata":{}},{"cell_type":"code","source":"# Ok: SLMHM Level 4.1\nplot_SLMHM_L(lr_predict(\"realtime\"))","metadata":{"execution":{"iopub.status.busy":"2023-07-15T03:29:25.647579Z","iopub.execute_input":"2023-07-15T03:29:25.647994Z","iopub.status.idle":"2023-07-15T03:29:26.112047Z","shell.execute_reply.started":"2023-07-15T03:29:25.647964Z","shell.execute_reply":"2023-07-15T03:29:26.110676Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[↑↑↑ Table of Contents ↑↑↑](#toc)","metadata":{}},{"cell_type":"code","source":"# Not Ok: SLMHM Level 4.2\n# To Do: Need check\nplot_SLMHM_L(lr_predict(\"knowledge extraction\"))","metadata":{"execution":{"iopub.status.busy":"2023-07-15T03:29:30.014731Z","iopub.execute_input":"2023-07-15T03:29:30.015178Z","iopub.status.idle":"2023-07-15T03:29:30.504570Z","shell.execute_reply.started":"2023-07-15T03:29:30.015148Z","shell.execute_reply":"2023-07-15T03:29:30.503431Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[↑↑↑ Table of Contents ↑↑↑](#toc)","metadata":{}},{"cell_type":"code","source":"# Ok: SLMHM Level 4.3\nplot_SLMHM_L(lr_predict(\"scaling\"))","metadata":{"execution":{"iopub.status.busy":"2023-07-15T03:29:40.324247Z","iopub.execute_input":"2023-07-15T03:29:40.324743Z","iopub.status.idle":"2023-07-15T03:29:40.789209Z","shell.execute_reply.started":"2023-07-15T03:29:40.324707Z","shell.execute_reply":"2023-07-15T03:29:40.788039Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[↑↑↑ Table of Contents ↑↑↑](#toc)","metadata":{}},{"cell_type":"code","source":"# Ok: SLMHM Level 4.4\nplot_SLMHM_L(lr_predict(\"worldwide\"))","metadata":{"execution":{"iopub.status.busy":"2023-07-15T03:29:44.234601Z","iopub.execute_input":"2023-07-15T03:29:44.235779Z","iopub.status.idle":"2023-07-15T03:29:44.729795Z","shell.execute_reply.started":"2023-07-15T03:29:44.235725Z","shell.execute_reply":"2023-07-15T03:29:44.728548Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[↑↑↑ Table of Contents ↑↑↑](#toc)","metadata":{}},{"cell_type":"markdown","source":"<a id='4'></a>\n# 4 | Drawbacks\n<div style='border-width: 2px;\n              border-bottom-width:4px;\n              border-bottom-color:#00FF00;\n              border-bottom-style: solid;'></div>\n\nThe drawback of the current version (July 3, 2023):\n- Training Database ([data](https://www.kaggle.com/datasets/dmitriygakh/see-slmhm-spem)) contains some incorrect relations phrase-level (levels 3.1 and 3.2);\n- Training Database is sligthtly disbalanced and need suplement;\n- AI model is overfitted;\n- Lack of validation data;\n- Lack of Society-Economy-Ecology relations.\n\n[↑↑↑ Table of Contents ↑↑↑](#toc)","metadata":{}},{"cell_type":"markdown","source":"<a id='5'></a>\n# 5 | Free Text\n<div style='border-width: 2px;\n              border-bottom-width:4px;\n              border-bottom-color:#00FF00;\n              border-bottom-style: solid;'></div>\n              \nYou can assess your own text.","metadata":{}},{"cell_type":"code","source":"my_text = \"My text\"\nplot_SLMHM_L(lr_predict(my_text))","metadata":{"execution":{"iopub.status.busy":"2023-07-15T03:29:51.394851Z","iopub.execute_input":"2023-07-15T03:29:51.395345Z","iopub.status.idle":"2023-07-15T03:29:51.841143Z","shell.execute_reply.started":"2023-07-15T03:29:51.395308Z","shell.execute_reply":"2023-07-15T03:29:51.839838Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[↑↑↑ Table of Contents ↑↑↑](#toc)","metadata":{}}]}