{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"![](https://875478.smushcdn.com/1946825/wp-content/uploads/2019/03/quick_overview_autoML_1600x700_web.jpg?lossy=1&strip=1&webp=1)","metadata":{}},{"cell_type":"markdown","source":"# Index: \n---\n1. [Basics of AutoML](https://www.kaggle.com/soham1024/know-about-different-automl-frameworks?scriptVersionId=43751621#Firstly,-let's-know-about-AutoML)                 \n    a. [When to use AutoML](https://www.kaggle.com/soham1024/know-about-different-automl-frameworks?scriptVersionId=43751621#When-to-use-AutoML:-classify,-regression,-&-forecast)          \n    b. [How AutoML Works](https://www.kaggle.com/soham1024/know-about-different-automl-frameworks?scriptVersionId=43751621#How-AutoML-works)           \n    c. [Feature Engineering](https://www.kaggle.com/soham1024/know-about-different-automl-frameworks?scriptVersionId=43751621#Feature-engineering)             \n    d. [Ensemble Models](https://www.kaggle.com/soham1024/know-about-different-automl-frameworks?scriptVersionId=43751621#Ensemble-models)                    \n    e. [Pros and Cons](https://www.kaggle.com/soham1024/know-about-different-automl-frameworks?scriptVersionId=43751621#Pros-and-cons)        \n2. [Types of AutoML](https://www.kaggle.com/soham1024/know-about-different-automl-frameworks?scriptVersionId=43751621#Types-of-AutoML)\n3. [Examples of AutoML](https://www.kaggle.com/soham1024/know-about-different-automl-frameworks?scriptVersionId=43751621#Examples)    \n    a. [Google AI](https://www.kaggle.com/soham1024/know-about-different-automl-frameworks?scriptVersionId=43751621#Example-on-Google-Cloud-AutoML)    \n    b. [H2O.Ai](https://www.kaggle.com/soham1024/know-about-different-automl-frameworks?scriptVersionId=43751621#Example-on-H2O.Ai)     \n    c. [Auto-Sklearn](https://www.kaggle.com/soham1024/know-about-different-automl-frameworks?scriptVersionId=43751621#Auto-Sklearnhttps://www.kaggle.com/soham1024/know-about-different-automl-frameworks?scriptVersionId=43751621#Auto-Sklearn)    \n    d. [ML Box](https://www.kaggle.com/soham1024/know-about-different-automl-frameworks?scriptVersionId=43751621#ML-Box)       \n    e. [Auto-Keras](https://www.kaggle.com/soham1024/know-about-different-automl-frameworks?scriptVersionId=43751621#Auto-Keras)        \n    f. [TPOT](https://www.kaggle.com/soham1024/know-about-different-automl-frameworks?scriptVersionId=43751621#TPOT)         \n    g. Pycaret(Coming soon)\n4. [Referances](https://www.kaggle.com/soham1024/know-about-different-automl-frameworks?scriptVersionId=43751621#Referances:)     ","metadata":{}},{"cell_type":"markdown","source":"---\n\n# Firstly, Let's Know About AutoML\n\n---","metadata":{}},{"cell_type":"markdown","source":"Automated machine learning, also referred to as automated ML or AutoML, is the process of automating the time consuming, iterative tasks of machine learning model development. It allows data scientists, analysts, and developers to build ML models with high scale, efficiency, and productivity all while sustaining model quality. \n\nTraditional machine learning model development is resource-intensive, requiring significant domain knowledge and time to produce and compare dozens of models. With automated machine learning, you'll accelerate the time it takes to get production-ready ML models with great ease and efficiency.","metadata":{}},{"cell_type":"markdown","source":"![](https://miro.medium.com/max/2400/1*Vubgyj96KsskE6eRuxq58w.png)","metadata":{}},{"cell_type":"markdown","source":"---\n\n# When to use AutoML: classify, regression, & forecast\n\n---","metadata":{}},{"cell_type":"markdown","source":"Apply automated ML when you want Azure Machine Learning to train and tune a model for you using the target metric you specify. Automated ML democratizes the machine learning model development process, and empowers its users, no matter their data science expertise, to identify an end-to-end machine learning pipeline for any problem.\n\nData scientists, analysts, and developers across industries can use automated ML to:\n\n    Implement ML solutions without extensive programming knowledge\n    Save time and resources\n    Leverage data science best practices\n    Provide agile problem-solving\n","metadata":{}},{"cell_type":"markdown","source":"---\n\n> ## Classification\n\n---\n\n![](https://miro.medium.com/max/1200/1*PM4dqcAe6N7kWRpXKwgWag.png)\n\n---\n\nClassification is a common machine learning task. Classification is a type of supervised learning in which models learn using training data, and apply those learnings to new data. Azure Machine Learning offers featurizations specifically for these tasks, such as deep neural network text featurizers for classification\n\nThe main goal of classification models is to predict which categories new data will fall into based on learnings from its training data. Common classification examples include fraud detection, handwriting recognition, and object detection\n\n---","metadata":{}},{"cell_type":"markdown","source":"---\n\n> ## Regression\n\n---\n\n![](https://miro.medium.com/max/4096/1*1sAafDO7V5HT5MzpknGw7w.png)\n\n---\n\nSimilar to classification, regression tasks are also a common supervised learning task\n\nDifferent from classification where predicted output values are categorical, regression models predict numerical output values based on independent predictors. In regression, the objective is to help establish the relationship among those independent predictor variables by estimating how one variable impacts the others. For example, automobile price based on features like, gas mileage, safety rating, etc.\n\n---","metadata":{}},{"cell_type":"markdown","source":"---\n\n> ## Time-series Forecasting\n\n---\n\n![](https://machinelearningblogcom.files.wordpress.com/2018/01/153775-636270895711868987-16x9.jpg?w=1400)\n\n---\n\nBuilding forecasts is an integral part of any business, whether it's revenue, inventory, sales, or customer demand. You can use automated ML to combine techniques and approaches and get a recommended, high-quality time-series forecast.   \n\nAn automated time-series experiment is treated as a multivariate regression problem. Past time-series values are \"pivoted\" to become additional dimensions for the regressor together with other predictors. This approach, unlike classical time series methods, has an advantage of naturally incorporating multiple contextual variables and their relationship to one another during training. Automated ML learns a single, but often internally branched model for all items in the dataset and prediction horizons. More data is thus available to estimate model parameters and generalization to unseen series becomes possible.\n\nAdvanced forecasting configuration includes:\n\n    holiday detection and featurization\n    time-series and DNN learners (Auto-ARIMA, Prophet, ForecastTCN)\n    many models support through grouping\n    rolling-origin cross validation\n    configurable lags\n    rolling window aggregate features\n\n---","metadata":{}},{"cell_type":"markdown","source":"---\n\n# How AutoML Works\n\n---\n\n![](https://9to5google.com/wp-content/uploads/sites/4/2019/04/google-cloud-automl-updates.jpg?quality=82&strip=all&w=1600)\n\n---","metadata":{}},{"cell_type":"markdown","source":"During training, Azure Machine Learning creates a number of pipelines in parallel that try different algorithms and parameters for you. The service iterates through ML algorithms paired with feature selections, where each iteration produces a model with a training score. The higher the score, the better the model is considered to \"fit\" your data. It will stop once it hits the exit criteria defined in the experiment.","metadata":{}},{"cell_type":"markdown","source":"you can design and run your automated ML training experiments with these steps:\n\n    Identify the ML problem to be solved: classification, forecasting, or regression\n\n    Choose whether you want to use the Python SDK or the studio web experience\n\n    Specify the source and format of the labeled training data: Numpy arrays or Pandas dataframe\n\n    Configure the compute target for model training\n\n    Configure the automated machine learning parameters that determine how many iterations over different models, hyperparameter settings, advanced preprocessing/featurization, and what metrics to look at when determining the best model.\n\n    Submit the training run.\n\n    Review the results\n","metadata":{}},{"cell_type":"markdown","source":"> ![](https://docs.microsoft.com/en-us/azure/machine-learning/media/concept-automated-ml/automl-concept-diagram2.png)","metadata":{}},{"cell_type":"markdown","source":"You can also inspect the logged run information, which contains metrics gathered during the run","metadata":{}},{"cell_type":"markdown","source":"![](https://miro.medium.com/max/5060/1*SV9MUnhPSxQFt37wOr2iyw.png)","metadata":{}},{"cell_type":"markdown","source":"---\n\n# Feature Engineering\n\n---\n\n![](https://miro.medium.com/max/1200/1*K6ctE0RZme0cqMtknrxq8A.png)\n\n---\n\nFeature engineering is the process of using domain knowledge of the data to create features that help ML algorithms learn better. In Azure Machine Learning, scaling and normalization techniques are applied to facilitate feature engineering. Collectively, these techniques and feature engineering are referred to as featurization.\n\nFor automated machine learning experiments, featurization is applied automatically, but can also be customized based on your data\n\n> Automated machine learning featurization steps (feature normalization, handling missing data, converting text to numeric, etc.) become part of the underlying model. When using the model for predictions, the same featurization steps applied during training are applied to your input data automatically.","metadata":{}},{"cell_type":"markdown","source":"---\n\n# Ensemble Models\n\n---\n\n![](https://miro.medium.com/max/2556/1*P0ns6A56MtpGFMQ2g47IYA.png)\n\n---\n\nAutomated machine learning supports ensemble models, which are enabled by default. Ensemble learning improves machine learning results and predictive performance by combining multiple models as opposed to using single models. The ensemble iterations appear as the final iterations of your run. Automated machine learning uses both voting and stacking ensemble methods for combining models:\n\n    Voting: predicts based on the weighted average of predicted class probabilities (for classification tasks) or predicted regression targets (for regression tasks).\n    Stacking: stacking combines heterogenous models and trains a meta-model based on the output from the individual models. The current default meta-models are LogisticRegression for classification tasks and ElasticNet for regression/forecasting tasks.\n\nThe [Caruana ensemble selection algorithm](http://www.niculescu-mizil.org/papers/shotgun.icml04.revised.rev2.pdf) with sorted ensemble initialization is used to decide which models to use within the ensemble. At a high level, this algorithm initializes the ensemble with up to five models with the best individual scores, and verifies that these models are within 5% threshold of the best score to avoid a poor initial ensemble. Then for each ensemble iteration, a new model is added to the existing ensemble and the resulting score is calculated. If a new model improved the existing ensemble score, the ensemble is updated to include the new model.","metadata":{}},{"cell_type":"markdown","source":"---\n\n# Pros and Cons\n\n---\n\n![](https://miro.medium.com/max/2896/1*yznfOpXlLD_kqPW-5ggKsg.png)\n\n---","metadata":{}},{"cell_type":"markdown","source":"---\n\n# Types of AutoML\n\n---","metadata":{}},{"cell_type":"markdown","source":"![](https://miro.medium.com/max/4180/1*d7n1wOYUE3e-fE6NVqTryg.png)","metadata":{}},{"cell_type":"markdown","source":"---\n---\n\n# Examples\n\n---","metadata":{}},{"cell_type":"markdown","source":"---\n\n> # Example on Google Cloud AutoML\n\n---","metadata":{}},{"cell_type":"markdown","source":"![](https://helpdev.eu/wp-content/uploads/2018/11/GoogleAdaNet.jpg)\n\n---\n\nGoogle launched the Google Auto ML framework which integrates the powers of neural network architecture. Its graphical user interface (GUI) is simple to use for model processing models which makes Google Cloud Auto ML useful for citizen developers and citizen data scientists who have limited ML knowledge to process ML-models development.\n\nHowever, Google Cloud Auto ML is a paid platform, which makes it feasible to use if only for commercial projects. Besides this Auto ML toolkit is available free of charge for research purposes throughout the year.","metadata":{}},{"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport time\nfrom datetime import datetime\n\nfrom sklearn.model_selection import train_test_split\n\nfrom google.cloud import storage\nfrom google.cloud import automl_v1beta1 as automl\n\n# workaround to fix gapic_v1 error\nfrom google.api_core.gapic_v1.client_info import ClientInfo\n\nfrom automlwrapper import AutoMLWrapper","metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"This notebook utilizes a utility script that wraps much of the AutoML Python client library, to make the code in this notebook easier to read. Feel free to check out the utility for all the details on how we are calling the underlying AutoML Client Library!","metadata":{}},{"cell_type":"code","source":"# Set your own values for these. bucket_name should be the project_id + '-lcm'.\nPROJECT_ID = 'cloudml-demo'\nbucket_name = 'cloudml-demo-lcm'\n\nregion = 'us-central1' # Region must be us-central1\ndataset_display_name = 'kaggle_tweets'\nmodel_display_name = 'kaggle_starter_model1'\n\nstorage_client = storage.Client(project=PROJECT_ID)\n\n# adding ClientInfo here to get the gapic_v1 call in place\nclient = automl.AutoMlClient(client_info=ClientInfo())\n\nprint(f'Starting AutoML notebook at {datetime.fromtimestamp(time.time()).strftime(\"%Y-%m-%d, %H:%M:%S UTC\")}')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Input data files are available in the \"../input/\" directory.\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# Any results you write to the current directory are saved as output.","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"nlp_train_df = pd.read_csv('/kaggle/input/nlp-getting-started/train.csv')\nnlp_test_df = pd.read_csv('/kaggle/input/nlp-getting-started/test.csv')\ndef callback(operation_future):\n    result = operation_future.result()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"nlp_train_df.tail()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Data spelunking\nHow often does 'fire' come up in this dataset?","metadata":{}},{"cell_type":"code","source":"nlp_train_df.loc[nlp_train_df['text'].str.contains('fire', na=False, case=False)]","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Does the presence of the word 'fire' help determine whether the tweets here are real or false?","metadata":{}},{"cell_type":"code","source":"nlp_train_df.loc[nlp_train_df['text'].str.contains('fire', na=False, case=False)].target.value_counts()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### GCS upload/download utilities\nThese functions make upload and download of files from the kernel to Google Cloud Storage easier. This is needed for AutoML","metadata":{}},{"cell_type":"code","source":"def upload_blob(bucket_name, source_file_name, destination_blob_name):\n    \"\"\"Uploads a file to the bucket. https://cloud.google.com/storage/docs/ \"\"\"\n    bucket = storage_client.get_bucket(bucket_name)\n    blob = bucket.blob(destination_blob_name)\n    blob.upload_from_filename(source_file_name)\n    print('File {} uploaded to {}'.format(\n        source_file_name,\n        'gs://' + bucket_name + '/' + destination_blob_name))\n    \ndef download_to_kaggle(bucket_name,destination_directory,file_name,prefix=None):\n    \"\"\"Takes the data from your GCS Bucket and puts it into the working directory of your Kaggle notebook\"\"\"\n    os.makedirs(destination_directory, exist_ok = True)\n    full_file_path = os.path.join(destination_directory, file_name)\n    blobs = storage_client.list_blobs(bucket_name,prefix=prefix)\n    for blob in blobs:\n        blob.download_to_filename(full_file_path)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"bucket = storage.Bucket(storage_client, name=bucket_name)\nif not bucket.exists():\n    bucket.create(location=region)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Export to CSV and upload to GCS","metadata":{}},{"cell_type":"code","source":"# Select the text body and the target value, for sending to AutoML NL\nnlp_train_df[['text','target']].to_csv('train.csv', index=False, header=False) ","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"nlp_train_df[['id','text','target']].head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"training_gcs_path = 'uploads/kaggle_getstarted/full_train.csv'\nupload_blob(bucket_name, 'train.csv', training_gcs_path)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Create our class instance","metadata":{}},{"cell_type":"code","source":"amw = AutoMLWrapper(client=client, \n                    project_id=PROJECT_ID, \n                    bucket_name=bucket_name, \n                    region='us-central1', \n                    dataset_display_name=dataset_display_name, \n                    model_display_name=model_display_name)\n       ","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Create (or retreive) dataset\nCheck to see if this dataset already exists. If not, create it","metadata":{}},{"cell_type":"code","source":"print(f'Getting dataset ready at {datetime.fromtimestamp(time.time()).strftime(\"%Y-%m-%d, %H:%M:%S UTC\")}')\nif not amw.get_dataset_by_display_name(dataset_display_name):\n    print('dataset not found')\n    amw.create_dataset()\n    amw.import_gcs_data(training_gcs_path)\n\namw.dataset\nprint(f'Dataset ready at {datetime.fromtimestamp(time.time()).strftime(\"%Y-%m-%d, %H:%M:%S UTC\")}')\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Kick off the training for the model\nAnd retrieve the training info after completion.      \nStart model deployment.","metadata":{}},{"cell_type":"code","source":"print(f'Getting model trained at {datetime.fromtimestamp(time.time()).strftime(\"%Y-%m-%d, %H:%M:%S UTC\")}')\n\nif not amw.get_model_by_display_name():\n    print(f'Training model at {datetime.fromtimestamp(time.time()).strftime(\"%Y-%m-%d, %H:%M:%S UTC\")}')\n    amw.train_model()\n\nprint(f'Model trained. Ensuring model is deployed at {datetime.fromtimestamp(time.time()).strftime(\"%Y-%m-%d, %H:%M:%S UTC\")}')\namw.deploy_model()\namw.model\nprint(f'Model trained and deployed at {datetime.fromtimestamp(time.time()).strftime(\"%Y-%m-%d, %H:%M:%S UTC\")}')\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"amw.model_full_path","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Prediction\nNote that prediction will not run until deployment finishes, which takes a bit of time.\nHowever, once you have your model deployed, this notebook won't re-train the model, thanks to the various safeguards put in place. Instead, it will take the existing (trained) model and make predictions and generate the submission file.","metadata":{}},{"cell_type":"code","source":"nlp_test_df.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(f'Begin getting predictions at {datetime.fromtimestamp(time.time()).strftime(\"%Y-%m-%d, %H:%M:%S UTC\")}')\n\n# Create client for prediction service.\nprediction_client = automl.PredictionServiceClient()\namw.set_prediction_client(prediction_client)\n\npredictions_df = amw.get_predictions(nlp_test_df, \n                                     input_col_name='text', \n#                                      ground_truth_col_name='target', # we don't have ground truth in our test set\n                                     limit=None, \n                                     threshold=0.5,\n                                     verbose=False)\n\nprint(f'Finished getting predictions at {datetime.fromtimestamp(time.time()).strftime(\"%Y-%m-%d, %H:%M:%S UTC\")}')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## (optional) Undeploy model\nUndeploy the model to stop charges","metadata":{}},{"cell_type":"code","source":"amw.undeploy_model()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Create submission output","metadata":{}},{"cell_type":"code","source":"predictions_df.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission_df = pd.concat([nlp_test_df['id'], predictions_df['class']], axis=1)\nsubmission_df.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# predictions_df['class'].iloc[:10]\n# nlp_test_df['id']","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission_df = submission_df.rename(columns={'class':'target'})\nsubmission_df.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n\n> # Example on H2O.Ai\n\n\n---\n\n![](https://analyticsindiamag.com/wp-content/uploads/2020/08/2020-08-20-2.jpg)\n\n---\n\nThis is an open-source, memory inclusive and distributed machine learning platform to build supervised and unsupervised machine learning models. It also includes a user-friendly UI platform called Flow where you can create these models. \n\nH2O AutoML framework is best suited to those who are searching for deep learning mechanisms. H2O AutoML can perform many tasks which requires many lines of code at the simultaneously.\n\nH2O AutoML supports both traditional neural networks and machine learning models. It is especially suitable for developers who want to automate deep learning.","metadata":{}},{"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n\ntest = pd.read_csv(\"../input/hackerearth-ml-challenge-pet-adoption/test.csv\")\ntrain = pd.read_csv(\"../input/hackerearth-ml-challenge-pet-adoption/train.csv\")\n\n#Train\ntrain['issue_date'] = pd.to_datetime(train['issue_date'])\ntrain['listing_date'] = pd.to_datetime(train['listing_date'])\ntrain['duration'] = (train['listing_date'] - train['issue_date']).dt.days  \ntrain = train.drop(['pet_id','issue_date','listing_date'],axis=1)\n\n\n#Test\ntest['issue_date'] = pd.to_datetime(test['issue_date'])\ntest['listing_date'] = pd.to_datetime(test['listing_date'])\ntest['duration'] = (test['listing_date'] - test['issue_date']).dt.days  \ntest = test.drop(['pet_id','issue_date','listing_date'],axis=1)\n\n\n\n#Train\nfrom sklearn.preprocessing import LabelEncoder\ntrain['color_number'] = LabelEncoder().fit_transform(train['color_type'])\n\n#Test\ntest['color_number'] = LabelEncoder().fit_transform(test['color_type'])\n\n#Train\ntrain = train.fillna(-1)\n#Test\ntest = test.fillna(-1)\n\n#train\ninfo_1 = pd.DataFrame()\ninfo_1['length(m)'] = [np.percentile(train['length(m)'],25*i) for i in range(1,4)]\ninfo_1['height(cm)'] = [np.percentile(train['height(cm)'],25*i) for i in range(1,4)]\ninfo_1['duration'] = [np.percentile(train['duration'],25*i) for i in range(1,4)]\ninfo_1\n\n#test\ninfo_2 = pd.DataFrame()\ninfo_2['length(m)'] = [np.percentile(test['length(m)'],25*i) for i in range(1,4)]\ninfo_2['height(cm)'] = [np.percentile(test['height(cm)'],25*i) for i in range(1,4)]\ninfo_2['duration'] = [np.percentile(test['duration'],25*i) for i in range(1,4)]\ninfo_2\n\n#Train\ninfo_1.loc[3] = [2.5*info_1.loc[0,column] - 1.5*info_1.loc[2,column] for column in info_1.columns]\ninfo_1.loc[4] = [2.5*info_1.loc[2,column] - 1.5*info_1.loc[0,column] for column in info_1.columns]\ninfo_1\n\n#test\ninfo_2.loc[3] = [2.5*info_2.loc[0,column] - 1.5*info_2.loc[2,column] for column in info_2.columns]\ninfo_2.loc[4] = [2.5*info_2.loc[2,column] - 1.5*info_2.loc[0,column] for column in info_2.columns]\ninfo_2\n\n#train\ndef range_part_train(column,value):\n    if value > info_1.loc[4,column]:\n        return 5\n    elif value > info_1.loc[2,column]:\n        return 4\n    elif value > info_1.loc[1,column]:\n        return 3\n    elif value > info_1.loc[0,column]:\n        return 2\n    elif value > info_1.loc[3,column]:\n        return 1\n    else:\n        return 0\n\n#test\ndef range_part_test(column,value):\n    if value > info_2.loc[4,column]:\n        return 5\n    elif value > info_2.loc[2,column]:\n        return 4\n    elif value > info_2.loc[1,column]:\n        return 3\n    elif value > info_2.loc[0,column]:\n        return 2\n    elif value > info_2.loc[3,column]:\n        return 1\n    else:\n        return 0\n\n#train\nfrom tqdm import tqdm\ntqdm.pandas()\ntrain['length_range'] = train['length(m)'].progress_apply(lambda x:range_part_train('length(m)',x))\ntrain['height_range'] = train['height(cm)'].progress_apply(lambda x:range_part_train('height(cm)',x))\ntrain['duration_range'] = train['duration'].progress_apply(lambda x:range_part_train('duration',x))\n\n#test\ntqdm.pandas()\ntest['length_range'] = test['length(m)'].progress_apply(lambda x:range_part_test('length(m)',x))\ntest['height_range'] = test['height(cm)'].progress_apply(lambda x:range_part_test('height(cm)',x))\ntest['duration_range'] = test['duration'].progress_apply(lambda x:range_part_test('duration',x))\n\n\n\n\nX_train = train.drop(['breed_category','pet_category','color_type'], axis=1)\nY_train = train[\"breed_category\"]\nZ_train = train[\"pet_category\"]\n\nX_test=test.drop(['color_type'], axis=1)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Now Preprocessing is Done. ","metadata":{}},{"cell_type":"code","source":"# # installing Automl\n# !pip install automl","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# importing \nimport h2o\nfrom h2o.automl import H2OAutoML\n\n# Forming H2O Friendly Dataframe\nh2o.init()\nX_y_train_h = h2o.H2OFrame(pd.concat([X_train, Y_train], axis='columns'))\nX_y_train_h['breed_category'] = X_y_train_h['breed_category'].asfactor()\n#  the target column should have categorical type for classification tasks\n#   (numerical type for regression tasks)\n\nX_test_h = h2o.H2OFrame(X_test)\n\nX_y_train_h.describe()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"aml = H2OAutoML(\n    max_runtime_secs=(3600*5),  # 5 hours\n    max_models=None,  # no limit\n    seed=42\n)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Defining Feature Columns\nfeature_cols = ['condition','length_range','height_range','duration_range','color_number','X1','X2']","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\n\naml.train(\n    x=feature_cols,\n    y='breed_category',\n    training_frame=X_y_train_h\n)\n\nlb = aml.leaderboard\nmodel_ids = list(lb['model_id'].as_data_frame().iloc[:,0])\nout_path = \".\"\n\nfor m_id in model_ids:\n    mdl = h2o.get_model(m_id)\n    h2o.save_model(model=mdl, path=out_path, force=True)\n\nh2o.export_file(lb, os.path.join(out_path, 'aml_leaderboard.h2o'), force=True)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Checking Leaderboard for the Models\nlb = h2o.automl.get_leaderboard(aml, extra_columns = 'ALL')\nlb","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# finding Prediction from the best fitted model\npred = aml.leader.predict(X_test_h)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n> # Example on Auto-Sklearn\n\n---\n\n![](https://www.automl.org/wp-content/uploads/2016/08/auto-sklearn-5.png)\n\n---\n\nAuto Sklearn is an automated machine learning toolkit based on Bayesian optimization, meta-learning, ensemble construction. It frees a machine learning user from algorithm selection and hyperparameter tuning.\n\nThe Auto Sklearn, AutoML package includes 15 classification algorithms besides 14 for feature pre-processing which defines the right algorithm to optimise parameter accuracy at a precision level of more than 0.98. Auto Sklean translates well for small and medium datasets, however, developers face a hiccup with dealing with large datasets.","metadata":{}},{"cell_type":"code","source":"!pip install auto-sklearn","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import autosklearn.classification\n\nimport sklearn.model_selection\n\nimport sklearn.datasets\n\nimport sklearn.metrics\n\nX, y = sklearn.datasets.load_digits(return_X_y=True)\n\nX_train, X_test, y_train, y_test = \\\n        sklearn.model_selection.train_test_split(X, y, random_state=1)\n\nautoml = autosklearn.classification.AutoSklearnClassifier()\n\nautoml.fit(X_train, y_train)\n\ny_hat = automl.predict(X_test)\n\nprint(\"Accuracy score\", sklearn.metrics.accuracy_score(y_test, y_hat))","metadata":{"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n\n> # Example on ML Box\n\n---\n\n![](https://mlbox.readthedocs.io/en/latest/_images/logo.png)\n\n---\n\nML Box is a data Python based library offering the features of read, pre-process, clean and format data with an option to choose specific features and detect a leak.ML Box Auto ML toolkit can classify and regress state-of-the-art models for predictions and model interpreting.\n\nML Box also offers developers with data preparation, model selection and hyper Parameter Search, however this AutoML toolkit is more suitable for the Linux operating systems. Windows and Mac users can experience some difficulties while installing ML Box","metadata":{}},{"cell_type":"code","source":"from mlbox.preprocessing import *\nfrom mlbox.optimisation import *\nfrom mlbox.prediction import *","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"paths = [\"../input/titanic/train.csv\",\"../input/titanic/test.csv\"]\ntarget_name = \"Survived\"","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"rd = Reader(sep = \",\")\ndf = rd.train_test_split(paths, target_name)   #reading and preprocessing (dates, ...)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dft = Drift_thresholder()\ndf = dft.fit_transform(df)   #removing non-stable features (like ID,...)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"opt = Optimiser(scoring = \"accuracy\", n_folds = 5)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# LightGBM\n\nspace = {\n    \n        'est__strategy':{\"search\":\"choice\",\n                                  \"space\":[\"LightGBM\"]},    \n        'est__n_estimators':{\"search\":\"choice\",\n                                  \"space\":[150]},    \n        'est__colsample_bytree':{\"search\":\"uniform\",\n                                  \"space\":[0.8,0.95]},\n        'est__subsample':{\"search\":\"uniform\",\n                                  \"space\":[0.8,0.95]},\n        'est__max_depth':{\"search\":\"choice\",\n                                  \"space\":[5,6,7,8,9]},\n        'est__learning_rate':{\"search\":\"choice\",\n                                  \"space\":[0.07]} \n    \n        }\n\nparams = opt.optimise(space, df,15)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"prd = Predictor()\nprd.fit_predict(params, df)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n\nBut you can also tune the whole Pipeline ! Indeed, you can choose:\n\n    different strategies to impute missing values\n    different strategies to encode categorical features (entity embeddings, ...)\n    different strategies and thresholds to select relevant features (random forest feature importance, l1 regularization, ...)\n    to add stacking meta-features !\n    different models and hyper-parameters (XGBoost, Random Forest, Linear, ...)\n\n","metadata":{}},{"cell_type":"code","source":"submit = pd.read_csv(\"../input/gendermodel.csv\",sep=',')\npreds = pd.read_csv(\"save/\"+target_name+\"_predictions.csv\")\n\nsubmit[target_name] =  preds[target_name+\"_predicted\"].values\n\nsubmit.to_csv(\"mlbox.csv\", index=False)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n\n> # Example on Auto-Keras\n\n---\n\n![](https://pyimagesearch.com/wp-content/uploads/2019/01/autokeras_header.jpg)\n\n---\n\nAutoKeras an open-source deep learning framework which is built on network morphism with an aim to boost Bayesian optimization. This AutoML framework can automatically search for hyperparameters and architecture for complex models. AutoKeras conducts searches with the help of Neural Architecture Search (NAS) algorithms to ultimately eliminate the need for deep learning engineers.\n\nThis Auto Machine Learning Toolkit follows the design of the classic scikit-learn API, however, it uses a powerful neural network search for model parameters using Keras.","metadata":{}},{"cell_type":"code","source":"!pip install -q  keras torch torchvision graphviz autokeras  onnxmltools\n!pip install -q    onnxmltools onnx-tf\n#!pip install --user autokeras \n!pip install git+https://github.com/jhfjhfj1/autokeras.git\nimport keras\nimport autokeras as ak\nimport onnxmltools","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train = pd.read_csv('../input/digit-recognizer/train.csv', header=None)\ntest = pd.read_csv('../input/digit-recognizer/test.csv', header=None)\ntrain.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_data = train.iloc[:, 1:]\ntrain_labels = train.iloc[:, 0]\ntest_data = test.iloc[:, 1:]\ntest_labels = test.iloc[:, 0]","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"clf = ak.ImageClassifier(verbose=True, augment=False)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"time_limit=7*3600","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"clf.fit(x_train, y_train, time_limit=time_limit)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"clf.final_fit(x_train, y_train, x_test, y_test)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"clf.evaluate(x_test, y_test)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"results = clf.predict(x_test)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model=clf.cnn.best_model #keras.models.load_model(\"model.h5\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import IPython\nfrom graphviz import Digraph\ndot = Digraph(comment='autokeras model')\ngraph=clf.cnn.best_model\nfor index, node in enumerate(graph.node_list):\n    dot.node(str(index), str(node.shape))\n\nfor u in range(graph.n_nodes):\n    for v, layer_id in graph.adj_list[u]:\n      dot.edge(str(u), str(v), str(graph.layer_list[layer_id]))\ndot.render(filename='model.png',format='png')\ndot.render(filename='model.svg',format='svg')\nIPython.display.Image(filename='model.png')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import IPython\n\nkeras_model=clf.cnn.best_model.produce_keras_model()\n\nkeras.utils.plot_model(keras_model, show_shapes=True, to_file='model_keras_mnist.png')\nIPython.display.Image(filename='model_keras_mnist.png')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"keras_model.summary()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"keras_model.compile(\"adam\",\"mse\")\nkeras_model.save(\"model.h5\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#clf.export_keras_model(\"model.h5\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"keras_model=keras.models.load_model(\"model.h5\")\nkeras_model.compile(\"adam\",\"mse\")\nonnx_model = onnxmltools.convert_keras(keras_model, target_opset=7) ","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Save as text\nonnxmltools.utils.save_text(onnx_model, 'model.json')\n\n# Save as protobuf\nonnxmltools.utils.save_model(onnx_model, 'model.onnx')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import keras2onnx\nonnx_model = keras2onnx.convert_keras(keras_model,\"autokeras nmist\")\n# Save as text\nonnxmltools.utils.save_text(onnx_model, 'model_keras2onnx.json')\n\n# Save as protobuf\nonnxmltools.utils.save_model(onnx_model, 'model_keras2onnx.onnx","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n\n> # Example on TPOT (Tree-basedPipeline Optimization Tool)\n\n---\n\n![](https://raw.githubusercontent.com/EpistasisLab/tpot/master/images/tpot-logo.jpg)\n\n---\n\nIt was 2018, that TPOT was put in the list of the most popular auto-machine learning frameworks on GitHub, and the popular AutoML framework has not looked back since then. The TPOT AutoML framework uses genetic programming to zero on a model for task implementation. TPOT AutoML framework can analyse thousands of pipelines to offer one with the best Python code.\n\nTPOT comes with its own regression and classification algorithms. However, its disadvantages include the inability to interact with categorical lines and natural language.","metadata":{}},{"cell_type":"code","source":"# Importing libraries \nfrom tpot import TPOTClassifier\nfrom sklearn.datasets import load_iris\nfrom sklearn.model_selection import train_test_split","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Loading data\niris = load_iris()\niris.data[0:5], iris.target","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Splitting data into training and test set\nX_train, X_test, y_train, y_test = train_test_split(iris.data, iris.target,\n                                                    train_size=0.75, test_size=0.25)\nX_train.shape, X_test.shape, y_train.shape, y_test.shape","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"tpot = TPOTClassifier(generations=8, population_size=50, verbosity=2)\ntpot.fit(X_train, y_train)\nprint(\"Accuracy is {}%\".format(tpot.score(X_test, y_test)*100))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\nYou can also export the optimized model as output in a .py file. Check the output section to view the file and see the chosen model.\nDue to genetic programming, the resulting model can be different every time you run the model","metadata":{}},{"cell_type":"code","source":"tpot.export('tpot_iris_pipeline.py')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n\n> # Example on Pycaret\n\n---","metadata":{}},{"cell_type":"markdown","source":"---\n![](https://cdn.analyticsvidhya.com/wp-content/uploads/2020/05/PyCaret-.jpg)\n\n---\n\nPyCaret is an open source machine learning library in Python to train and deploy supervised and unsupervised machine learning models in a low-code environment. It is known for its ease of use and efficiency.\n\nIn comparison with the other open source machine learning libraries, PyCaret is an alternate low-code library that can be used to replace hundreds of lines of code with a few words only.","metadata":{}},{"cell_type":"code","source":"!pip install pycaret","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from pycaret.datasets import get_data\ndataset = get_data('credit')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data = dataset.sample(frac=0.95, random_state=786)\ndata_unseen = dataset.drop(data.index)\ndata.reset_index(inplace=True, drop=True)\ndata_unseen.reset_index(inplace=True, drop=True)\nprint('Data for Modeling: ' + str(data.shape))\nprint('Unseen Data For Predictions: ' + str(data_unseen.shape))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from pycaret.classification import *","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"exp_clf101 = setup(data = data, target = 'default', session_id=123)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"best_model = compare_models()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(best_model)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"models()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dt = create_model('dt')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(dt)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"knn = create_model('knn')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"rf = create_model('rf')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"tuned_dt = tune_model(dt)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(tuned_dt)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import numpy as np\ntuned_knn = tune_model(knn, custom_grid = {'n_neighbors' : np.arange(0,50,1)})","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(tuned_knn)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"tuned_rf = tune_model(rf)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_model(tuned_rf, plot = 'auc')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_model(tuned_rf, plot = 'pr')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_model(tuned_rf, plot='feature')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_model(tuned_rf, plot = 'confusion_matrix')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"evaluate_model(tuned_rf)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"predict_model(tuned_rf);","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"final_rf = finalize_model(tuned_rf)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(final_rf)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"predict_model(final_rf);","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"unseen_predictions = predict_model(final_rf, data=data_unseen)\nunseen_predictions.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from pycaret.utils import check_metric\ncheck_metric(unseen_predictions.default, unseen_predictions.Label, 'Accuracy')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"save_model(final_rf,'Final RF Model 08Feb2020')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"saved_final_rf = load_model('Final RF Model 08Feb2020')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"new_prediction = predict_model(saved_final_rf, data=data_unseen)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"new_prediction.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from pycaret.utils import check_metric\ncheck_metric(new_prediction.default, new_prediction.Label, 'Accuracy')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> ### This one is example of Classification, To know about other methods, go to this [link](https://pycaret.org/tutorial/)","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"![](https://www.ibmx.website/wp-content/uploads/2019/10/evolution-chart-02.jpg)","metadata":{}},{"cell_type":"markdown","source":"# References: \n1. https://www.kaggle.com/yufengg/automl-getting-started-notebook\n2. https://docs.microsoft.com/en-us/azure/machine-learning/concept-automated-ml\n3. https://automl.github.io/auto-sklearn/master/\n4. https://www.kaggle.com/axelderomblay/running-mlbox-auto-ml-package-on-titanic\n5. https://www.kaggle.com/cedriclacrambe/autokeras-emnist\n6. https://www.kaggle.com/thebrownviking20/tpot-a-great-tool-to-automate-your-ml-workflow/data?\n7. https://pycaret.org/","metadata":{}},{"cell_type":"markdown","source":"# If you find this notebook useful leave an upvote to keep it in your favourites. \nAlso, If there is any error or any thing wrong written, please let me know! Thanks in Advance. ","metadata":{}},{"cell_type":"markdown","source":"![](https://www.charitydynamics.com/wp-content/uploads/2017/04/AdobeStock_66910958.jpeg)","metadata":{}}]}