{"cells":[{"cell_type":"markdown","metadata":{"_cell_guid":"6573f5e4-0eee-b1eb-e1e8-9555c67e5d4b"},"source":"## Kaggle - Bosch\n\n#### What we know:\n\nData represens measurements and their timestamps on products as they move through production. At the end quality coontrol accepts or rejects them. The data is anonymized, has many features, is sparse and is ~8gb in csv. \n\n#### What we want to learn through exploration:\n\n1) Are there many different products or just one? Is the data sparse because measurements are missing, or for a given product are all measurements performed. With multiple products do we need different models? To what extent does the anonymization of features effect our ability to analyze these?\n\n2) How can we use the timestamps? For a given product, what is the distribution of the lags between each step. Are many tests performed at once, or one at a time after each production step? Do outliers, very short or very long lags, predict failure?\n\n3) It would be good to visualize the measurements and lags over the course of production. For a given product, see the distribution of measurements and lags at each step. Flagging of outliers and QC failures in the training set."},{"cell_type":"markdown","metadata":{"_cell_guid":"bb370211-d630-9b87-9cbd-d71825994088"},"source":"#### Inital loading and exploration:"},{"cell_type":"code","execution_count":null,"metadata":{"_cell_guid":"52ad0ff6-9215-c760-2a3c-1c65a1f66e1f"},"outputs":[],"source":"import pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n\n%matplotlib inline"},{"cell_type":"code","execution_count":null,"metadata":{"_cell_guid":"3a17fc01-3fe2-9aff-071f-06f4c9b1f441"},"outputs":[],"source":"sample_submission = pd.read_csv(\"../input/sample_submission.csv\")\nsample_submission # ~1.2 million rows (ids of individual products)"},{"cell_type":"markdown","metadata":{"_cell_guid":"dd882679-428c-60ed-cac7-83f9a71eaffe"},"source":"*** \n\n### How imbalanced are the classes?"},{"cell_type":"code","execution_count":null,"metadata":{"_cell_guid":"8586917d-c466-aa2b-d5df-5db51e63653b"},"outputs":[],"source":"train_response = pd.read_csv(\"../input/train_numeric.csv\", usecols=['Response'])\ntrain_response.sum() / train_response.count()\n"},{"cell_type":"markdown","metadata":{"_cell_guid":"765b65f7-831f-9adb-207f-7df3690cc94d"},"source":" 0.6% failure\n--- \n\n### Do the lines (L0, L1, ...) appear to represent different products?\n\n#### The feature names increment stations across lines, but it is not clear that represents a product moves across lines...`enter code here`"},{"cell_type":"code","execution_count":null,"metadata":{"_cell_guid":"35aeef7a-3548-bd17-8966-a73109bd18f5"},"outputs":[],"source":""}],"metadata":{"_change_revision":0,"_is_fork":false,"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.5.2"}},"nbformat":4,"nbformat_minor":0}