{"metadata": {"kernelspec": {"name": "python3", "language": "python", "display_name": "Python 3"}, "language_info": {"name": "python", "mimetype": "text/x-python", "pygments_lexer": "ipython3", "file_extension": ".py", "version": "3.6.4", "nbconvert_exporter": "python", "codemirror_mode": {"name": "ipython", "version": 3}}}, "cells": [{"source": ["As it was disscussed in [this topic](https://www.kaggle.com/c/data-science-bowl-2018/discussion/47640) dataset contains images from 3 modalities (microscopy types). Let's separate them in train set."], "cell_type": "markdown", "metadata": {"_cell_guid": "bfa45ba0-9a5b-4fee-b17f-c0a0d71c0a99", "_uuid": "e5f630fb6c5a4fdff21f81bc7a46e1981deffcf4"}}, {"source": ["import os\n", "import glob\n", "import shutil\n", "import numpy as np\n", "import pandas as pd\n", "import matplotlib.pyplot as plt\n", "%matplotlib inline\n", "import cv2"], "cell_type": "code", "execution_count": null, "outputs": [], "metadata": {"_cell_guid": "5b898c83-d67d-4242-99fd-a83d35674389", "collapsed": true, "_uuid": "7fb96014d85acfe718dec3dc04aae1a28544c550"}}, {"source": ["train_path = \"../input/stage1_train/\"\n", "file_paths = glob.glob(train_path + \"*/images/*.png\")"], "cell_type": "code", "execution_count": null, "outputs": [], "metadata": {"_cell_guid": "5224c963-0493-44a1-b997-1259043a60d4", "collapsed": true, "_uuid": "9389d86170de2a4a064a287391a33532db28504a"}}, {"source": ["Not sure about corectness of names but this ones will be used. "], "cell_type": "markdown", "metadata": {"_cell_guid": "3a1054b8-6a42-4fc1-bbf8-377d4473fd2c", "_uuid": "26b52946cc52ef455c90a96457ca0a6fbe19fb03"}}, {"source": ["# 0 - staining;\n", "# 1 - fluorescent microscopy;\n", "# 2 - brightfield microscopy;\n", "type_names = {0: \"staining\", 1: \"fluorescent\", 2: \"brightfield\"}"], "cell_type": "code", "execution_count": null, "outputs": [], "metadata": {"_cell_guid": "19b1f697-1ab3-487d-bea4-467ea74987c1", "collapsed": true, "_uuid": "6daa407bf45403a04964e3d4d0f4753dc1da1e4f"}}, {"source": ["It's ease to see that staining contain more violet pixels. Naive condition that define violet pixel is: red_channel > green_channel AND blue_channel > green_channel."], "cell_type": "markdown", "metadata": {"_cell_guid": "239399f3-a737-4682-9de7-374396f81664", "_uuid": "57a94eb341e1daa83e266752e0538c6460f6c00f"}}, {"source": ["def get_violet_num(img):\n", "    violet_num = 0\n", "    h, w = img.shape[:2]\n", "    for y in range(h):\n", "        for x in range(w):\n", "            if img[y, x][0] > img[y, x][1] and img[y, x][2] > img[y, x][1]:\n", "                violet_num += 1\n", "\n", "    return violet_num"], "cell_type": "code", "execution_count": null, "outputs": [], "metadata": {"_cell_guid": "cfdcf5e8-50f6-416a-a160-d3a7645263c2", "collapsed": true, "_uuid": "ed509f94bdee9cd3c4848e167cad3b30e72603ec"}}, {"source": ["# staining example\n", "img_id = '00ae65c1c6631ae6f2be1a449902976e6eb8483bf6b0740d00530220832c6d3e'\n", "img_path = os.path.join(train_path, img_id, \"images\", img_id + \".png\")\n", "stain_img = cv2.imread(img_path)\n", "plt.imshow(stain_img)\n", "plt.show()"], "cell_type": "code", "execution_count": null, "outputs": [], "metadata": {"_cell_guid": "b255b458-b8a9-4cf9-82fc-910b0ef5efe9", "collapsed": true, "_uuid": "99539240b94d6f6444ae558eb6cf258658522d57"}}, {"source": ["mean_int = stain_img.mean()\n", "print(mean_int)"], "cell_type": "code", "execution_count": null, "outputs": [], "metadata": {"_cell_guid": "d1237199-54bd-44d1-bf9f-e1a946e1845b", "collapsed": true, "_uuid": "8c4fe05f2d73c4ad8c9a7a1893d661e61fbcbb0b"}}, {"source": ["print(get_violet_num(stain_img))"], "cell_type": "code", "execution_count": null, "outputs": [], "metadata": {"_cell_guid": "8431d686-938b-492e-ae97-e525c05fab63", "collapsed": true, "_uuid": "d5d2e020438579b20888273ee3e9e019dfc2a319"}}, {"source": ["Most of pixels are violet."], "cell_type": "markdown", "metadata": {"_cell_guid": "a13de95a-fcb2-4273-85bf-9463f576fc6f", "_uuid": "2f583830580baa7ce01bdd8c5f5647e2f9fd7765"}}, {"source": ["print(stain_img[0,0])"], "cell_type": "code", "execution_count": null, "outputs": [], "metadata": {"_cell_guid": "d25161f1-1ed8-4133-86e2-8bea06fe3942", "collapsed": true, "_uuid": "555ceb68baffd8ab77dc8e0bfae7f789a3fa195a"}}, {"source": ["# fluorescent example\n", "img_id = '0acd2c223d300ea55d0546797713851e818e5c697d073b7f4091b96ce0f3d2fe'\n", "img_path = os.path.join(train_path, img_id, \"images\", img_id + \".png\")\n", "fluo_img = cv2.imread(img_path)\n", "plt.imshow(fluo_img)\n", "plt.show()"], "cell_type": "code", "execution_count": null, "outputs": [], "metadata": {"_cell_guid": "120329de-98e7-4a2c-8f71-2651e791b7fa", "collapsed": true, "_uuid": "f236dcca67f359a1b4a24a8cf9e883654f533326"}}, {"source": ["mean_int = fluo_img.mean()\n", "print(mean_int)"], "cell_type": "code", "execution_count": null, "outputs": [], "metadata": {"_cell_guid": "16ed2a2e-33fc-4c7d-b98e-6c443686a216", "collapsed": true, "_uuid": "05bb6c06f527677dd41afdaf3de5b18fc977d084"}}, {"source": ["Mean intensity can be used for distinquish fluorescent and brightfield images. For fluorescent it's quite low because most of image is black background."], "cell_type": "markdown", "metadata": {"_cell_guid": "4c86828e-ba54-4ff4-8788-8f47403cb975", "_uuid": "e2247c0a065d359a30ad194a2587a22b3fa96e91"}}, {"source": ["print(get_violet_num(fluo_img))"], "cell_type": "code", "execution_count": null, "outputs": [], "metadata": {"_cell_guid": "8a3b3436-2373-48c6-9084-fa6d60fb6f2f", "collapsed": true, "_uuid": "e0518da0f80c1713232ffd29dba528362767a8e2"}}, {"source": ["Any violet pixels. Actually, in grayscale images (fluorescent and brightfield) each pixel has equal values for 3 channels."], "cell_type": "markdown", "metadata": {"_cell_guid": "fa32eb99-3de9-4c7f-ba64-bfc2a95cd503", "_uuid": "572df5311933feaaa71a16e92afc00bd34d7bdbc"}}, {"source": ["# brightfield example\n", "img_id = '54793624413c7d0e048173f7aeee85de3277f7e8d47c82e0a854fe43e879cd12'\n", "img_path = os.path.join(train_path, img_id, \"images\", img_id + \".png\")\n", "bright_img = cv2.imread(img_path)\n", "plt.imshow(bright_img)\n", "plt.show()"], "cell_type": "code", "execution_count": null, "outputs": [], "metadata": {"_cell_guid": "61eb45a2-12f7-40be-aaf0-dc02f4372734", "collapsed": true, "_uuid": "37513a8367874c18ec2da8cd4b7bf929a1bb79af"}}, {"source": ["mean_int = bright_img.mean()\n", "print(mean_int)"], "cell_type": "code", "execution_count": null, "outputs": [], "metadata": {"_cell_guid": "a974fb58-5291-4b02-b8b1-0fb8824d8c26", "collapsed": true, "_uuid": "684f191ce7f2ee99c6c5b37ffc6094170ff72692"}}, {"source": ["As expected, mean intensity is much higher than for fluorescent images."], "cell_type": "markdown", "metadata": {"_cell_guid": "6459d736-6053-4c36-a70a-b8bd07d30870", "_uuid": "ea2b1e489e281ddc0434e054f142381e392c5d1e"}}, {"source": ["print(get_violet_num(fluo_img))"], "cell_type": "code", "execution_count": null, "outputs": [], "metadata": {"_cell_guid": "0daa2255-54cb-464f-b23a-05ad2b75d7d9", "collapsed": true, "_uuid": "cbbdf8282712c7944b2d2541dc3bf8897d5021e1"}}, {"source": ["def get_microscopy_type(img):\n", "    violet_num = get_violet_num(img)\n", "    if violet_num > 0:\n", "        return 0\n", "    mean_int = img.mean()\n", "    # randomly picked threshold for distinquishing fluorescent and brightfield\n", "    if mean_int > 100:\n", "        return 2\n", "    return 1"], "cell_type": "code", "execution_count": null, "outputs": [], "metadata": {"_cell_guid": "d633e7e9-f3ff-418a-961f-88818f83e2a4", "collapsed": true, "_uuid": "4de2c9f0e32837e95062ad5271fc9696d39a6f1c"}}, {"source": ["clusters_path = \"../input/clusters\"\n", "# if not os.path.isdir(clusters_path):\n", "#     os.mkdir(clusters_path)\n", "\n", "# for type_name in type_names.values():\n", "#     if not os.path.isdir(os.path.join(clusters_path, type_name)):\n", "#         os.mkdir(os.path.join(clusters_path, type_name))"], "cell_type": "code", "execution_count": null, "outputs": [], "metadata": {"_cell_guid": "6d9bd33f-6040-415c-b272-241774284144", "collapsed": true, "_uuid": "03df99e583e766eab278c2241692edce028d3e9b"}}, {"source": ["types_list = []\n", "for file_path in file_paths:\n", "    img = cv2.imread(file_path)\n", "    img_type = type_names[get_microscopy_type(img)]\n", "    types_list.append(get_microscopy_type(img))\n", "    #uncomment for copy to separate folder \n", "    #shutil.copy(file_path, os.path.join(clusters_path, img_type))"], "cell_type": "code", "execution_count": null, "outputs": [], "metadata": {"_cell_guid": "db7bb10d-0925-4a90-917d-8afea4e7bba9", "collapsed": true, "_uuid": "b1d23c2220bccce537a47bf039493ba5e8d8e85a"}}, {"source": ["# sanity check\n", "rand_idx = np.random.randint(len(types_list))\n", "print(rand_idx)\n", "print(type_names[types_list[rand_idx]])\n", "img = cv2.imread(file_paths[rand_idx])\n", "plt.imshow(img)\n", "plt.show()"], "cell_type": "code", "execution_count": null, "outputs": [], "metadata": {"_cell_guid": "3f0a9ff2-9b16-435c-b4ef-bd2992b1580e", "collapsed": true, "_uuid": "262c1810ed37a72affdd421cb17aaec2e9dc4617"}}], "nbformat_minor": 1, "nbformat": 4}