{"metadata": {"kernelspec": {"display_name": "R", "name": "ir", "language": "R"}, "language_info": {"version": "3.4.2", "pygments_lexer": "r", "mimetype": "text/x-r-source", "name": "R", "file_extension": ".r", "codemirror_mode": "r"}}, "nbformat_minor": 1, "nbformat": 4, "cells": [{"metadata": {"_cell_guid": "33508444-2b30-4653-9d77-f34f95c406d6", "_uuid": "a38f0a1557841279de99073a1720039049a34983"}, "cell_type": "markdown", "source": ["# Exploratory Analysis\n", "\n"]}, {"metadata": {"_cell_guid": "f7124d20-3c5e-4079-aa19-24de649e30cc", "_uuid": "70d0732da182085181702f9b772cf61a5cd80d92"}, "outputs": [], "execution_count": null, "cell_type": "code", "source": ["suppressPackageStartupMessages({\n", "    library(tidyverse)\n", "    library(EBImage)\n", "    library(gridExtra)\n", "    library(RGraphics)\n", "    library(magrittr)\n", "    library(skimr)\n", "})\n", "set.seed(5)"]}, {"metadata": {"_cell_guid": "26a7587b-bf39-4829-806a-74a37cb3a5c4", "_uuid": "a229b32388c9f28efc86236817eb7ee0496b947d"}, "cell_type": "markdown", "source": ["Lets begin by reading in the labels CSV which will give us the image name"]}, {"metadata": {"_cell_guid": "a24e4cbe-98f5-48d5-912e-d88bc7479922", "_uuid": "7de919c750ea4f6370a9aab80fbd88e1fd513f7b"}, "outputs": [], "execution_count": null, "cell_type": "code", "source": ["labels_df = suppressMessages(read_csv(\"../input/stage1_train_labels.csv\"))"]}, {"metadata": {"_cell_guid": "799f0f34-6d50-428c-a637-ee023e39df88", "_uuid": "52505635e21ba723f7a608c74d968e61c78dc1b9"}, "cell_type": "markdown", "source": ["Take a quick peak at the image data using the `EBImage::readImage` function."]}, {"metadata": {"_cell_guid": "353dd70d-4a92-4ed7-840b-1622322dfb6f", "_uuid": "ee61d8b0fbaa68f3d2df2b6a13c54e9dac14ab82"}, "outputs": [], "execution_count": null, "cell_type": "code", "source": ["firstImage = labels_df$ImageId[[1]]\n", "f = sprintf(\"../input/stage1_train/%s/images/%s.png\", firstImage, firstImage)\n", "img = readImage(f)\n", "img"]}, {"metadata": {"_cell_guid": "e24acc8d-1810-4817-aee9-dada4775f9dc", "_uuid": "ac441341261e1ef4c18e92de89218870037e051a"}, "cell_type": "markdown", "source": ["Seems like the way readImage reads there's an additional channel, (total 4). Anyway lets print it out to give a look."]}, {"metadata": {"_cell_guid": "98adfea6-557b-401d-bb1c-d31dd5ef50c7", "_uuid": "8410d7063dab7b874b6c694cf815f634995f7080"}, "outputs": [], "execution_count": null, "cell_type": "code", "source": ["options(EBImage.display = \"raster\")\n", "display(img)"]}, {"metadata": {"_cell_guid": "357bed3e-9450-4a0f-9a97-05aa56d7bb4c", "_uuid": "01a3b4a103bac29ce89fc9fbcacfd1b758d20e72"}, "cell_type": "markdown", "source": ["Some summary stats for this image and well as its range"]}, {"metadata": {"_cell_guid": "b01e4448-96e2-4210-a8da-2f3d2f136b48", "_uuid": "a0894644ab9ac86b096b0c8d3ff30e998bf092a2"}, "outputs": [], "execution_count": null, "cell_type": "code", "source": ["hist(img)\n", "rangeVal = range(img)\n", "message(sprintf(\"Ranges from %s to %s\", rangeVal[[1]] , rangeVal[[2]]))"]}, {"metadata": {"_cell_guid": "787f5764-b3b1-4b88-8d90-9e0b0d9298b1", "_uuid": "938012b68795c23b85f42f0a59ec47f865ebb235"}, "cell_type": "markdown", "source": ["Lets see its dimensions."]}, {"metadata": {"_cell_guid": "a248bcbb-e046-4e3c-b1e0-e9c36da272b8", "_uuid": "6ba4fd1a58e70c2e47714bd18b1a83d1ba3bb5f2"}, "outputs": [], "execution_count": null, "cell_type": "code", "source": ["setNames(as.tibble(matrix(dim(img), nrow=1)), c(\"x\", \"y\", \"channels\"))"]}, {"metadata": {"_cell_guid": "1efe7787-e0e3-42a0-8ded-5d3caf8ecc52", "_uuid": "47f966f8a526838d97611576dfad09deaeb20242"}, "cell_type": "markdown", "source": ["Cause reading in all the images is going to take awhile, lets sample a bit more images, to be exact 10% of the images. "]}, {"metadata": {"_cell_guid": "bc1286ed-6866-46c7-95ef-64848c5ca944", "_uuid": "9d44217f3f3a415057a4f8376825819190d84e40"}, "outputs": [], "execution_count": null, "cell_type": "code", "source": ["sampleIndex = sample(1:nrow(labels_df), round(0.1*nrow(labels_df)))\n", "labels_df$ImageId[sampleIndex] %>%\n", "  map_df(~ as.tibble(matrix(dim(readImage(sprintf(\"../input/stage1_train/%s/images/%s.png\", ., .))), nrow=1))) %>%\n", "  unique %>% setNames(c(\"X\", \"Y\", \"channel\")) %>% select(-channel)"]}, {"metadata": {"_cell_guid": "dcc6ff5f-5276-4b3c-be69-dfb907fda9d0", "_uuid": "be23b42b0a87dd22f79456ef8e13738247f0e988"}, "cell_type": "markdown", "source": ["There seems to be some variation in the data with regards to the image dimensions.. Now lets see what the mean intensity is for the sampled 10%"]}, {"metadata": {"_cell_guid": "2fc59698-3c6f-4a5a-b114-c866118dd3d4", "_uuid": "844545aa161c3747c8cb245ca0f629b8b1e363ad"}, "outputs": [], "execution_count": null, "cell_type": "code", "source": ["labels_df$ImageId[sampleIndex] %>%\n", "  map(~ mean(imageData(readImage(sprintf(\"../input/stage1_train/%s/images/%s.png\", ., .)))[,,1:3]))  %>% do.call(c,.) %>% \n", "  hist"]}, {"metadata": {"_cell_guid": "f85fc543-a9f3-42a0-98a9-a4120d028ca5", "_uuid": "21b36bddd41b7bc5b4b82e70d73fff7b1fa8d5cf"}, "outputs": [], "execution_count": null, "cell_type": "code", "source": ["#helper function\n", "\n", "ebimageGrob <-\n", "function (pic, x=0.5, y=0.5, scale=1, raster=FALSE, angle = NULL, ...)\n", "{\n", "  dims <- dim(pic)\n", "  colours = t(channel(pic, \"x11\"))\n", "  width = unit(scale*dims[1], \"points\")\n", "  height = unit(scale*dims[2], \"points\")\n", "  angle <- if(is.null(angle)) 0 else angle\n", "  vp <- viewport(x=x, y=y, width=width, height=height, angle=angle)\n", "  if(raster){\n", "    child <-\n", "      rasterGrob(colours, vp=vp, ...)\n", "  } else {\n", "  colours <- colours[ rev(seq_len(nrow(colours))), ]\n", "    require(RGraphics)\n", "    child <-\n", "      imageGrob(dims[2], dims[1], col=colours, gp=gpar(col=colours), by=FALSE, vp=vp, ...)\n", "  }\n", "  gTree(width= width[[1]],\n", "        height = height[[1]],\n", "        children=gList(child), cl=\"ebimage\")\n", "}\n", "masks = paste(sprintf(\"../input/stage1_train/%s/masks/\", firstImage), \n", "              list.files(sprintf(\"../input/stage1_train/%s/masks/\", firstImage)), sep=\"/\") %>% \n", "map(~ ebimageGrob(readImage(.), scale=0.2))\n", "grid.arrange(grobs = masks, ncol=5)"]}, {"metadata": {"_cell_guid": "1bf1a26b-27c5-4aa2-ba24-f0aa467b0079", "_uuid": "5d9464c0aba29e1a925103ff9e35437d3580c04e"}, "cell_type": "markdown", "source": ["Interesting it seems to be a bimodal distribution in terms of its mean intensity. Its not entirely out of expectation as the data description did mention there's going to be a mixture of two types of microscopy images\n", "1. brightfield \n", "2. flourescence (ones with the black background like the one we drew out)\n", "\n", "Here's an example of a Brightfield microscopy image\n", "![Brightfield](https://micro.magnet.fsu.edu/primer/anatomy/brightfieldgallery/images/amphiumaliver40xlarge.jpg)\n", "\n", "And below's an example of a flourescence microscopy image\n", "![Fluorescent](https://i.pinimg.com/736x/99/43/7c/99437cf167987fd0e3a73ad289694329--fluorescence-microscopy-mitosis.jpg)"]}, {"metadata": {"_cell_guid": "4b54f6d9-db77-4e48-9fa4-a3bbdea5102f", "_uuid": "92295072624d246587f8dc8fbea87f3034c8dc4c"}, "outputs": [], "execution_count": null, "cell_type": "code", "source": ["meanIntensities = labels_df$ImageId[sampleIndex] %>%\n", "  map(~ mean(imageData(readImage(sprintf(\"../input/stage1_train/%s/images/%s.png\", ., .)))[,,1:3]))  %>% do.call(c,.)\n", "intensityDF = tibble(sample = sampleIndex, intensities = meanIntensities)\n", "intensityDF %<>% mutate(ImageId = labels_df$ImageId[sampleIndex])\n", "intensityDF %>% head"]}, {"metadata": {"_cell_guid": "fbb5572f-efce-4b40-949e-8e1019f873cd", "_uuid": "3bc07e3b3cba562b0b5316218fd847956c45cc67"}, "cell_type": "markdown", "source": ["Here's one example of a brightfield image"]}, {"metadata": {"_cell_guid": "7b194c3f-144f-4f12-8b31-dca2f5710e6f", "_uuid": "548549842f2bbc8f55b12e6e88fdda608cc13f66"}, "outputs": [], "execution_count": null, "cell_type": "code", "source": ["brightField = intensityDF %>% \n", "  filter(intensities > 0.6) %>% \n", "  head(n=1) %>% \n", "  pull(ImageId)\n", "\n", "paste(sprintf(\"../input/stage1_train/%s/images/%s.png\", brightField,brightField)) %>% \n", "  readImage %>% display"]}, {"metadata": {}, "outputs": [], "execution_count": null, "cell_type": "code", "source": ["masks = paste(sprintf(\"../input/stage1_train/%s/masks/\", brightField), \n", "              list.files(sprintf(\"../input/stage1_train/%s/masks/\", brightField)), sep=\"/\") %>% \n", "head(2) %>%\n", "map(~ ebimageGrob(readImage(.), scale=0.2))\n", "grid.arrange(grobs = masks, ncol=5)"]}, {"metadata": {}, "cell_type": "markdown", "source": ["Printing out a few of the brightfield images"]}, {"metadata": {"_cell_guid": "cdef6d33-0f17-47eb-bdda-6fa3bad1d29a", "_uuid": "00d36b918b0a1bf583902ea879af2c3727ae533e"}, "outputs": [], "execution_count": null, "cell_type": "code", "source": ["grobs = intensityDF %>% filter(intensities > 0.6) %>% head(6) %>% pull(ImageId) %>% \n", "  map(~ paste(sprintf(\"../input/stage1_train/%s/images/%s.png\", ., .))) %>% map(~ readImage(.)) %>% map(~ ebimageGrob(., scale=1))\n", "grid.arrange(grobs=grobs, ncol=3)"]}, {"metadata": {}, "cell_type": "markdown", "source": ["Now a few of the flourence microscopy images as well"]}, {"metadata": {"_cell_guid": "266bc3c4-5d3a-4269-8b78-97413d976492", "_uuid": "bb9b1c31e16051eebb974216c8b22b53515fcdc5"}, "outputs": [], "execution_count": null, "cell_type": "code", "source": ["grobs = intensityDF %>% filter(intensities < 0.3) %>% head(6) %>% pull(ImageId) %>% \n", "  map(~ paste(sprintf(\"../input/stage1_train/%s/images/%s.png\", ., .))) %>% map(~ readImage(.)) %>% map(~ ebimageGrob(., scale=1))\n", "grid.arrange(grobs=grobs, ncol=3)"]}, {"metadata": {"_cell_guid": "65b9eaf8-6153-4e89-a677-76da8517c827", "_uuid": "23fcd74cd4d843de8760afa6b259f8dfb1c9996f"}, "cell_type": "markdown", "source": ["Along with the images are the masks, lets visualise the masks for the 1st image"]}, {"metadata": {"_cell_guid": "bfafdae6-0467-46fd-9eeb-0d9f22b9952a", "_uuid": "fffba07665f83a03686677320b2327e07b9c58b3"}, "outputs": [], "execution_count": null, "cell_type": "code", "source": []}, {"metadata": {"_cell_guid": "0d2b73d4-fe35-4e9d-bbe5-536e229a5ee2", "_uuid": "d44446e7a1a0d62b51f2be7f5bb5a5cdb8eb7185"}, "outputs": [], "execution_count": null, "cell_type": "code", "source": ["devtools::session_info()"]}]}