{"cells":[{"metadata":{},"cell_type":"markdown","source":"# CONVERT CSV 2 SHP + RASTER 101 in R\n\nThis short notebook might be useful for you if:\n* ...you want to work with the factories csv as a shapefile - in this case you can use my code below to export the csv as a shp via R\n* ...you want to learn basic raster manipulation in R such as: reading in tif files, etxtracting and stacking bands, plotting rasters and raster values (time series)\n\n\nLet's start and ...\n\nRead in the data into a dataframe and parse the coordinates (lat, lon) into separate columns"},{"metadata":{},"cell_type":"markdown","source":"# Read In Factories csv"},{"metadata":{"trusted":true},"cell_type":"code","source":"library(raster)\nlibrary(tidyverse)\n\nshp_root <- \"../input/ds4g-environmental-insights-explorer/eie_data/gppd/gppd_120_pr.csv\"\nshp_csv  <- read_csv(shp_root)\n\nshp_csv <- shp_csv %>% rowwise() %>% #extract coordinates\n  mutate(lon=strsplit(gsub(x=gsub(x=.geo, pattern='.*\\\\[', replacement=\"\"),pattern=\"]}\", replacement=\"\"),\",\")[[1]][1]) %>% \n  mutate(lat=strsplit(gsub(x=gsub(x=.geo, pattern='.*\\\\[', replacement=\"\"),pattern=\"]}\", replacement=\"\"),\",\")[[1]][2])\n\nhead(shp_csv)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Convert the dataframe into a SpatialPointsDataframe (spdf) and export the spdf into a shapefile (.shp)"},{"metadata":{"_uuid":"051d70d956493feee0c6d64651c6a088724dca2a","_execution_state":"idle","trusted":true},"cell_type":"code","source":"\n# make the spatial points data frame object\nspdf <- sp::SpatialPointsDataFrame(coords = data.frame(lon=as.numeric(shp_csv$lon), lat=as.numeric(shp_csv$lat)),\n                               data = shp_csv %>% select(-lat,-lon), \n                               proj4string = CRS(\"+init=epsg:4326\"))\n\nplot(spdf)\nhead(spdf)\n\n#export to shp\nshapefile(spdf, \"../output/kaggle/working/gppd_120_pr.shp\", overwrite=T)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Read in NO2 Sentinel-5 Data"},{"metadata":{"trusted":true},"cell_type":"code","source":"no2_root   <- \"../input/ds4g-environmental-insights-explorer/eie_data/s5p_no2/\"\nno2_files  <- list.files(no2_root, full.names=T, pattern=\".tif$\")\nhead(no2_files)\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's have a look at the metadata of the first image..."},{"metadata":{"trusted":true},"cell_type":"code","source":"raster::stack(no2_files[1])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"When we look at dimension, we can see that the image has 12 bands with the following names:\ns5p_no2_2//7T175356.1, s5p_no2_2//7T175356.2, s5p_no2_2//7T175356.3, s5p_no2_2//7T175356.4, s5p_no2_2//7T175356.5, s5p_no2_2//7T175356.6, s5p_no2_2//7T175356.7, s5p_no2_2//7T175356.8, s5p_no2_2//7T175356.9, s5p_no2_2//T175356.10, s5p_no2_2//T175356.11, s5p_no2_2//T175356.12 \n\nFor an explanation of the band designations have a look here:\nhttps://developers.google.com/earth-engine/datasets/catalog/COPERNICUS_S5P_OFFL_L3_NO2\n\n\nFor NO2 the first four bands are relevant.\nBand 1 is the NO2 column number density, this will very likely be the most useful band. For more information also have a look here:\nhttps://earth.esa.int/web/sentinel/technical-guides/sentinel-5p/level-2/doas-method\n"},{"metadata":{},"cell_type":"markdown","source":"Lets loop through the images and only extract the first band:"},{"metadata":{"trusted":true},"cell_type":"code","source":"\n#No2 Density\nno2_density_list         <- list()\n\nfor(i in 1:length(no2_files)){\n  no2_density_list         <- append(no2_density_list,raster::stack(no2_files[i])[[1]])\n  \n}","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's stack the first 50 images and have a look at their values \n\n(**Note:** I tried to stack all images, but it seems that the images (from 50 onwards) don't have the same extent anymore. This causes error messages during the stacking process. If you want to stack all images in R, you first need to make a virtual raster stack out of the images - this can be done with gdal (https://gdal.org/programs/gdalbuildvrt.html) )"},{"metadata":{"trusted":true},"cell_type":"code","source":"\nno2_density_stack <- raster::stack(no2_density_list[1:50])\nvalues            <- raster::extract(no2_density_stack, spdf)\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Plot the first NO2 band"},{"metadata":{"trusted":true},"cell_type":"code","source":"plot(no2_density_stack[[1]])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Plot first 50 NO2 values for first factory"},{"metadata":{"trusted":true},"cell_type":"code","source":" plot(x= 1:nrow(values), y=values[,1], col=\"red\", type=\"l\", main=spdf@data[1,\"name\"])","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"R","language":"R","name":"ir"},"language_info":{"mimetype":"text/x-r-source","name":"R","pygments_lexer":"r","version":"3.4.2","file_extension":".r","codemirror_mode":"r"}},"nbformat":4,"nbformat_minor":1}