{"cells":[{"metadata":{},"cell_type":"markdown","source":"* # For R users, who love to do in local desktops or AWS free account, don't worry of \"test file\" size of this competition\n* # Follow below steps for efficient reading and loading data for all your future analysis\n* # Use data.table \"fread\" to load data\n* # Save in \"fst\" format for all future uses\n* # Below e.g. with respect to \"test.csv\" which is about 1.3 gb of size\n* # When we convert it to fst format and load it again\n* # You can observe both gain in size (compressed to 103 MB) and\n* # Gain in loading the same file (few seconds)"},{"metadata":{"trusted":true,"_kg_hide-output":true},"cell_type":"code","source":"# don't run below code here\n# run all in your local or server\n# reading using data.table \"fread\"\n# set your working directory first as \"setwd(~/..../...)\" point to the file in the direcroy\nsystem.time(testdt <- data.table::fread(\"test.csv\"))\n# saving it to fst\nfst::write.fst(testdt, \"testdt.fst\",100)\n# delete testdt\nrm(testdt)\n\n# re-loading it to R\nsystem.time(fst::read.fst(\"testdt.fst\"))","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"R","language":"R","name":"ir"},"language_info":{"mimetype":"text/x-r-source","name":"R","pygments_lexer":"r","version":"3.4.2","file_extension":".r","codemirror_mode":"r"}},"nbformat":4,"nbformat_minor":1}