Instead of storing this in a CSV, I stored this as a pickled numpy array file inside the dictionary. Just to make your life a little bit easier. 
Take a look at the sample kernel to see how to get started.
Inside the dictionary, there are the images (2D ndarray) and the labels(also a numpy array but 1D this time). The images are under the "images" label and the labels under the "labels" data. Not difficult.