# 7th Place short overview.

Hello guys, here we provide a first, quick overview over the 7th place solution. We will follow that up with a more detailed write-up, this just serves as a short methodological overview. Congrats to the top performing teams!
DL package
: We implemented all networks directly in tensorflow on ubuntu 14.04 machines.
Approach
: We solely relied on the nodule annotations of LUNA/LIDC and the dsb cancer labels for building our framework. The general approach can be divided into 3 main steps, namely:
1) Segmenting nodules:
We first trained a segmentation network (standard UNET architecture) with the LUNA/LIDC information to segment lungs nodules. The network was trained on random 128x128 crops of lung slices which were sampled randomly from all 3 room axis (not only axial) with the corresponding nodule annotation mask (we used elliptical annotations). We just sampled slices which section a nodule annotation. We treated every sampled slice as an independent data record. At prediction time we apply this network to all slices of a CT Scan
in all 3 room axis
. This leaves us with three 3D probability map tensors, which we simply average. This results in a single 3D probabilty map tensor for each lung CT scan.
2) Find nodule candidates:
We use an iterative cluster algorithm to find clusters in the 3D probability clouds from step 1). The main idea here was to threshold the 3D probability maps from the segmentation network with a low value (0.2) and run a clustering. This will result in probability cloud clusters of arbitrary size. If the size of a cluster exceeds the plausible nodule size, we further increase the threshold in this cloud and again run the clustering to detect single or multiple subclusters. We repeat this process until all probability cloud cluster is "condensed" to single clusters of a plausible nodule size. The center of these final clusters is used to extract a 64x64x64 region from this location from the lung. This will result in <100 candidates for each lung. We sort the candidate tensors with the sum of the probability contained in the 64x64x64 crop and
only use the first 20 candidates
for the final step.
2.1) (optional / we wish we had) At this step it would be straight forward to apply an malignancy classification network to each of the extracted candidates and obtain a more meaningful candidate set for the final classifier. Unfortunately, we did not notice the malignancy information contained in the LIDC dataset. There is a good chance for improvement here.
3) Predict cancer on patient level
: From step 2)
we obtain 20 64x64x64 candidates for each patient
in the dsb dataset. We use those candidates to train a 3D resnet for cancer classification. We do not only use the pixel information from the lung CT scan but also the probability map information from the segmentation network as a second input channel. The intuition behind including the probability map from the segmentation network is that the segmentation is trained on larger patches (128x128) and has, therefore, more context information than the stage 2 classifier. By including the probability map the stage 2 classifier is able to reuse the extracted context information of the segmentation network, which is contained in the predicted segmentation maps.
We
feed the 20 candidates simultaneously
in the network. This means a single input record has the dimension (20,64,64,64,2), where the 1. dimension stands for the 20 candidates, 2nd-4th are the spatial dimensions, and the 5th dimension encodes the two channels (lung CT pixel information, 3D probability output from the segmentation net in stage1). We assign the dsb patient label to this 5D tensor, resulting in the following
datarecord-label-pair: (20,64,64,64,2) <-> cancer label
.
The network architecture is a 3D convolution network designed in a way that it reuses its weights in the candidate dimension. We did that by simply reshaping the candidate dimension into the batch dimension while training/predicting: (Batchsize,20,64,64,64,2) -> (Batchsize x 20,64,64,64,2). We then apply a resnet 3D network, which convolves the 3 spatial dimensions and reduces the input tensor with shape (Batchsize x 20,64,64,64,2) to an output tensor with dimension (Batchsize x 20, 1). On this tensor, we apply a sigmoid function the normalize the values. We reshape this tensor back to (Batchsize, 20) and
apply a reduction function
to arrive at a single probability for each patient. We experimented with a few different reduction functions, for example, Noisy-AND/Noisy-OR, but in practice, a simple maximum function performed the best. The max(tensor, axis=1)-function reduces the (Batchsize, 20)-tensor to an (Batchsize, 1)-tensor. The intuition behind reducing the 20 candidate probabilities with the maximum function to a single patient probability is, that the most "cancerous" nodule also determines the cancer likelihood of the whole patient. This final output scalar will be the prediction for a patient. This setup is directly end-to-end trainable from X candidates per patient to a single prediction per patient.
Additional notes on the multicandidate classifier
: It is important to use a reduction function which is
invariant to permutation of the candidates
(Max, Average, Noisy-AND, ...) since their ordering is expected to be imperfect due to the fact that the segmentation network which produces the candidates is trained to predict
all
nodules, not only malignant ones. So when e.g. using an MLP to reduce the (Batchsize, 20) tensor to a single (Batchsize, 1) output, the classifier can actually fail when permutating the 20 candidates, since a individual weight is used for every candidate. A simple example with 5 candidates, where only one correlates with the actual patient label: A (0, cancer, 0, 0, 0) input should produce the same prediction as a (cancer, 0, 0, 0, 0) input. When having a relatively small dataset where it's necessary to combine multiple equivalent candidates one can greatly reduce overfitting effects by making the classifier invariant to those candidate permutations.
General Model Optimization:
We did spend nearly no time optimizing the network architectures or optimizer parameters, since we simply ran out of time and the runs are lengthy.