Image submission
For online testing convenience, this competition currently evaluates an
image submission track
:
participants upload their locally rendered test-view images. The scorer compares them against hidden GT images.
Evaluation Protocol
In
Stage 1
, evaluation is performed on the
cam01
and
cam09
views only.
These two views are reserved for evaluation and
must not
be used for training.
All other provided views may be used for training.
In
Stage 2
, participants are required to use only the views specified by the
provided camera parameters
for training. Evaluation will be performed by the organizers using
hidden custom camera parameters
, which are
not publicly released
. The final result will be determined by the
Stage 2 score
.
Metrics
PSNR
(higher is better)
SSIM
(higher is better)
Size
(lower is better)
tOF
(lower is better)
Definitions
PSNR
\( \mathrm{MSE}=\frac{1}{3HW}\sum_{c=1}^{3}\sum_{h=1}^{H}\sum_{w=1}^{W}\big(\hat I_{c,h,w}-I_{c,h,w}\big)^2 \)
\( \mathrm{PSNR}=10\log_{10}\left(\frac{1}{\mathrm{MSE}}\right) \)
SSIM
\( \mathrm{SSIM}(x,y)=\frac{(2\mu_x\mu_y+C_1)(2\sigma_{xy}+C_2)}{(\mu_x^2+\mu_y^2+C_1)(\sigma_x^2+\sigma_y^2+C_2)} \)
tOF (TecoGAN)
\( F^{GT}_t=\mathrm{OF}(I_{t-1},I_t),\quad F^{Pred}_t=\mathrm{OF}(\hat I_{t-1},\hat I_t) \)
\( \mathrm{tOF}=\frac{1}{T-1}\sum_{t=2}^{T}\left(\frac{1}{HW}\sum_{h=1}^{H}\sum_{w=1}^{W}\left\|F^{Pred}_t(h,w)-F^{GT}_t(h,w)\right\|_{1}\right) \)
Leaderboard Scoring
The final leaderboard score is computed from four normalized sub-scores:
PSNR
,
SSIM
,
tOF
, and
Size
.
Higher PSNR and SSIM are better, while lower tOF and Size are better.
The overall score is defined as:
\[
  \mathrm{Score}
  =
  0.40\,s_{\mathrm{PSNR}}
  +0.30\,s_{\mathrm{Size}}
  +0.20\,s_{\mathrm{tOF}}
  +0.10\,s_{\mathrm{SSIM}}
  \]
Normalized PSNR score
PSNR is linearly normalized to the range \([0,1]\) with clipping:
\[
  s_{\mathrm{PSNR}}
  =
  \frac{\operatorname{clip}(\mathrm{PSNR},\,0,\,60)}{60}
  \]
Normalized SSIM score
SSIM is linearly normalized to the range \([0,1]\) with clipping:
\[
  s_{\mathrm{SSIM}}
  =
  \operatorname{clip}(\mathrm{SSIM},\,0,\,1)
  \]
Normalized tOF score
tOF is a lower-is-better metric. Its leaderboard sub-score is computed using a logarithmic mapping:
\[
  s_{\mathrm{tOF}}
  =
  1-
  \operatorname{clip}
  \left(
  \frac{\log(1+\mathrm{tOF})}{\log(1+1)},
  0,1
  \right)
  \]
Normalized Size score
Size is a lower-is-better metric. Its leaderboard sub-score is computed on a log-scale:
\[
  s_{\mathrm{Size}}
  =
  1-
  \frac{\log_{10}(\mathrm{Size})-\log_{10}(0.1)}
       {\log_{10}(500)-\log_{10}(0.1)}
  \]
If the submission size exceeds the predefined upper bound (\(500\)), the size sub-score is set to \(-0.5\) as an out-of-range penalty.
Overall Ranking
The final leaderboard score is computed as a weighted sum of normalized metric scores, where PSNR and SSIM are higher-is-better, while Size and tOF are lower-is-better (with monotonic normalization applied before aggregation).
\[
  \mathrm{Score}=0.40\,s_{\mathrm{PSNR}}+0.30\,s_{\mathrm{Size}}+0.20\,s_{\mathrm{tOF}}+0.10\,s_{\mathrm{SSIM}}
  \]
Submission Package (template)
Unless otherwise stated, please prepare a single zip file with the following structure:
<scene_name>/
  frame000000/
      cam00.png
      cam01.png
      ...
  frame000001/
      cam00.png
    ...
Note:
Your submission must contain the required images for each GT frame/camera you want to be evaluated on.
Contact
For questions about evaluation rules or submission formatting, please contact the organizers listed on the Organizers page.