{"metadata":{"kernelspec":{"name":"ir","display_name":"R","language":"R"},"language_info":{"name":"R","codemirror_mode":"r","pygments_lexer":"r","mimetype":"text/x-r-source","file_extension":".r","version":"4.4.0"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":59156,"databundleVersionId":13874099,"sourceType":"competition"}],"dockerImageVersionId":30749,"isInternetEnabled":true,"language":"r","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"
\n
\n 🦫 🦭 🦑 πŸ€ 🐀 πŸ¦’ 🦜 πŸͺ° 🐌 πŸ¦™ 🐦 🦦 πŸͺΌ πŸ† πŸ‡\n
\n An Exploratory Data Safari\n
","metadata":{}},{"cell_type":"markdown","source":"! Note since version 8 - running on the updated train/test datasets. Info was posted [here](https://www.kaggle.com/competitions/MABe-mouse-behavior-detection/discussion/609470)","metadata":{}},{"cell_type":"code","source":"library(data.table)\nlibrary(duckdb)\nlibrary(arrow)\nlibrary(fs)\nlibrary(ggplot2)\nlibrary(gganimate)\nlibrary(tictoc)","metadata":{"_uuid":"051d70d956493feee0c6d64651c6a088724dca2a","_execution_state":"idle","trusted":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2025-10-26T08:04:50.100497Z","iopub.execute_input":"2025-10-26T08:04:50.102369Z","iopub.status.idle":"2025-10-26T08:04:51.741202Z","shell.execute_reply":"2025-10-26T08:04:51.739501Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"emoji_map <- c(\n Snail = \"🐌\",\n Movies = \"🎬\",\n Keypoints = \"πŸ“\",\n CalMS21 = \"πŸ§ͺ\",\n Tapir = \"πŸ¦™\",\n Swallow = \"🐦\",\n Meerkat = \"🦦\",\n Jellyfish = \"πŸͺΌ\",\n Panther = \"πŸ†\",\n Hare = \"πŸ‡\",\n CRIM13 = \"πŸ”¬\",\n Ferret = \"🦫\",\n Manatee = \"🦭\",\n Mink = \"🦑\",\n Shrew = \"πŸ€\",\n Goldfinch = \"🐀\",\n Giraffe = \"πŸ¦’\",\n Parrot = \"🦜\",\n Fly = \"πŸͺ°\"\n)\n\nadd_emojis <- function(lab) {\n hits <- names(emoji_map)[\n sapply(names(emoji_map), \\(k) grepl(k, lab, ignore.case = TRUE))\n ]\n paste0(lab, \" \", paste(emoji_map[hits], collapse = \"\"))\n}\nfmt <- function(x, n_vid) sprintf(\"%d (%.1f%%)\", x, 100 * x / n_vid)\n\nfig <- function(width, heigth){\n options(repr.plot.width = width, repr.plot.height = heigth)\n}","metadata":{"trusted":true,"_kg_hide-input":true,"execution":{"iopub.status.busy":"2025-10-26T08:04:51.743720Z","iopub.execute_input":"2025-10-26T08:04:51.783361Z","iopub.status.idle":"2025-10-26T08:04:51.806107Z","shell.execute_reply":"2025-10-26T08:04:51.804151Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"#

🦜 Data overview

","metadata":{}},{"cell_type":"markdown","source":"## Train and test meta data","metadata":{}},{"cell_type":"code","source":"dir <- \"/kaggle/input/MABe-mouse-behavior-detection/\"\ntrain_meta <- fread(file.path(dir, \"train.csv\"))\ntest_meta <- fread(file.path(dir, \"test.csv\"))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-26T08:04:51.809286Z","iopub.execute_input":"2025-10-26T08:04:51.811488Z","iopub.status.idle":"2025-10-26T08:04:51.917103Z","shell.execute_reply":"2025-10-26T08:04:51.915310Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"print_meta_overview <- function(dt, name = \"Dataset\", n = NULL, max_width = 40) {\n cat(\"=== \", name, \" ===\\n\", sep = \"\")\n cat(nrow(dt), \"rows,\", ncol(dt), \"columns\\n\")\n cat(uniqueN(dt$lab_id), \" labs (\", uniqueN(dt$video_id), \" videos)\\n\", sep = \"\")\n if (is.null(n)) n <- dt[, .N]\n sample_dt <- dt[1:n]\n sample_dt[, body_parts_tracked := substr(body_parts_tracked, 1, max_width)]\n sample_dt[, behaviors_labeled := substr(behaviors_labeled, 1, max_width)]\n head(sample_dt, n)\n}","metadata":{"trusted":true,"_kg_hide-input":true,"execution":{"iopub.status.busy":"2025-10-26T08:04:51.919567Z","iopub.execute_input":"2025-10-26T08:04:51.920911Z","iopub.status.idle":"2025-10-26T08:04:51.933956Z","shell.execute_reply":"2025-10-26T08:04:51.932263Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"print_meta_overview(train_meta, \"Train\", n = 3, max_width = 40)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-26T08:04:51.936365Z","iopub.execute_input":"2025-10-26T08:04:51.937714Z","iopub.status.idle":"2025-10-26T08:04:52.010176Z","shell.execute_reply":"2025-10-26T08:04:52.008399Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"print_meta_overview(test_meta, \"Test\", max_width = 40)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-26T08:04:52.012757Z","iopub.execute_input":"2025-10-26T08:04:52.014189Z","iopub.status.idle":"2025-10-26T08:04:52.055291Z","shell.execute_reply":"2025-10-26T08:04:52.053563Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"names(train_meta)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-26T08:04:52.057770Z","iopub.execute_input":"2025-10-26T08:04:52.059124Z","iopub.status.idle":"2025-10-26T08:04:52.074586Z","shell.execute_reply":"2025-10-26T08:04:52.072912Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"
\n
Each row corresponds to one video, and the columns provide metadata about the recording setup as well as basic information about up to 4 mice that may appear in the video.\n\n- Train set: 8,790 (8,789 after data update) videos from 21 labs, 38 columns.\n- Test set: 1 video from 1 lab, with the same structure.\n\nAccording to the competition [data page](https://www.kaggle.com/competitions/MABe-mouse-behavior-detection/data), the key fields are:\n\n- lab_id - The pseudonym of the lab that provided the data.\n- video_id - A unique identifier for the video.\n- mouse[1-4] [strain/color/sex/id/age/condition] - Basic information about each mouse.\n- frames per second\n- video duration (sec)\n- pix per cm (approx)\n- video [width/height]\n- arena [width/height] (cm)\n- arena shape\n- arena type\n- body parts tracked - Each lab used different technology to track their mice, so the specific body parts tracked may vary.\n- behaviors labeled - The behaviors labeled in this video. Necessary as the annotations are sparse. Each lab annotated their videos differently, may not have annotated all behaviors for a video, and may not have annotated all mice in each video.\n- tracking method - The model used to track the animal's pose.
\n
","metadata":{}},{"cell_type":"markdown","source":"
\n πŸ’‘ Note: From the \n \n competition description\n , the test video here is just a placeholder to let us build and debug our pipelines: \"This competition uses a hidden test set. When your submitted notebook is scored, the actual test data (including a full length sample submission) will be made available to your notebook. Expect to see roughly 200 videos in the hidden test set.\"\n
","metadata":{}},{"cell_type":"markdown","source":"## Train tracking parquet files","metadata":{}},{"cell_type":"markdown","source":"How 1 tracking file looks like?","metadata":{}},{"cell_type":"code","source":"track_files <- dir_ls(\n file.path(dir, \"train_tracking\"),\n recurse = TRUE, glob = \"*.parquet\"\n)\ntrain_tracks <- rbindlist(\n lapply(track_files[grepl(\"754974789\", track_files)], \\(f) {\n dt <- as.data.table(read_parquet(f))\n fn <- gsub(\"\\\\.parquet\", \"\", basename(f))\n dt[, video_file := fn]\n })\n)\ncat(\"Number of files:\", length(track_files))\nhead(train_tracks)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-26T08:04:52.077056Z","iopub.execute_input":"2025-10-26T08:04:52.078377Z","iopub.status.idle":"2025-10-26T08:05:02.625754Z","shell.execute_reply":"2025-10-26T08:05:02.624094Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"train_tracks[, uniqueN(video_frame)]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-26T08:05:02.628211Z","iopub.execute_input":"2025-10-26T08:05:02.629610Z","iopub.status.idle":"2025-10-26T08:05:02.648829Z","shell.execute_reply":"2025-10-26T08:05:02.647097Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"track_files <- dir_ls(\n file.path(dir, \"train_tracking\"),\n recurse = TRUE, glob = \"*.parquet\"\n)\ntrain_tracks <- rbindlist(\n lapply(track_files[1:3], \\(f) {\n dt <- as.data.table(read_parquet(f))\n fn <- gsub(\"\\\\.parquet\", \"\", basename(f))\n dt[, video_file := fn]\n })\n)\ncat(\"Number of files:\", length(track_files))\nhead(train_tracks)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-26T08:05:02.651910Z","iopub.execute_input":"2025-10-26T08:05:02.653343Z","iopub.status.idle":"2025-10-26T08:05:08.751355Z","shell.execute_reply":"2025-10-26T08:05:08.749583Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"train_tracks[, unique(bodypart)]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-26T08:05:08.753777Z","iopub.execute_input":"2025-10-26T08:05:08.755141Z","iopub.status.idle":"2025-10-26T08:05:09.054340Z","shell.execute_reply":"2025-10-26T08:05:09.051897Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"Well, here we have (x,y) coordinates of multiple body parts for each mouse across frames. Each row = one body part of one mouse in one frame. Body parts include core points (body_center, nose, tail_tip), lateral points (lateral_left, lateral_right, ear_left, ear_right), and detailed headpiece markers (eight positions around the head).","metadata":{}},{"cell_type":"markdown","source":"Load summary statistics from all available tracking data:","metadata":{}},{"cell_type":"code","source":"con <- dbConnect(duckdb())\ndbExecute(con, \"PRAGMA threads=4;\")\n\nbase <- file.path(dir, \"train_tracking\")\n\nquery <- sprintf(\"\nSELECT\n filename,\n MIN(video_frame) AS first_frame,\n MAX(video_frame) AS last_frame,\n COUNT(DISTINCT video_frame) AS num_distinct_frames,\n COUNT(DISTINCT mouse_id) AS num_mice,\n COUNT(DISTINCT bodypart) AS num_bodyparts,\n MIN(x) AS min_x, MAX(x) AS max_x,\n MIN(y) AS min_y, MAX(y) AS max_y,\n COUNT(*) - COUNT(x) AS num_na_x,\n COUNT(*) - COUNT(y) AS num_na_y\nFROM read_parquet('%s/*/*.parquet', filename=true)\nGROUP BY filename\n\", base)\n\ntic()\ntrack_summary <- as.data.table(dbGetQuery(con, query))\ntoc()\ntrack_summary[, video_id := sub(\"\\\\.parquet$\", \"\", basename(filename))]\ntrack_summary[, filename := NULL]\ntrack_summary[, span_frames := last_frame - first_frame + 1L]\nsetcolorder(track_summary, \"video_id\", before = \"first_frame\")\n\ncat(nrow(track_summary), \"rows,\", ncol(track_summary), \"columns\\n\")\nhead(track_summary)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-26T08:05:09.058285Z","iopub.execute_input":"2025-10-26T08:05:09.059813Z","iopub.status.idle":"2025-10-26T08:06:53.396994Z","shell.execute_reply":"2025-10-26T08:06:53.394464Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"track_summary[\n , lapply(.SD, function(col) range(col)),\n .SDcols = c(\n \"num_distinct_frames\", \"num_mice\", \"num_bodyparts\",\n \"num_na_x\", \"num_na_y\"\n )\n]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-26T08:06:53.401018Z","iopub.execute_input":"2025-10-26T08:06:53.402429Z","iopub.status.idle":"2025-10-26T08:06:53.429053Z","shell.execute_reply":"2025-10-26T08:06:53.426569Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"
\n πŸ’‘ Note: \n \n
","metadata":{}},{"cell_type":"code","source":"track_summary[video_id == \"1728439177\"]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-26T08:06:53.433061Z","iopub.execute_input":"2025-10-26T08:06:53.434575Z","iopub.status.idle":"2025-10-26T08:06:53.472890Z","shell.execute_reply":"2025-10-26T08:06:53.470549Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Train annotation files","metadata":{}},{"cell_type":"code","source":"ann_files <- dir_ls(\n file.path(dir, \"train_annotation\"),\n recurse = TRUE, glob = \"*.parquet\"\n)\ntrain_ann <- rbindlist(\n lapply(ann_files, \\(f) {\n dt <- as.data.table(read_parquet(f))\n fn <- gsub(\"\\\\.parquet\", \"\", basename(f))\n dt[, video_file := fn]\n }),\n use.names = TRUE, fill = TRUE\n)\ncat(\"Number of files:\", length(ann_files))\nhead(train_ann)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-26T08:06:53.476762Z","iopub.execute_input":"2025-10-26T08:06:53.478316Z","iopub.status.idle":"2025-10-26T08:07:01.548768Z","shell.execute_reply":"2025-10-26T08:07:01.547040Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"saveRDS(train_ann, \"train_ann.RDS\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-26T08:07:01.551366Z","iopub.execute_input":"2025-10-26T08:07:01.552757Z","iopub.status.idle":"2025-10-26T08:07:01.763464Z","shell.execute_reply":"2025-10-26T08:07:01.761570Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"ann_labs <- sapply(strsplit(ann_files, \"/\"), function(p) {\n i <- match(\"train_annotation\", p)\n p[i + 1L]\n})\nsetdiff(train_meta[, unique(lab_id)], ann_labs)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-26T08:07:01.766139Z","iopub.execute_input":"2025-10-26T08:07:01.767573Z","iopub.status.idle":"2025-10-26T08:07:01.794064Z","shell.execute_reply":"2025-10-26T08:07:01.792398Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"
\n πŸ’‘ Note: the annotations for the two largest datasets from 'MABe22_keypoints''MABe22_movies' are missing. It seems that the actual training set is only 863 (847 after update) videos. I am doing the following EDA for all videos described in train.csv, but I may update this further.\n
","metadata":{}},{"cell_type":"code","source":"train_ann[, unique(action)]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-26T08:07:01.796599Z","iopub.execute_input":"2025-10-26T08:07:01.797955Z","iopub.status.idle":"2025-10-26T08:07:01.815112Z","shell.execute_reply":"2025-10-26T08:07:01.813374Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"Let's also check what is expected in the submission","metadata":{}},{"cell_type":"code","source":"fread(file.path(dir, \"sample_submission.csv\"))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-26T08:07:01.817641Z","iopub.execute_input":"2025-10-26T08:07:01.819110Z","iopub.status.idle":"2025-10-26T08:07:01.848679Z","shell.execute_reply":"2025-10-26T08:07:01.846872Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"So, the prediction target is temporal action segments (start_frame - stop_frame). In other words, we’re asked to predict who-what-when for each video: which mouse (agent) did what action to which mouse (target), and over which time interval. That’s a lot at once!","metadata":{}},{"cell_type":"markdown","source":"#

🎬 Animate action

","metadata":{}},{"cell_type":"markdown","source":"Let's build a schematic mouse skeleton from tracked body parts and render the action window as an animation - playing frames in sequence to visualize motion over time.","metadata":{}},{"cell_type":"code","source":"make_mouse_schematic <- function(action_tracks, frame_id) {\n edges_def <- data.table(\n path = c(\"ears\",\"ears\",\"ears\",\"spine\",\"tail\",\"tail\",\"tail\"),\n from = c(\"ear_left\",\"ear_left\",\"ear_right\", \"nose\",\"body_center\",\"tail_base\",\"tail_midpoint\"),\n to = c(\"ear_right\",\"nose\",\"nose\", \"body_center\",\"tail_base\",\"tail_midpoint\",\"tail_tip\")\n )\n dt <- action_tracks[video_frame == frame_id]\n seg <- merge(\n dt, edges_def, by.x = \"bodypart\", by.y = \"from\",\n allow.cartesian = TRUE\n )\n seg <- merge(\n seg,\n dt[, .(mouse_id, bodypart, x_to = x, y_to = y)],\n by.x = c(\"mouse_id\",\"to\"),\n by.y = c(\"mouse_id\",\"bodypart\"),\n all.x = TRUE\n )\n setnames(seg, c(\"x\",\"y\"), c(\"x_from\",\"y_from\"))\n seg <- seg[!is.na(x_from) & !is.na(y_from) & !is.na(x_to) & !is.na(y_to)]\n seg[, .(video_frame, mouse_id, path, from = bodypart, to, x_from, y_from, x_to, y_to)]\n}\n\nanimate_action <- function(dt_meta, train_ann, train_tracks, video.id, show_action, step = 10) {\n ann_row <- train_ann[video_file == video.id & action == show_action]\n if (nrow(ann_row) == 0L) {\n avail <- train_ann[video_file == video.id, unique(action)]\n cat(sprintf(\n \"No '%s' action found for video %s.\\nAvailable actions: %s\\n\",\n show_action, video.id,\n if (length(avail)) paste(avail, collapse = \", \") else \"\"\n ))\n return(invisible(NULL))\n } else {\n ann_row <- ann_row[1]\n }\n\n frames <- ann_row$start_frame:ann_row$stop_frame\n frames <- frames[seq(1L, length(frames), by = step)]\n\n action_tracks <- train_tracks[\n video_file == video.id &\n mouse_id %in% c(ann_row$agent_id, ann_row$target_id) &\n video_frame %in% frames\n ]\n\n schem_dt <- rbindlist(lapply(frames, function(fr)\n make_mouse_schematic(action_tracks, fr)\n ))\n\n vw <- dt_meta[video_id == video.id, video_width_pix][1L]\n vh <- dt_meta[video_id == video.id, video_height_pix][1L]\n fps <- dt_meta[video_id == video.id, frames_per_second][1L]\n\n anim <- ggplot(schem_dt, aes(\n x = x_from, y = y_from,\n xend = x_to, yend = y_to,\n color = factor(mouse_id),\n group = interaction(mouse_id, path)\n )) +\n geom_segment(linewidth = 1) +\n coord_fixed(xlim = c(0, vw), ylim = c(0, vh)) +\n scale_y_reverse() +\n labs(\n x = \"x (px)\", y = \"y (px)\", color = \"mouse_id\",\n title = \"frame: {frame_time}\"\n ) +\n theme_bw(base_size = 22) +\n transition_time(video_frame) +\n shadow_mark(past = TRUE, future = FALSE, alpha = 0.15)\n\n h <- 640; w <- round(h * vw / vh)\n dur_frames <- ann_row$stop_frame - ann_row$start_frame + 1\n dur_sec <- dur_frames / fps\n gif <- animate(\n anim, nframes = length(frames),\n duration = dur_sec,\n width = w, height = h,\n renderer = gifski_renderer()\n )\n \n file_name <- sprintf(\n \"anim_%s_%s.gif\", video.id,\n tolower(gsub(\"\\\\s+\",\"_\", ann_row$action))\n )\n anim_save(file_name, gif)\n \n msg <- sprintf(\n \"Video %s: mouse %d (agent) performs '%s' on mouse %d from frame %d to %d (%d frames, %.2f s).\",\n video.id, ann_row$agent_id, ann_row$action, ann_row$target_id,\n ann_row$start_frame, ann_row$stop_frame, dur_frames, dur_sec\n )\n cat(msg, \"\\n\")\n IRdisplay::display_html(paste0(\"\"))\n}","metadata":{"trusted":true,"_kg_hide-input":true,"execution":{"iopub.status.busy":"2025-10-26T08:07:01.851407Z","iopub.execute_input":"2025-10-26T08:07:01.852787Z","iopub.status.idle":"2025-10-26T08:07:01.868005Z","shell.execute_reply":"2025-10-26T08:07:01.866236Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"animate_action(train_meta, train_ann, train_tracks, video.id = \"1212811043\", show_action = \"chase\", step = 2)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-26T08:07:01.870580Z","iopub.execute_input":"2025-10-26T08:07:01.872107Z","iopub.status.idle":"2025-10-26T08:07:08.381827Z","shell.execute_reply":"2025-10-26T08:07:08.380035Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"
\n πŸ’‘ Note: \n \n
","metadata":{}},{"cell_type":"code","source":"animate_action(train_meta, train_ann, train_tracks, video.id = \"1260392287\", show_action = \"chase\", step = 2)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-26T08:07:08.393531Z","iopub.execute_input":"2025-10-26T08:07:08.395595Z","iopub.status.idle":"2025-10-26T08:07:13.200206Z","shell.execute_reply":"2025-10-26T08:07:13.198348Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"
\n πŸ’‘ Note: This 'chase' example looks much more realistic!\n
","metadata":{}},{"cell_type":"markdown","source":"#

πŸ¦’ Video characteristics

","metadata":{}},{"cell_type":"markdown","source":"Here I isolate video-level characteristics (fps, duration, pixel-to-cm scale, and resolution) and compute train min/median/max, then place the test values alongside. This quickly shows whether the test video example sits inside typical train ranges, and what are these ranges.","metadata":{}},{"cell_type":"code","source":"dt_meta <- rbind(\n train_meta[, dataset := \"train\"],\n test_meta[, dataset := \"test\"]\n)\ndt_meta[, video_file := as.character(video_id)]\ndt_meta[, has_ann := behaviors_labeled != \"\"]\nn_with_ann <- dt_meta[has_ann == TRUE, uniqueN(video_id)]\nn_total <- dt_meta[, uniqueN(video_id)]\n\ncat(\"Column names:\", names(dt_meta))\ncat(\"\\nVideos with annotations:\", n_with_ann)\ncat(\"\\nVideos without annotations:\", n_total - n_with_ann)\ncat(\"\\nTotal videos:\", n_total)","metadata":{"trusted":true,"_kg_hide-input":false,"execution":{"iopub.status.busy":"2025-10-26T08:07:13.203917Z","iopub.execute_input":"2025-10-26T08:07:13.205451Z","iopub.status.idle":"2025-10-26T08:07:13.247800Z","shell.execute_reply":"2025-10-26T08:07:13.246108Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"There is only 847 annotated files, so it should be 848 with behaviors_labeled in dt_meta (1 extra is the test video), but there are 849, meaning that there is one video without any annotations:","metadata":{}},{"cell_type":"code","source":"with_ann_vid <- dt_meta[has_ann == TRUE & dataset == \"train\", unique(video_file)]\ncheck_vid <- setdiff(with_ann_vid, train_ann$video_file)\ncheck_vid","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-26T08:07:13.250491Z","iopub.execute_input":"2025-10-26T08:07:13.251915Z","iopub.status.idle":"2025-10-26T08:07:13.277788Z","shell.execute_reply":"2025-10-26T08:07:13.276144Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"For this video the annotation file is missing. Clarified [here](https://www.kaggle.com/competitions/MABe-mouse-behavior-detection/discussion/608413#3293833): \n*So an important detail here: in some videos, the labs annotated for (ie looked for) a behavior, but it never actually occurred. In this case, we still list that behavior in behaviors_labeled because it is safe to use the video as a source of negative training data. (In contrast, if a behavior doesn't show up in the behaviors_labeled column that just means that researchers never checked for it, not that it never happened.)In the case of PleasantMeerkat, the annotators looked for several behaviors, but the mice in this experiment interacted so rarely that for that particular video none of them occurred, hence the annotation file for video 1375833299 is completely empty. I'll see though if we can provide an empty placeholder file just so this doesn't trip up people's analysis code.*\nNow let's compute train min/median/max for the video characteristics: fps, duration, pix_per_cm, and resolution","metadata":{}},{"cell_type":"code","source":"id_cols <- c(\"dataset\", \"has_ann\", \"lab_id\", \"video_file\")\nvars <- c(\n \"frames_per_second\",\n \"video_duration_sec\",\n \"pix_per_cm_approx\",\n \"video_width_pix\",\n \"video_height_pix\"\n)\ndt_video <- dt_meta[, c(id_cols, vars), with = FALSE]\n\ndt_video[, `:=`(\n aspect_ratio = video_width_pix / video_height_pix,\n pixel_area_mpx = video_width_pix * video_height_pix / 1e6\n)]\n\ndt_video[, (vars) := lapply(.SD, as.numeric), .SDcols = vars]\ndt <- melt(dt_video, id.vars = id_cols, variable.factor = FALSE)\n# dt[is.na(value) | value == \"\"] # none","metadata":{"trusted":true,"_kg_hide-input":false,"execution":{"iopub.status.busy":"2025-10-26T08:07:13.280323Z","iopub.execute_input":"2025-10-26T08:07:13.281682Z","iopub.status.idle":"2025-10-26T08:07:13.308412Z","shell.execute_reply":"2025-10-26T08:07:13.306708Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Videos per lab","metadata":{}},{"cell_type":"code","source":"track_summary[video_id == \"438887472\"]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-26T08:15:44.623610Z","iopub.execute_input":"2025-10-26T08:15:44.625344Z","iopub.status.idle":"2025-10-26T08:15:44.655957Z","shell.execute_reply":"2025-10-26T08:15:44.653674Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"vid_stats <- dt_meta[dataset == \"train\", .(lab_id, video_file, has_ann)]\nvid_stats[track_summary, on = .(video_file = video_id), n_frames := i.num_distinct_frames]\nvid_stats <- vid_stats[, .(\n n_videos = uniqueN(video_file),\n n_annotated = uniqueN(video_file[has_ann == TRUE]),\n n_frames = sum(n_frames)\n), by = .(lab_id)]\n\nvid_stats[order(-n_frames)]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-26T08:17:20.909517Z","iopub.execute_input":"2025-10-26T08:17:20.911114Z","iopub.status.idle":"2025-10-26T08:17:20.971787Z","shell.execute_reply":"2025-10-26T08:17:20.970204Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## All videos","metadata":{}},{"cell_type":"code","source":"stats_dt <- dt[\n dataset == \"train\", .(\n n = .N,\n min = min(value),\n q25 = quantile(value, 0.25),\n median = median(value),\n q75 = quantile(value, 0.75),\n max = max(value)\n ), by = .(variable)]\n\nstats_dt <- merge(\n stats_dt, dt[dataset == \"test\", .(variable, test_value = value)],\n by = \"variable\", all.x = TRUE, sort = FALSE\n)\nstats_dt","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-26T08:18:04.625285Z","iopub.execute_input":"2025-10-26T08:18:04.626779Z","iopub.status.idle":"2025-10-26T08:18:04.674705Z","shell.execute_reply":"2025-10-26T08:18:04.672945Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"fig(20, 10)\nplot_data <- dt[\n !variable %in% c(\"video_height_pix\", \"video_width_pix\")\n]\nggplot(plot_data[dataset == \"train\"], aes(x = value)) +\n geom_density(fill = \"skyblue\", alpha = 0.5) +\n geom_vline(\n data = plot_data[dataset == \"test\"], aes(xintercept = value),\n color = \"red\", linewidth = 0.8\n ) +\n facet_wrap(~ variable, scales = \"free\", ncol = 3) +\n labs(\n x = NULL,\n y = \"Density\",\n title = \"Train distributions (8789 videos) & example test video values (red line)\"\n ) +\n theme_bw(base_size = 22)","metadata":{"trusted":true,"_kg_hide-input":true,"execution":{"iopub.status.busy":"2025-10-26T08:18:04.677388Z","iopub.execute_input":"2025-10-26T08:18:04.678863Z","iopub.status.idle":"2025-10-26T08:18:05.728633Z","shell.execute_reply":"2025-10-26T08:18:05.725829Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"
\n πŸ’‘ Note: \n \n
","metadata":{}},{"cell_type":"markdown","source":"## Annotated videos","metadata":{}},{"cell_type":"code","source":"stats_dt <- dt[\n dataset == \"train\" & has_ann == TRUE, .(\n n = .N,\n min = min(value),\n q25 = quantile(value, 0.25),\n median = median(value),\n q75 = quantile(value, 0.75),\n max = max(value)\n ), by = .(variable)]\n\nstats_dt <- merge(\n stats_dt, dt[dataset == \"test\", .(variable, test_value = value)],\n by = \"variable\", all.x = TRUE, sort = FALSE\n)\nstats_dt","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-26T08:18:05.732923Z","iopub.execute_input":"2025-10-26T08:18:05.735305Z","iopub.status.idle":"2025-10-26T08:18:05.780596Z","shell.execute_reply":"2025-10-26T08:18:05.778231Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"fig(20, 10)\nplot_data <- dt[\n !variable %in% c(\"video_height_pix\", \"video_width_pix\") &\n has_ann == TRUE\n]\nggplot(plot_data[dataset == \"train\"], aes(x = value)) +\n geom_density(fill = \"skyblue\", alpha = 0.5) +\n geom_vline(\n data = plot_data[dataset == \"test\"], aes(xintercept = value),\n color = \"red\", linewidth = 0.8\n ) +\n facet_wrap(~ variable, scales = \"free\", ncol = 3) +\n labs(\n x = NULL,\n y = \"Density\",\n title = \"Train distributions (848 annotated videos) & example test video values (red line)\"\n ) +\n theme_bw(base_size = 22)","metadata":{"trusted":true,"_kg_hide-input":true,"execution":{"iopub.status.busy":"2025-10-26T08:18:05.784561Z","iopub.execute_input":"2025-10-26T08:18:05.786000Z","iopub.status.idle":"2025-10-26T08:18:06.596910Z","shell.execute_reply":"2025-10-26T08:18:06.594278Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"
\n πŸ’‘ Note: For the annotated subset (848 videos), the example test video looks more representative\n
","metadata":{}},{"cell_type":"markdown","source":"## By lab\nWe now stratify these same variables by laboratory to see whether the distributions differ by lab:","metadata":{}},{"cell_type":"code","source":"fig(25, 10)\nplot_data <- dt[\n !variable %in% c(\"video_height_pix\", \"video_width_pix\")\n]\nggplot(plot_data[dataset == \"train\"], aes(x = value, y = lab_id)) +\n geom_boxplot(outlier.alpha = 0.2, width = 0.6, orientation = \"y\") +\n geom_vline(\n data = plot_data[dataset == \"test\"],\n aes(xintercept = value),\n colour = \"red\", linewidth = 0.7\n ) +\n facet_grid(. ~ variable, scales = \"free_x\") +\n labs(title = \"Per-lab train distributions (8789 videos) (red = test value)\",\n x = \"Value\", y = \"Lab\") +\n theme_bw(base_size = 22) +\n theme(strip.text.y = element_text(angle = 0))","metadata":{"trusted":true,"_kg_hide-input":true,"execution":{"iopub.status.busy":"2025-10-26T08:18:06.601020Z","iopub.execute_input":"2025-10-26T08:18:06.603174Z","iopub.status.idle":"2025-10-26T08:18:08.782788Z","shell.execute_reply":"2025-10-26T08:18:08.779774Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"
\n πŸ’‘ Per-lab heterogeneity.\n \n
","metadata":{}},{"cell_type":"markdown","source":"Video resolution by laboratory:","metadata":{}},{"cell_type":"code","source":"fig(20, 10)\nggplot(\n dt_video[dataset == \"train\"],\n aes(x = video_width_pix, y = video_height_pix, color = lab_id)\n ) +\n geom_point(alpha = 0.45, size = 2) +\n geom_point(\n data = dt_video[dataset == \"test\"],\n aes(x = video_width_pix, y = video_height_pix),\n color = \"black\", fill = \"red\", size = 5, shape = 21, stroke = 0\n ) +\n coord_equal() +\n labs(\n x = \"Video width (px)\",\n y = \"Video height (px)\",\n color = \"Lab\",\n title = \"Resolution scatter: train by lab; test in red\"\n ) +\n theme_bw(base_size = 22)","metadata":{"trusted":true,"_kg_hide-input":true,"execution":{"iopub.status.busy":"2025-10-26T08:18:08.786989Z","iopub.execute_input":"2025-10-26T08:18:08.789161Z","iopub.status.idle":"2025-10-26T08:18:09.880559Z","shell.execute_reply":"2025-10-26T08:18:09.878615Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"
\n πŸ’‘ Note: Labs use distinctly different resolutions (β‰ˆ200–2000 px on each side).\n
","metadata":{}},{"cell_type":"markdown","source":"#

πŸ† Experiment characteristics

","metadata":{}},{"cell_type":"markdown","source":"## Body parts","metadata":{}},{"cell_type":"markdown","source":"Extract all body parts tracked acrosss all videos:","metadata":{}},{"cell_type":"code","source":"split_body_parts <- function(x){\n x <- gsub('\"\"','', x, fixed = TRUE)\n x <- gsub(\"^\\\\[|\\\\]$\", \"\", x)\n trimws(strsplit(x, \", \", fixed = TRUE)[[1L]])\n}\nbp <- dt_meta[, .(\n body_part = split_body_parts(body_parts_tracked)\n), by = .(dataset, has_ann, lab_id, video_id)]\n\nn_miss <- bp[is.na(body_part) | body_part == \"\", uniqueN(video_id)]\n\ncat(sprintf(\"Unique labs: %d\\n\", bp[, uniqueN(lab_id)]))\ncat(sprintf(\"Unique videos: %d\\n\", bp[, uniqueN(video_id)]))\ncat(sprintf(\"Videos with missing body parts: %d\\n\", n_miss))\ncat(sprintf(\"Unique body parts: %d\\n\", bp[, uniqueN(body_part)]))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-26T08:18:09.883152Z","iopub.execute_input":"2025-10-26T08:18:09.884493Z","iopub.status.idle":"2025-10-26T08:18:10.558738Z","shell.execute_reply":"2025-10-26T08:18:10.556591Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"So for all videos of all labs we have data about what body parts were tracked. There are 29 different body parts in total. Now let's calculate for train videos:\n- how many training videos in total contain each body part\n- how many annotated training videos contain each body part","metadata":{}},{"cell_type":"code","source":"stats <- bp[dataset == \"train\", .(\n n_train_videos = uniqueN(video_id),\n n_train_ann_videos = uniqueN(video_id[has_ann == TRUE])\n ), by = body_part\n]\nstats[order(-n_train_videos)]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-26T08:18:10.562642Z","iopub.execute_input":"2025-10-26T08:18:10.564081Z","iopub.status.idle":"2025-10-26T08:18:10.616529Z","shell.execute_reply":"2025-10-26T08:18:10.614113Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"Overall, this shows a strong core set of consistently tracked points across nearly all labs, and a variety of specific body parts with very limited availability (e.g. all headpiece_* were tracked in only 7 train videos)","metadata":{}},{"cell_type":"markdown","source":"## Behaviors","metadata":{}},{"cell_type":"markdown","source":"The annotated actions I extract form the annotation files (847 in total). Here I calculate:\n- the number of events per action and\n- the number of unique videos where each action appears.","metadata":{}},{"cell_type":"code","source":"self_actions <- train_ann[agent_id == target_id, unique(action)]\ncat(\"Self actions:\", paste(self_actions, collapse = \", \"))\npair_actions <- train_ann[agent_id != target_id, unique(action)]\ncat(\"\\nPair actions:\", paste(pair_actions, collapse = \", \"))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-26T08:18:10.620378Z","iopub.execute_input":"2025-10-26T08:18:10.621820Z","iopub.status.idle":"2025-10-26T08:18:10.647999Z","shell.execute_reply":"2025-10-26T08:18:10.646252Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"**Self actions**","metadata":{}},{"cell_type":"code","source":"bh <- train_ann[\n agent_id == target_id,\n .(video_file, action, num_frames = stop_frame - start_frame + 1L)\n]\nbh[dt_meta, on = .(video_file), `:=` (fps = i.frames_per_second, lab_id = i.lab_id)]\nbh[, duration := num_frames / fps]\nbh <- bh[, .(\n n_events = .N,\n n_train_ann_videos = uniqueN(video_file),\n n_labs = uniqueN(lab_id),\n min_sec = round(min(duration), 2),\n mean_sec = round(mean(duration), 2),\n max_sec = round(max(duration), 2),\n sum_sec = round(sum(duration), 2)\n), by = .(action)]\n\nbh[order(-n_events, -n_train_ann_videos)]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-26T08:18:10.650697Z","iopub.execute_input":"2025-10-26T08:18:10.652129Z","iopub.status.idle":"2025-10-26T08:18:10.734364Z","shell.execute_reply":"2025-10-26T08:18:10.731654Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"fig(15, 6)\nggplot(bh, aes(x = reorder(action, n_events), y = n_events)) +\n geom_col(fill = \"steelblue\") +\n coord_flip() +\n labs(x = NULL, y = \"Number of events\") +\n theme_bw(base_size = 22)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-26T08:18:10.739129Z","iopub.execute_input":"2025-10-26T08:18:10.741085Z","iopub.status.idle":"2025-10-26T08:18:11.013611Z","shell.execute_reply":"2025-10-26T08:18:11.010794Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"ggplot(bh, aes(x = reorder(action, mean_sec), y = mean_sec)) +\n geom_col(fill = \"tomato\") +\n coord_flip() +\n labs(x = NULL, y = \"Mean duration (sec)\") +\n theme_bw(base_size = 22)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-26T08:18:11.017921Z","iopub.execute_input":"2025-10-26T08:18:11.019340Z","iopub.status.idle":"2025-10-26T08:18:11.327594Z","shell.execute_reply":"2025-10-26T08:18:11.324791Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"ggplot(bh, aes(x = n_events, y = mean_sec, size = n_train_ann_videos)) +\n geom_point(alpha = 0.7) +\n geom_text(aes(label = action), hjust = 1, vjust = 1, size = 5) +\n scale_x_log10() + scale_y_log10() +\n labs(x = \"Number of events (log)\", y = \"Mean duration (sec, log)\",\n size = \"Videos\") +\n theme_bw(base_size = 22)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-26T08:18:11.332005Z","iopub.execute_input":"2025-10-26T08:18:11.333487Z","iopub.status.idle":"2025-10-26T08:18:11.712297Z","shell.execute_reply":"2025-10-26T08:18:11.709566Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"**Pair actions**","metadata":{}},{"cell_type":"code","source":"bh <- train_ann[\n agent_id != target_id,\n .(video_file, action, num_frames = stop_frame - start_frame + 1L)\n]\nbh[dt_meta, on = .(video_file), `:=` (fps = i.frames_per_second, lab_id = i.lab_id)]\nbh[, duration := num_frames / fps]\nbh <- bh[, .(\n n_events = .N,\n n_train_ann_videos = uniqueN(video_file),\n n_labs = uniqueN(lab_id),\n min_sec = round(min(duration), 2),\n mean_sec = round(mean(duration), 2),\n max_sec = round(max(duration), 2),\n sum_sec = round(sum(duration), 2)\n), by = .(action)]\n\nbh[order(-n_events, -n_train_ann_videos)]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-26T08:18:11.716492Z","iopub.execute_input":"2025-10-26T08:18:11.718391Z","iopub.status.idle":"2025-10-26T08:18:11.799251Z","shell.execute_reply":"2025-10-26T08:18:11.796740Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"fig(15, 10)\nggplot(bh, aes(x = reorder(action, n_events), y = n_events)) +\n geom_col(fill = \"steelblue\") +\n coord_flip() +\n labs(x = NULL, y = \"Number of events\") +\n theme_bw(base_size = 22)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-26T08:18:11.803157Z","iopub.execute_input":"2025-10-26T08:18:11.804521Z","iopub.status.idle":"2025-10-26T08:18:12.145726Z","shell.execute_reply":"2025-10-26T08:18:12.143784Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"ggplot(bh, aes(x = reorder(action, mean_sec), y = mean_sec)) +\n geom_col(fill = \"tomato\") +\n coord_flip() +\n labs(x = NULL, y = \"Mean duration (sec)\") +\n theme_bw(base_size = 22)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-26T08:18:12.148389Z","iopub.execute_input":"2025-10-26T08:18:12.149829Z","iopub.status.idle":"2025-10-26T08:18:12.480166Z","shell.execute_reply":"2025-10-26T08:18:12.478219Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"ggplot(bh, aes(x = n_events, y = mean_sec, size = n_train_ann_videos)) +\n geom_point(alpha = 0.7) +\n geom_text(aes(label = action), hjust = 1, vjust = 1, size = 5) +\n scale_x_log10() + scale_y_log10() +\n labs(x = \"Number of events (log)\", y = \"Mean duration (sec, log)\",\n size = \"Videos\") +\n theme_bw(base_size = 22)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-26T08:18:12.482914Z","iopub.execute_input":"2025-10-26T08:18:12.484378Z","iopub.status.idle":"2025-10-26T08:18:12.905051Z","shell.execute_reply":"2025-10-26T08:18:12.903169Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"
\n πŸ’‘ Note: \n \n
","metadata":{}},{"cell_type":"code","source":"bh <- train_ann[, .(video_file, action)][, unique(.SD)]\nbh[dt_meta, on = .(video_file), lab_id := i.lab_id][, video_file := NULL]\n\nbh[, action_type := fifelse(action %in% self_actions, \"self actions\", \"pair actions\")]\nbh <- dcast(\n bh, lab_id ~ action_type, value.var = \"action\",\n fun.aggregate = \\(x) paste(sort(unique(x)), collapse = \", \")\n)\nbh[order(`pair actions`, `self actions`)]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-26T08:18:12.907805Z","iopub.execute_input":"2025-10-26T08:18:12.909323Z","iopub.status.idle":"2025-10-26T08:18:12.974941Z","shell.execute_reply":"2025-10-26T08:18:12.973208Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"Now let's see which actions were annotated in the test video (the one example of the ~200 hidden videos):","metadata":{}},{"cell_type":"code","source":"test_actions <- test_meta$behaviors_labeled\ntest_actions <- gsub('\"\"','', gsub(\"^\\\\[|\\\\]$\", \"\", test_actions), fixed = TRUE)\nparts <- unlist(strsplit(test_actions, \", \"))\ndt_actions <- as.data.table(tstrsplit(parts, \",\", fixed = TRUE))\nsetnames(dt_actions, c(\"agent_id\", \"target_id\", \"action\"))\ndt_actions[, c(\"agent_id\", \"target_id\", \"action\") := lapply(.SD, trimws)]\n\nhead(dt_actions)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-26T08:18:12.977717Z","iopub.execute_input":"2025-10-26T08:18:12.979322Z","iopub.status.idle":"2025-10-26T08:18:13.026846Z","shell.execute_reply":"2025-10-26T08:18:13.024758Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"dt_actions[\n , .(actions = paste(sort(action), collapse = \",\")),\n by = .(agent_id, target_id)\n]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-26T08:18:13.029440Z","iopub.execute_input":"2025-10-26T08:18:13.030824Z","iopub.status.idle":"2025-10-26T08:18:13.058508Z","shell.execute_reply":"2025-10-26T08:18:13.056689Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"
\n πŸ’‘ Note: If behaviors_labeled is a per-video whitelist of what’s eligible to be annotated, it seems for the test videos we'll have a limited set of actions to predict for each agent-target pair. If the hidden test also include this meta-info, the task becomes easier - we only need to decide when those actions in behaviors_labeled happen, not which actions exist at all (and yes, the hosts confirmed it is so).\n
","metadata":{}},{"cell_type":"markdown","source":"# Action frames per video","metadata":{}},{"cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Arena","metadata":{}},{"cell_type":"markdown","source":"Number of videos using specific arena shape and type:","metadata":{}},{"cell_type":"code","source":"names(dt_meta)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-26T08:18:13.061174Z","iopub.execute_input":"2025-10-26T08:18:13.062560Z","iopub.status.idle":"2025-10-26T08:18:13.078456Z","shell.execute_reply":"2025-10-26T08:18:13.076725Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"dt <- dt_meta[\n , .(lab_id, video_id, arena_shape, arena_type, arena_width_cm, arena_height_cm)\n]\ndt[, .N, by = .(arena_shape, arena_type)][order(-N)]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-26T08:18:13.081089Z","iopub.execute_input":"2025-10-26T08:18:13.082528Z","iopub.status.idle":"2025-10-26T08:18:13.109675Z","shell.execute_reply":"2025-10-26T08:18:13.108002Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"By lab:","metadata":{}},{"cell_type":"code","source":"dt[, .N, by = .(lab_id, arena_shape, arena_type, arena_width_cm, arena_height_cm)\n ][order(lab_id, -N)]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-26T08:18:13.112222Z","iopub.execute_input":"2025-10-26T08:18:13.113582Z","iopub.status.idle":"2025-10-26T08:18:13.164997Z","shell.execute_reply":"2025-10-26T08:18:13.162732Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"
\n πŸ’‘ Note: All labs use always the same arena shape. Only SparklingTapir use different arena types and sizes in their video, ElegantMink use two different sizes of the arena, while all other labs are consistent and stick to a single arena configuration. This means that for every lab except SparklingTapir/ElegantMink, the arena setup is effectively a proxy for lab_id.\n
","metadata":{}},{"cell_type":"code","source":"miss <- dt[, .(\n miss_type = sum(is.na(arena_type) | arena_type == \"\"),\n miss_type_pct = round(\n 100 * sum(is.na(arena_type) | arena_type == \"\") / .N, 1\n )\n), by = lab_id]\nmiss[miss_type > 0][order(-miss_type_pct)]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-26T08:18:13.168996Z","iopub.execute_input":"2025-10-26T08:18:13.170624Z","iopub.status.idle":"2025-10-26T08:18:13.195363Z","shell.execute_reply":"2025-10-26T08:18:13.193631Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"
\n πŸ’‘ Note: SparklingTapir lab did not provide arena type for 10 (14.5%) of their videos\n
","metadata":{}},{"cell_type":"markdown","source":"#

πŸ€ Mouse-level data

","metadata":{}},{"cell_type":"markdown","source":"To analyze mice consistently across videos and labs, I reshape these columns into a long + tidy representation, where each row corresponds to one mouse in one video, with attributes like strain, color, sex, age, and condition.","metadata":{}},{"cell_type":"code","source":"mouse_cols <- grep(\"^mouse\\\\d+_\", names(dt_meta), value = TRUE)\ndt_meta[,(mouse_cols) := lapply(.SD, as.character), .SDcols = mouse_cols]\n\ndt_mice <- melt(\n dt_meta,\n id.vars = c(\"lab_id\", \"video_id\", \"dataset\"),\n measure.vars = mouse_cols,\n variable.name = \"mouse_var\",\n value.name = \"value\",\n variable.factor = FALSE\n)\ndt_mice <- dt_mice[!(is.na(value) | value == \"\")]\n\ndt_mice[, mouse_id := as.integer(sub(\"^mouse(\\\\d+)_.*\", \"\\\\1\", mouse_var))]\ndt_mice[, mouse_var := sub(\"^mouse\\\\d+_\", \"\", mouse_var)]\n\ndt_mice <- dcast(dt_mice, ... ~ mouse_var, value.var = \"value\")\nhead(dt_mice)","metadata":{"trusted":true,"_kg_hide-input":false,"execution":{"iopub.status.busy":"2025-10-26T08:18:13.197949Z","iopub.execute_input":"2025-10-26T08:18:13.199365Z","iopub.status.idle":"2025-10-26T08:18:13.420986Z","shell.execute_reply":"2025-10-26T08:18:13.419265Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"Let's check whether id is a consistent mouse identifier? If an id shows up in multiple videos but with conflicting attributes, it cannot be a global identifier.","metadata":{}},{"cell_type":"code","source":"by_mouse <- dt_mice[, .(\n n_videos = uniqueN(video_id),\n n_sex = uniqueN(sex[!is.na(sex)]),\n n_color = uniqueN(color[!is.na(color)]),\n n_strain = uniqueN(strain[!is.na(strain)]),\n n_age = uniqueN(age[!is.na(age)]),\n n_cond = uniqueN(condition[!is.na(condition)])\n), by = .(lab_id, id)]\n\nsuspect <- by_mouse[\n !is.na(id) & n_videos > 1 &\n (n_sex > 1 | n_color > 1 | n_strain > 1 | n_age > 1 | n_cond > 1)\n][order(-n_videos)]\n\nhead(suspect)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-26T08:18:13.423572Z","iopub.execute_input":"2025-10-26T08:18:13.424951Z","iopub.status.idle":"2025-10-26T08:18:13.520891Z","shell.execute_reply":"2025-10-26T08:18:13.519187Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"This suggests that id is not a reliable unique identifier πŸ˜… and should not be used as a global key. Actually it seems redundant/useless, as sometimes even is missing.","metadata":{}},{"cell_type":"markdown","source":"At this point I noticed I really like the labs’ pseudonyms! So let’s make it even more fun.\nNext, we continue with per-video statistics:\n\n- how many mice are tracked in each video,\n- whether all mice in a video share the same sex, color, age, or strain,\n- and whether any of these fields are missing.","metadata":{"_kg_hide-input":false}},{"cell_type":"code","source":"dt_mice[, lab_id := vapply(lab_id, add_emojis, character(1))]\nvideo_stats <- dt_mice[, .(\n n_mice = .N,\n same_sex = length(unique(sex[!is.na(sex)])) == 1,\n same_color = length(unique(color[!is.na(color)])) == 1,\n same_age = length(unique(age[!is.na(age)])) == 1,\n same_strain = length(unique(strain[!is.na(strain)])) == 1,\n na_sex = any(is.na(sex)),\n na_color = any(is.na(color)),\n na_age = any(is.na(age)),\n na_strain = any(is.na(strain))\n), by = .(lab_id, dataset, video_id)]","metadata":{"trusted":true,"_kg_hide-input":false,"execution":{"iopub.status.busy":"2025-10-26T08:18:13.523500Z","iopub.execute_input":"2025-10-26T08:18:13.524873Z","iopub.status.idle":"2025-10-26T08:18:19.880759Z","shell.execute_reply":"2025-10-26T08:18:19.878749Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"And aggregate the per-video stats by lab to see how different the labs are.\nFor each lab I summarize:\n\n- total number of videos and the min/max number of mice\n- how often all mice share the same sex/color/age/strain (counts)","metadata":{}},{"cell_type":"code","source":"lab_summary <- video_stats[, {\n n_vid <- .N\n .(\n n_videos = n_vid,\n mice_min = min(n_mice),\n mice_max = max(n_mice),\n same_sex = fmt(sum(same_sex), n_vid),\n same_color = fmt(sum(same_color), n_vid),\n same_age = fmt(sum(same_age), n_vid),\n same_strain = fmt(sum(same_strain), n_vid)\n )\n}, by = .(lab_id, dataset)]\n\nmissing_summary <- video_stats[, {\n n_vid <- .N\n .(\n n_videos = n_vid,\n full_data = fmt(sum(!na_sex & !na_color & !na_age & !na_strain), n_vid),\n missing_sex = fmt(sum(na_sex), n_vid),\n missing_color = fmt(sum(na_color), n_vid),\n missing_age = fmt(sum(na_age), n_vid),\n missing_strain = fmt(sum(na_strain), n_vid),\n missing_all = fmt(sum(na_sex & na_color & na_age & na_strain), n_vid)\n )\n}, by = .(lab_id, dataset)]","metadata":{"trusted":true,"_kg_hide-input":false,"execution":{"iopub.status.busy":"2025-10-26T08:18:19.883593Z","iopub.execute_input":"2025-10-26T08:18:19.885065Z","iopub.status.idle":"2025-10-26T08:18:19.906661Z","shell.execute_reply":"2025-10-26T08:18:19.904778Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"lab_summary[order(-n_videos)]","metadata":{"trusted":true,"_kg_hide-input":false,"execution":{"iopub.status.busy":"2025-10-26T08:18:19.910104Z","iopub.execute_input":"2025-10-26T08:18:19.911542Z","iopub.status.idle":"2025-10-26T08:18:19.955789Z","shell.execute_reply":"2025-10-26T08:18:19.953894Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"
\n πŸ’‘ Note, we can see strong contrasts across labs:\n \n
","metadata":{"execution":{"iopub.status.busy":"2025-09-19T23:12:12.632671Z","iopub.execute_input":"2025-09-19T23:12:12.634281Z","iopub.status.idle":"2025-09-19T23:12:12.645592Z","shell.execute_reply":"2025-09-19T23:12:12.643523Z"}}},{"cell_type":"markdown","source":"And what about data completeness: how many videos have complete mouse attributes vs. missing sex/color/age/strain (and cases missing all)?","metadata":{}},{"cell_type":"code","source":"missing_summary[order(-n_videos)]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-26T08:18:19.966832Z","iopub.execute_input":"2025-10-26T08:18:19.968902Z","iopub.status.idle":"2025-10-26T08:18:20.017405Z","shell.execute_reply":"2025-10-26T08:18:20.015535Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"
\n πŸ’‘ Note:\n \n
","metadata":{}},{"cell_type":"markdown","source":"We can already see which labs are likely to produce easier vs. harder data for behavior modeling. Let’s draw this out!","metadata":{}},{"cell_type":"code","source":"lab_summary <- video_stats[, .(\n n_videos = .N,\n mice_min = min(n_mice),\n mice_max = max(n_mice),\n same_sex = sum(same_sex),\n same_color = sum(same_color),\n same_age = sum(same_age),\n same_strain = sum(same_strain)\n), by = .(lab_id, dataset)]\n\nplot_data <- melt(\n lab_summary,\n id.vars = c(\"lab_id\",\"dataset\",\"n_videos\"),\n measure.vars = c(\"same_sex\",\"same_color\",\"same_age\",\"same_strain\"),\n variable.name = \"attribute\",\n value.name = \"count\"\n)\nplot_data[, perc := 100 * count / n_videos]\nplot_data[dataset == \"test\", lab_id := paste(lab_id, \"TEST\")]","metadata":{"trusted":true,"_kg_hide-input":true,"execution":{"iopub.status.busy":"2025-10-26T08:18:20.020070Z","iopub.execute_input":"2025-10-26T08:18:20.021481Z","iopub.status.idle":"2025-10-26T08:18:20.044152Z","shell.execute_reply":"2025-10-26T08:18:20.042377Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"fig(15, 15)\nggplot(\n plot_data, aes(x = attribute, y = perc, fill = attribute)\n) +\ngeom_col() +\nfacet_wrap(~lab_id, ncol = 4) +\ncoord_flip() +\nlabs(y = \"% of videos with all mice same\", x = NULL,\n title = \"Consistency of mouse attributes across labs\") +\ntheme_bw(base_size = 22) +\ntheme(legend.position = \"none\")","metadata":{"trusted":true,"_kg_hide-input":true,"execution":{"iopub.status.busy":"2025-10-26T08:18:20.046762Z","iopub.execute_input":"2025-10-26T08:18:20.048243Z","iopub.status.idle":"2025-10-26T08:18:22.150490Z","shell.execute_reply":"2025-10-26T08:18:22.147732Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"missing_summary <- video_stats[, .(\n n_videos = .N,\n missing_sex = sum(na_sex),\n missing_color = sum(na_color),\n missing_age = sum(na_age),\n missing_strain = sum(na_strain)\n), by = .(lab_id, dataset)]\n\nplot_data <- melt(\n missing_summary,\n id.vars = c(\"lab_id\",\"dataset\",\"n_videos\"),\n measure.vars = c(\"missing_sex\",\"missing_color\",\"missing_age\",\"missing_strain\"),\n variable.name = \"attribute\",\n value.name = \"count\"\n)\n\nplot_data[, perc := 100 * count / n_videos]\nplot_data[dataset == \"test\", lab_id := paste(lab_id, \"TEST\")]","metadata":{"trusted":true,"_kg_hide-input":true,"execution":{"iopub.status.busy":"2025-10-26T08:18:22.154022Z","iopub.execute_input":"2025-10-26T08:18:22.155984Z","iopub.status.idle":"2025-10-26T08:18:22.181714Z","shell.execute_reply":"2025-10-26T08:18:22.179908Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"fig(15, 15)\nggplot(\n plot_data, aes(x = attribute, y = perc, fill = attribute)\n) +\n geom_col() +\n facet_wrap(~lab_id, ncol = 4) +\n coord_flip() +\n labs(y = \"% of videos with missing mouse attributes\", x = NULL,\n title = \"Missing mouse-level metadata across labs\") +\n theme_bw(base_size = 22) +\n theme(legend.position = \"none\")","metadata":{"trusted":true,"_kg_hide-input":true,"execution":{"iopub.status.busy":"2025-10-26T08:18:22.184445Z","iopub.execute_input":"2025-10-26T08:18:22.186000Z","iopub.status.idle":"2025-10-26T08:18:24.309992Z","shell.execute_reply":"2025-10-26T08:18:24.307143Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"(same data and same interpretaion as for the tables above, the plots are just for visual expection)","metadata":{"_kg_hide-input":false}}]}