{"metadata":{"kernelspec":{"name":"ir","display_name":"R","language":"R"},"language_info":{"name":"R","codemirror_mode":"r","pygments_lexer":"r","mimetype":"text/x-r-source","file_extension":".r","version":"4.4.0"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":59156,"databundleVersionId":13874099,"sourceType":"competition"}],"dockerImageVersionId":30749,"isInternetEnabled":true,"language":"r","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"
\n Data Constraints and Modeling Implications\n
","metadata":{},"attachments":{}},{"cell_type":"markdown","source":"

The goal of this notebook:

\n
\n \n
","metadata":{}},{"cell_type":"code","source":"library(data.table)\nlibrary(duckdb)\nlibrary(arrow)\nlibrary(fs)\nlibrary(ggplot2)\nlibrary(patchwork)\nlibrary(tictoc)","metadata":{"_uuid":"051d70d956493feee0c6d64651c6a088724dca2a","_execution_state":"idle","trusted":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2025-12-16T07:24:00.649229Z","iopub.execute_input":"2025-12-16T07:24:00.650970Z","iopub.status.idle":"2025-12-16T07:24:00.677690Z","shell.execute_reply":"2025-12-16T07:24:00.675564Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"fig <- function(width, heigth){\n options(repr.plot.width = width, repr.plot.height = heigth)\n}\n\nget_whitelist <- function(lab_name, train_meta) {\n dt <- train_meta[lab_id == lab_name, .(video_id, behaviors_labeled)]\n pat <- \"(?i)([[:alnum:]_]+),([[:alnum:]_]+),([a-z]+)\"\n res_list <- lapply(seq_len(nrow(dt)), function(i) {\n v <- as.character(dt[i, video_id])\n s <- dt[i, behaviors_labeled]\n \n if (is.na(s) || s == \"\") return(NULL)\n \n s2 <- gsub(\"[^[:alnum:]_,]+\", \"\", s)\n m <- gregexpr(pat, s2, perl = TRUE)\n triads <- regmatches(s2, m)[[1]]\n \n if (length(triads) == 0) return(NULL)\n \n parts <- tstrsplit(triads, \",\", fixed = TRUE)\n \n parts[[2]][tolower(parts[[2]]) == \"self\"] <- \n parts[[1]][tolower(parts[[2]]) == \"self\"]\n \n data.table(\n video_id = v,\n agent_id = as.integer(sub(\"mouse\", \"\", parts[[1]])),\n target_id = as.integer(sub(\"mouse\", \"\", parts[[2]])),\n action = tolower(parts[[3]])\n )\n })\n unique(rbindlist(res_list))\n}\n\nprint_meta_overview <- function(dt, name = \"Dataset\", n = NULL, max_width = 40) {\n cat(\"=== \", name, \" ===\\n\", sep = \"\")\n cat(nrow(dt), \"rows,\", ncol(dt), \"columns\\n\")\n cat(uniqueN(dt$lab_id), \" labs (\", uniqueN(dt$video_id), \" videos)\\n\", sep = \"\")\n if (is.null(n)) n <- dt[, .N]\n sample_dt <- dt[1:n]\n sample_dt[, body_parts_tracked := substr(body_parts_tracked, 1, max_width)]\n sample_dt[, behaviors_labeled := substr(behaviors_labeled, 1, max_width)]\n head(sample_dt, n)\n}","metadata":{"trusted":true,"_kg_hide-input":true,"execution":{"iopub.status.busy":"2025-12-16T08:53:17.309962Z","iopub.execute_input":"2025-12-16T08:53:17.311614Z","iopub.status.idle":"2025-12-16T08:53:17.329603Z","shell.execute_reply":"2025-12-16T08:53:17.328047Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"

Data overview\n

","metadata":{}},{"cell_type":"markdown","source":"MABe Challenge Social Action Recognition in Mice is a Kaggle competition on recognizing mouse behavior from pose-tracking keypoints. Given video-level trajectories of one or more mice, the task is to detect time intervals of specific self and social actions (e.g. attack, chase, sniff, rear). The data come from multiple laboratories with different setups, frame rates, tracking schemes, and annotation quality, leading to strong distributional heterogeneity across labs. Each video includes a behaviors_labeled whitelist that specifies which actions were intended to be annotated. Scoring is based on interval-level F-beta, averaged across laboratories.\n\nTrain meta data:","metadata":{}},{"cell_type":"code","source":"dir <- \"/kaggle/input/MABe-mouse-behavior-detection/\"\ntrain_meta <- fread(file.path(dir, \"train.csv\"))\ntrain_meta[, video_id := as.character(video_id)]\n\nprint_meta_overview(train_meta, \"Train\", n = 3, max_width = 40)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:24:00.706703Z","iopub.execute_input":"2025-12-16T07:24:00.708235Z","iopub.status.idle":"2025-12-16T07:24:00.764789Z","shell.execute_reply":"2025-12-16T07:24:00.762856Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"names(train_meta)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:24:00.868334Z","iopub.execute_input":"2025-12-16T07:24:00.870101Z","iopub.status.idle":"2025-12-16T07:24:00.889575Z","shell.execute_reply":"2025-12-16T07:24:00.886936Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"Train annotation files:","metadata":{}},{"cell_type":"code","source":"ann_files <- dir_ls(\n file.path(dir, \"train_annotation\"),\n recurse = TRUE, glob = \"*.parquet\"\n)\ntrain_ann <- rbindlist(\n lapply(ann_files, \\(f) {\n dt <- as.data.table(read_parquet(f))\n fn <- gsub(\"\\\\.parquet\", \"\", basename(f))\n dt[, video_file := fn]\n }),\n use.names = TRUE, fill = TRUE\n)\ntrain_ann <- train_ann[!action == \"ejaculate\"]\nsetnames(train_ann, \"video_file\", \"video_id\")\ntrain_ann[train_meta, on = .(video_id), lab_id := i.lab_id]\n\ncat(\"Number of files:\", length(ann_files))\nhead(train_ann)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:24:01.034257Z","iopub.execute_input":"2025-12-16T07:24:01.035931Z","iopub.status.idle":"2025-12-16T07:24:09.122987Z","shell.execute_reply":"2025-12-16T07:24:09.120585Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"This analysis focuses on annotated behavior intervals. Therefore, we restrict the dataset to videos that contain at least one\nannotation and are also present in the test set:","metadata":{}},{"cell_type":"code","source":"train_ann <- train_ann[!lab_id %in% c(\"CRIM13\", \"CalMS21_task2\", \"CalMS21_task1\", \"CalMS21_supplemental\")]\ncat(\"Rows:\", train_ann[, .N])","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:24:09.127108Z","iopub.execute_input":"2025-12-16T07:24:09.138181Z","iopub.status.idle":"2025-12-16T07:24:09.164825Z","shell.execute_reply":"2025-12-16T07:24:09.162553Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"train_meta <- train_meta[video_id %in% train_ann$video_id]\ncat(\"Rows:\", train_meta[, .N])","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:24:09.168884Z","iopub.execute_input":"2025-12-16T07:24:09.170322Z","iopub.status.idle":"2025-12-16T07:24:09.196268Z","shell.execute_reply":"2025-12-16T07:24:09.193919Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"train_ann[, unique(action)]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:24:09.200225Z","iopub.execute_input":"2025-12-16T07:24:09.201694Z","iopub.status.idle":"2025-12-16T07:24:09.220625Z","shell.execute_reply":"2025-12-16T07:24:09.218236Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"

Body parts\n

","metadata":{}},{"cell_type":"markdown","source":"This section describes the pose-tracking configurations present in the training set, in order to identify lab-specific constraints that must be respected during feature extraction and modeling.\n\nFirst, examine what body parts were tracked acrosss all annotated videos of interest to assess completeness and variability of pose information across laboratories:","metadata":{}},{"cell_type":"code","source":"split_body_parts <- function(x){\n x <- gsub('\"\"','', x, fixed = TRUE)\n x <- gsub(\"^\\\\[|\\\\]$\", \"\", x)\n trimws(strsplit(x, \", \", fixed = TRUE)[[1L]])\n}\nbp <- train_meta[, .(\n body_part = split_body_parts(body_parts_tracked)\n), by = .(lab_id, video_id)]\n\nn_miss <- bp[is.na(body_part) | body_part == \"\", uniqueN(video_id)]\n\ncat(sprintf(\"Unique labs: %d\\n\", bp[, uniqueN(lab_id)]))\ncat(sprintf(\"Unique videos: %d\\n\", bp[, uniqueN(video_id)]))\ncat(sprintf(\"Videos with missing body parts: %d\\n\", n_miss))\ncat(sprintf(\"Unique body parts: %d\\n\", bp[, uniqueN(body_part)]))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:24:09.223772Z","iopub.execute_input":"2025-12-16T07:24:09.225145Z","iopub.status.idle":"2025-12-16T07:24:09.286471Z","shell.execute_reply":"2025-12-16T07:24:09.284307Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"So for all videos of all labs we have data about what body parts were tracked. There are 25 different body parts in total. Now let's calculate for train videos:\n- how many annotated training videos contain each body part\n- how many different body part sets is there for each lab","metadata":{}},{"cell_type":"code","source":"stats <- bp[, .(\n n_labs = uniqueN(lab_id),\n n_train_videos = uniqueN(video_id)\n ), by = body_part\n]\nstats[order(-n_train_videos)]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:24:11.235015Z","iopub.execute_input":"2025-12-16T07:24:11.236700Z","iopub.status.idle":"2025-12-16T07:24:11.277128Z","shell.execute_reply":"2025-12-16T07:24:11.273262Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"Some body parts are nearly universal, while others are present only in a limited subset of videos and labs.","metadata":{}},{"cell_type":"code","source":"per_vid_bp <- bp[\n , .(parts = paste(sort(unique(body_part)), collapse = \", \")),\n by = .(lab_id, video_id)\n ]\nper_vid_bp[, num_bp_sets := uniqueN(parts), by = .(lab_id)]\nper_vid_bp[order(-num_bp_sets), .(lab_id, num_bp_sets)] |> unique()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:24:38.683841Z","iopub.execute_input":"2025-12-16T07:24:38.685528Z","iopub.status.idle":"2025-12-16T07:24:38.738618Z","shell.execute_reply":"2025-12-16T07:24:38.736555Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"Although most laboratories use a single, consistent set of tracked body parts, at least one lab employs multiple tracking configurations. Inspect which body-part sets are used in this case:","metadata":{}},{"cell_type":"code","source":"per_vid_bp[num_bp_sets > 1, .(lab_id, parts, num_bp_sets)] |> unique()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-17T08:28:59.608113Z","iopub.execute_input":"2025-11-17T08:28:59.609919Z","iopub.status.idle":"2025-11-17T08:28:59.633972Z","shell.execute_reply":"2025-11-17T08:28:59.632113Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"And body part sets for other labs: ","metadata":{}},{"cell_type":"code","source":"per_vid_bp[num_bp_sets == 1, .(lab_id, parts, num_bp_sets)] |> unique()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:31:38.647613Z","iopub.execute_input":"2025-12-16T07:31:38.649450Z","iopub.status.idle":"2025-12-16T07:31:38.682755Z","shell.execute_reply":"2025-12-16T07:31:38.680188Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"💡 Note: The set of tracked body parts is large and varies across videos, and some parts are rare. At the same time, many parts are similar in meaning and refer to the same anatomical regions. This leads to two practical approaches:\n\n- training separate models per lab using all available body parts, or\n- training a single model across labs using a reduced representation. The reasonable variation of such reduction is to compute two derived points per lab - an upper-body point and a lower-body point - by aggregating all relevant tracked parts (e.g. using their centroid), and to use only these two points for feature computation in a shared model.","metadata":{}},{"cell_type":"markdown","source":"

Behaviors\n

","metadata":{}},{"cell_type":"markdown","source":"Now, let's analyze annotated actions in order to quantify label coverage, class imbalance, and temporal scale of events.\nThe annotated actions I extract form the annotation files. Here I calculate:\n- the number of events per action\n- the number of unique videos and labs where each action appears\n- action durations","metadata":{}},{"cell_type":"code","source":"self_actions <- train_ann[agent_id == target_id, unique(action)]\ncat(\"Self actions:\", length(self_actions), \"\\n\")\ncat(paste(self_actions, collapse = \", \"))\n\npair_actions <- train_ann[agent_id != target_id, unique(action)]\ncat(\"\\nPair actions:\", length(pair_actions), \"\\n\")\ncat(paste(pair_actions, collapse = \", \"))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:44:53.235941Z","iopub.execute_input":"2025-12-16T07:44:53.237694Z","iopub.status.idle":"2025-12-16T07:44:53.268210Z","shell.execute_reply":"2025-12-16T07:44:53.266544Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"Self actions correspond to behaviors where the agent and the target are the same mouse:","metadata":{}},{"cell_type":"code","source":"bh <- train_ann[\n agent_id == target_id,\n .(video_id, action, num_frames = stop_frame - start_frame)\n]\nbh[train_meta, on = .(video_id), `:=` (fps = i.frames_per_second, lab_id = i.lab_id)]\nbh[, duration := num_frames / fps]\nbh <- bh[, .(\n n_events = .N,\n n_train_ann_videos = uniqueN(video_id),\n n_labs = uniqueN(lab_id),\n min_sec = round(min(duration), 2),\n mean_sec = round(mean(duration), 2),\n max_sec = round(max(duration), 2),\n sum_sec = round(sum(duration), 2)\n), by = .(action)]\n\nbh[order(-n_events, -n_train_ann_videos)]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:48:39.869198Z","iopub.execute_input":"2025-12-16T07:48:39.870877Z","iopub.status.idle":"2025-12-16T07:48:39.918505Z","shell.execute_reply":"2025-12-16T07:48:39.916808Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"p1 <- ggplot(bh, aes(x = reorder(action, n_events), y = n_events)) +\n geom_col(fill = \"steelblue\") +\n coord_flip() +\n labs(x = NULL, y = \"Number of events\") +\n theme_bw(base_size = 22)\n\np2 <- ggplot(bh, aes(x = reorder(action, mean_sec), y = mean_sec)) +\n geom_col(fill = \"tomato\") +\n coord_flip() +\n labs(x = NULL, y = \"Mean duration (sec)\") +\n theme_bw(base_size = 22)\n\nfig(25, 6)\np1 | p2","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:48:44.443522Z","iopub.execute_input":"2025-12-16T07:48:44.445157Z","iopub.status.idle":"2025-12-16T07:48:45.510180Z","shell.execute_reply":"2025-12-16T07:48:45.507571Z"},"_kg_hide-input":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"Pair actions describe social behaviors involving two distinct mice and form the majority of annotated events in the dataset:","metadata":{}},{"cell_type":"code","source":"bh <- train_ann[\n agent_id != target_id,\n .(video_id, action, num_frames = stop_frame - start_frame + 1L)\n]\nbh[train_meta, on = .(video_id), `:=` (fps = i.frames_per_second, lab_id = i.lab_id)]\nbh[, duration := num_frames / fps]\nbh <- bh[, .(\n n_events = .N,\n n_train_ann_videos = uniqueN(video_id),\n n_labs = uniqueN(lab_id),\n min_sec = round(min(duration), 2),\n mean_sec = round(mean(duration), 2),\n max_sec = round(max(duration), 2),\n sum_sec = round(sum(duration), 2)\n), by = .(action)]\n\nbh[order(-n_events, -n_train_ann_videos)]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:49:12.336089Z","iopub.execute_input":"2025-12-16T07:49:12.337680Z","iopub.status.idle":"2025-12-16T07:49:12.399024Z","shell.execute_reply":"2025-12-16T07:49:12.396483Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"p1 <- ggplot(bh, aes(x = reorder(action, n_events), y = n_events)) +\n geom_col(fill = \"steelblue\") +\n coord_flip() +\n labs(x = NULL, y = \"Number of events\") +\n theme_bw(base_size = 22)\n\np2 <- ggplot(bh, aes(x = reorder(action, mean_sec), y = mean_sec)) +\n geom_col(fill = \"tomato\") +\n coord_flip() +\n labs(x = NULL, y = \"Mean duration (sec)\") +\n theme_bw(base_size = 22)\n\nfig(25, 10)\np1 | p2","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:49:17.388145Z","iopub.execute_input":"2025-12-16T07:49:17.389850Z","iopub.status.idle":"2025-12-16T07:49:18.082116Z","shell.execute_reply":"2025-12-16T07:49:18.070358Z"},"_kg_hide-input":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"Make a list of actions annotated in each lab:","metadata":{}},{"cell_type":"code","source":"bh <- train_ann[, .(video_id, action)][, unique(.SD)]\nbh[train_meta, on = .(video_id), lab_id := i.lab_id][, video_id := NULL]\n\nbh[, action_type := fifelse(action %in% self_actions, \"self actions\", \"pair actions\")]\nbh <- dcast(\n bh, lab_id ~ action_type, value.var = \"action\",\n fun.aggregate = \\(x) paste(sort(unique(x)), collapse = \", \")\n)\nbh[order(`pair actions`, `self actions`)]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:49:40.454051Z","iopub.execute_input":"2025-12-16T07:49:40.455618Z","iopub.status.idle":"2025-12-16T07:49:40.508274Z","shell.execute_reply":"2025-12-16T07:49:40.505854Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"💡 Note: \n- there are 33 distinct action types in total, coming from 15 animal-ajective labs and 352 videos with annotations\n- the most prevalent actions are pair actions (involving 2 mice): sniff (11,366 events across 200 videos) and attack (3,067 events across 170 - notably, self-actions are not well represented and are annotated by one to 5 labs. The most prevalent is rear (4241\tevents across 69 videos)\n- overall, the distribution is highly imbalanced: a small number of behaviors dominate, while many are extremely rare","metadata":{}},{"cell_type":"markdown","source":"Actions annotated in each video (to use in analyses below):","metadata":{}},{"cell_type":"code","source":"acts_by_vid <- train_ann[, .(action, video_id)][, unique(.SD)]\nhead(acts_by_vid)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:49:49.232016Z","iopub.execute_input":"2025-12-16T07:49:49.233955Z","iopub.status.idle":"2025-12-16T07:49:49.264065Z","shell.execute_reply":"2025-12-16T07:49:49.261757Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"

Whitelist Coverage\n

","metadata":{}},{"cell_type":"markdown","source":"The behaviors_labeled field specifies which actions were intended to be annotated in each video. This whitelist is critical for interpretation, as the absence of an annotation may indicate either a true negative or unlabeled train examples (actions that were just out of the scope).\n\nLet's compare the declared whitelists with the actually observed annotations:","metadata":{}},{"cell_type":"code","source":"wl_combo_dt <- lapply(train_ann[, unique(lab_id)], \\(lab) {\n get_whitelist(lab, train_meta) \n}) |> rbindlist(use.names = TRUE)\nwl_combo_dt[train_meta, on = .(video_id\t), lab_id := i.lab_id]\nhead(wl_combo_dt[order(video_id)])","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:51:08.892995Z","iopub.execute_input":"2025-12-16T07:51:08.894943Z","iopub.status.idle":"2025-12-16T07:51:09.436313Z","shell.execute_reply":"2025-12-16T07:51:09.433900Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"First, let's see in what labs all videos have the same whitelist, and which have different whitelist by video","metadata":{}},{"cell_type":"code","source":"per_vid_acts <- wl_combo_dt[\n , .(actions = paste(sort(unique(action)), collapse = \", \")),\n by = .(lab_id, video_id)\n ]\nper_vid_acts[, num_actions_sets := uniqueN(actions), by = .(lab_id)]\nper_vid_acts[order(-num_actions_sets), .(lab_id, num_actions_sets)] |> unique()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:55:15.407042Z","iopub.execute_input":"2025-12-16T07:55:15.408840Z","iopub.status.idle":"2025-12-16T07:55:15.453863Z","shell.execute_reply":"2025-12-16T07:55:15.451983Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"per_vid_acts[num_actions_sets > 1, .(lab_id, actions, num_actions_sets)] |> unique()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:55:22.137125Z","iopub.execute_input":"2025-12-16T07:55:22.138774Z","iopub.status.idle":"2025-12-16T07:55:22.200978Z","shell.execute_reply":"2025-12-16T07:55:22.199135Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"Most laboratories use a single whitelist across all their videos. Bu several labs defines multiple distinct whitelists!\n\nNow let's find out whether there are any labs where some actions were intended to be annotated but none of those actions were actually annotated:","metadata":{}},{"cell_type":"code","source":"ann_acts <- train_ann[, .(lab_id, video_id, agent_id, target_id, action)] |> unique()\n\nmiss <- wl_combo_dt[!ann_acts, on = .(video_id, agent_id, target_id, action)]\ncat(\"N missing combos (video x agent x target x action):\", miss[, .N])\nhead(miss)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:51:09.541992Z","iopub.execute_input":"2025-12-16T07:51:09.543532Z","iopub.status.idle":"2025-12-16T07:51:09.745785Z","shell.execute_reply":"2025-12-16T07:51:09.743232Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"miss[, unique(lab_id)]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:51:09.749750Z","iopub.execute_input":"2025-12-16T07:51:09.751214Z","iopub.status.idle":"2025-12-16T07:51:09.768873Z","shell.execute_reply":"2025-12-16T07:51:09.766602Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"Actions present in `behaviors_labeled` of any video of the lab, that were newer detected in train videos:","metadata":{}},{"cell_type":"code","source":"wl_acts_by_lab <- wl_combo_dt[, .(lab_id, action)] |> unique()\nann_acts_by_lab <- train_ann[, .(lab_id, action)] |> unique()\n\nwl_acts_by_lab[!ann_acts_by_lab, on = .(lab_id, action)]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:51:09.772886Z","iopub.execute_input":"2025-12-16T07:51:09.774318Z","iopub.status.idle":"2025-12-16T07:51:09.809103Z","shell.execute_reply":"2025-12-16T07:51:09.806713Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"💡 Note: These cases indicate actions that are declared as in-scope by the whitelist but have zero positive examples in the available training annotations for the corresponding lab. In this setting, a supervised model cannot learn a lab-specific decision boundary for such\nactions from training data alone. Without cross-lab transfer or additional supervision, predicting such actions is effectively impossible in a statistically grounded sense, because the training data contain no positive evidence.","metadata":{}},{"cell_type":"markdown","source":"

Arena configurations\n

","metadata":{}},{"cell_type":"markdown","source":"In this section, I analyze how arena geometry and experimental setup relate to the set of annotated actions. The goal is to understand whether arena properties constrain which behaviors can be observed and annotated.","metadata":{}},{"cell_type":"code","source":"dt <- train_meta[\n , .(lab_id, video_id, arena_shape, arena_type, arena_width_cm, arena_height_cm)\n]\ndt <- dt[acts_by_vid, on = .(video_id)]\n\nhead(dt)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:51:09.871983Z","iopub.execute_input":"2025-12-16T07:51:09.873493Z","iopub.status.idle":"2025-12-16T07:51:09.908460Z","shell.execute_reply":"2025-12-16T07:51:09.905980Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"dt[, .(\n n_videos = .N,\n n_labs = uniqueN(lab_id),\n actions = paste(sort(unique(action)), collapse = \", \")\n ), by = .(arena_shape, arena_type)\n][order(-n_videos)]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:51:09.912568Z","iopub.execute_input":"2025-12-16T07:51:09.914028Z","iopub.status.idle":"2025-12-16T07:51:09.952137Z","shell.execute_reply":"2025-12-16T07:51:09.949551Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"By lab:","metadata":{}},{"cell_type":"code","source":"dt[\n , .N, by = .(lab_id, arena_shape, arena_type, arena_width_cm, arena_height_cm)\n][order(lab_id, -N)]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:51:09.956212Z","iopub.execute_input":"2025-12-16T07:51:09.957717Z","iopub.status.idle":"2025-12-16T07:51:09.995701Z","shell.execute_reply":"2025-12-16T07:51:09.993123Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"💡 Note: All labs use always the same arena shape. Only SparklingTapir use different arena types and sizes in their video, ElegantMink use two different sizes of the arena, while all other labs are consistent and stick to a single arena configuration. This means that for every lab except SparklingTapir/ElegantMink, the arena setup is effectively a proxy for lab_id.","metadata":{}},{"cell_type":"markdown","source":"### SparklingTapir","metadata":{}},{"cell_type":"markdown","source":"Actions intended to detect (whitelist):","metadata":{}},{"cell_type":"code","source":"LAB_NAME <- \"SparklingTapir\"","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:51:09.999805Z","iopub.execute_input":"2025-12-16T07:51:10.001322Z","iopub.status.idle":"2025-12-16T07:51:10.015379Z","shell.execute_reply":"2025-12-16T07:51:10.013616Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"st_whitelist <- get_whitelist(LAB_NAME, train_meta)\nst_wl_actions <- st_whitelist[, .(video_id, wl_action = action)][, unique(.SD)]\nst_wl_actions[,\n .(actions = paste(wl_action, collapse = \", \")),\n by = .(video_id)\n][order(actions)]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:51:10.019441Z","iopub.execute_input":"2025-12-16T07:51:10.021250Z","iopub.status.idle":"2025-12-16T07:51:10.131736Z","shell.execute_reply":"2025-12-16T07:51:10.129160Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"By explicitly comparing whitelisted actions with the actions actually detected in annotations, we assess whether arena context further restricts the effective label space:","metadata":{}},{"cell_type":"code","source":"st_dt <- dt[lab_id == LAB_NAME]\nst_dt <- st_dt[st_wl_actions, on = .(video_id)]\n\nst_dt[, .(\n n_videos = .N,\n n_labs = uniqueN(lab_id),\n actions = paste(sort(unique(action)), collapse = \", \"),\n wl_actions = paste(sort(unique(wl_action)), collapse = \", \")\n ), by = .(arena_shape, arena_type)\n][order(-n_videos)]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:51:10.142476Z","iopub.execute_input":"2025-12-16T07:51:10.144017Z","iopub.status.idle":"2025-12-16T07:51:10.178730Z","shell.execute_reply":"2025-12-16T07:51:10.176347Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"

Mouse-level data\n

","metadata":{}},{"cell_type":"markdown","source":"To analyze mice consistently across videos and labs, I reshape these columns into a long + tidy representation, where each row corresponds to one mouse in one video, with attributes like strain, color, sex, age, and condition.","metadata":{}},{"cell_type":"code","source":"mouse_cols <- grep(\"^mouse\\\\d+_\", names(train_meta), value = TRUE)\ntrain_meta[,(mouse_cols) := lapply(.SD, as.character), .SDcols = mouse_cols]\n\ndt_mice <- melt(\n train_meta,\n id.vars = c(\"lab_id\", \"video_id\"),\n measure.vars = mouse_cols,\n variable.name = \"mouse_var\",\n value.name = \"value\",\n variable.factor = FALSE\n)\ndt_mice <- dt_mice[!(is.na(value) | value == \"\")]\n\ndt_mice[, mouse_id := as.integer(sub(\"^mouse(\\\\d+)_.*\", \"\\\\1\", mouse_var))]\ndt_mice[, mouse_var := sub(\"^mouse\\\\d+_\", \"\", mouse_var)]\n\ndt_mice <- dcast(dt_mice, ... ~ mouse_var, value.var = \"value\")\nhead(dt_mice)","metadata":{"trusted":true,"_kg_hide-input":false,"execution":{"iopub.status.busy":"2025-12-16T07:51:10.182617Z","iopub.execute_input":"2025-12-16T07:51:10.184212Z","iopub.status.idle":"2025-12-16T07:51:10.237953Z","shell.execute_reply":"2025-12-16T07:51:10.236125Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"Add mice features to annotations data by agent then by target:","metadata":{}},{"cell_type":"code","source":"dt_mice[, id := NULL]\nmice_cols <- c(\"age\", \"color\", \"condition\", \"sex\", \"strain\")\n\ndt <- train_ann[, .(lab_id, video_id, agent_id, target_id, action)]\ncombo_dt <- dt[dt_mice, on = .(lab_id, video_id, agent_id = mouse_id), nomatch = 0]\nsetnames(combo_dt, mice_cols, paste0(\"ag_\", mice_cols))\n\ncombo_dt <- combo_dt[dt_mice, on = .(lab_id, video_id, target_id = mouse_id), nomatch = 0]\nsetnames(combo_dt, mice_cols, paste0(\"tg_\", mice_cols))\n\nhead(combo_dt)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:51:10.240618Z","iopub.execute_input":"2025-12-16T07:51:10.241959Z","iopub.status.idle":"2025-12-16T07:51:10.305154Z","shell.execute_reply":"2025-12-16T07:51:10.303351Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"Agent mice with actions in train_annotations but without any info in train_meta:","metadata":{}},{"cell_type":"code","source":"dt[!combo_dt, on = .(lab_id, video_id, agent_id)][, !\"action\"] |> unique()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:51:10.307867Z","iopub.execute_input":"2025-12-16T07:51:10.309278Z","iopub.status.idle":"2025-12-16T07:51:10.868770Z","shell.execute_reply":"2025-12-16T07:51:10.867040Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"Target mice with actions in train_annotations but without any info in train_meta:","metadata":{}},{"cell_type":"code","source":"dt[!combo_dt, on = .(lab_id, video_id, target_id)][, !\"action\"] |> unique()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:51:10.871353Z","iopub.execute_input":"2025-12-16T07:51:10.872699Z","iopub.status.idle":"2025-12-16T07:51:11.384262Z","shell.execute_reply":"2025-12-16T07:51:11.382394Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"💡 Note: There is a mistake in mice ID in train annotations files, but it is not clear if same present in test or how to fix it. See the related discussion","metadata":{}},{"cell_type":"markdown","source":"Collect whitelists from each video and combine with mice data the same way:","metadata":{}},{"cell_type":"code","source":"wl_combo_dt <- lapply(train_ann[, unique(lab_id)], \\(lab) {\n lab_whitelist <- get_whitelist(lab, train_meta)\n\n wl_combo_dt <- lab_whitelist[dt_mice, on = .(video_id, agent_id = mouse_id), nomatch = 0]\n setnames(wl_combo_dt, mice_cols, paste0(\"ag_\", mice_cols))\n \n wl_combo_dt <- wl_combo_dt[dt_mice, on = .(lab_id, video_id, target_id = mouse_id), nomatch = 0]\n setnames(wl_combo_dt, mice_cols, paste0(\"tg_\", mice_cols))\n}) |> rbindlist(use.names = TRUE)\n\nhead(wl_combo_dt)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:51:11.387497Z","iopub.execute_input":"2025-12-16T07:51:11.388968Z","iopub.status.idle":"2025-12-16T07:51:11.928044Z","shell.execute_reply":"2025-12-16T07:51:11.926247Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"self_actions <- combo_dt[agent_id == target_id, unique(action)]\nself_actions","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:51:11.930798Z","iopub.execute_input":"2025-12-16T07:51:11.932270Z","iopub.status.idle":"2025-12-16T07:51:11.949917Z","shell.execute_reply":"2025-12-16T07:51:11.948121Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Sex","metadata":{}},{"cell_type":"markdown","source":"This section tests whether the apparent presence of some actions depends on the sex of the acting mouse and the sex pairing of the interacting mice. The goal is to detect dataset constraints that can create spurious \"sex-specific\" patterns and can leak lab-specific annotation rules into\na model.","metadata":{}},{"cell_type":"code","source":"dt_mice[, unique(sex)]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T08:24:00.265405Z","iopub.execute_input":"2025-12-16T08:24:00.267098Z","iopub.status.idle":"2025-12-16T08:24:00.283484Z","shell.execute_reply":"2025-12-16T08:24:00.281759Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"I start by summarizing, for each paired action, the proportion of episodes observed in each sex pairing (MM, FF, MF, and missing/unknown). This identifies actions that appear exclusively within a single sex pairing in the annotated training data. The same analysis is done for self actions by mouse sex.","metadata":{}},{"cell_type":"code","source":"dt <- copy(combo_dt[!action %in% self_actions])\ndt[, ag := fifelse(is.na(ag_sex), \"(NA)\", ag_sex)]\ndt[, tg := fifelse(is.na(tg_sex), \"(NA)\", tg_sex)]\ndt[, sex_pair := paste(ag, tg, sep = \"→\")]\n\ncounts <- dt[, .N, by = .(action, sex_pair)]\ncounts[, total := sum(N), by = action]\ncounts[, action := factor(action, levels = counts[\n , .(total = unique(total)), by = action][order(-total)]$action)\n]\n\nfig(16, 10)\nggplot(counts, aes(x = action, y = N, fill = sex_pair)) +\n geom_col(position = \"fill\") +\n coord_flip() +\n scale_y_continuous(labels = scales::percent_format(accuracy = 1)) +\n labs(\n x = NULL,\n y = \"Proportion of episodes (%)\",\n title = \"Annotated paired actions by sex pair\",\n fill = \"Sex pair\"\n ) +\n theme_bw(base_size = 22)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:51:11.972340Z","iopub.execute_input":"2025-12-16T07:51:11.973844Z","iopub.status.idle":"2025-12-16T07:51:12.473847Z","shell.execute_reply":"2025-12-16T07:51:12.472044Z"},"_kg_hide-input":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"dt <- copy(combo_dt[action %in% self_actions])\ndt[, sex := fifelse(is.na(ag_sex), \"(NA)\", ag_sex)]\n\ncounts <- dt[, .N, by = .(action, sex)]\ncounts[, total := sum(N), by = action]\ncounts[, action := factor(action, levels = counts[\n , .(total = unique(total)), by = action][order(-total)]$action)\n]\n\nfig(16, 6)\nggplot(counts, aes(x = action, y = N, fill = sex)) +\n geom_col(position = \"fill\") +\n coord_flip() +\n scale_y_continuous(labels = scales::percent_format(accuracy = 1)) +\n labs(\n x = NULL,\n y = \"Proportion of episodes (%)\",\n title = \"Annotated self actions by sex\",\n fill = \"Sex\"\n ) +\n theme_bw(base_size = 22)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:51:12.476478Z","iopub.execute_input":"2025-12-16T07:51:12.477864Z","iopub.status.idle":"2025-12-16T07:51:12.994146Z","shell.execute_reply":"2025-12-16T07:51:12.991324Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"💡 Note: \n- Several paired/self actions were observed exclusively within specific sex pairings (for example, shepherd occurred only in F–F pairs, while submit, chaseattack, follow, etc. appeared only in M–M pairs, and attemptmount only in M–F pairs)\n- Importantly, this pattern reflects what was present in the recorded material, and does not imply that the behaviors are biologically restricted to those pairings\n- So we must verify whether the dataset even includes other sex-pair combinations in which those actions could have occurred","metadata":{"execution":{"iopub.status.busy":"2025-11-13T09:57:16.337330Z","iopub.execute_input":"2025-11-13T09:57:16.338867Z","iopub.status.idle":"2025-11-13T09:57:16.361994Z","shell.execute_reply":"2025-11-13T09:57:16.360367Z"}}},{"cell_type":"markdown","source":"Therefore, we next restrict attention to laboratories\nthat contain mixed-sex videos, and re-check exclusivity within this subset.","metadata":{}},{"cell_type":"code","source":"video_comp <- dt_mice[\n , .(n_male = sum(sex == \"male\", na.rm = TRUE),\n n_female = sum(sex == \"female\", na.rm = TRUE)),\n by = .(lab_id, video_id)\n]\nmixed_labs <- unique(video_comp[n_male > 0 & n_female > 0, lab_id])\nmixed_labs","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:51:12.998640Z","iopub.execute_input":"2025-12-16T07:51:13.000219Z","iopub.status.idle":"2025-12-16T07:51:13.035462Z","shell.execute_reply":"2025-12-16T07:51:13.033401Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"mixed_labs_dt <- combo_dt[lab_id %chin% mixed_labs]\n\nmixed_labs_dt[, sex_pair := fcase(\n ag_sex == \"male\" & tg_sex == \"male\", \"MM\",\n ag_sex == \"female\" & tg_sex == \"female\", \"FF\",\n ag_sex %chin% c(\"male\",\"female\") & tg_sex %chin% c(\"male\",\"female\"), \"MF\",\n default = NA_character_\n)]\n\nmixed_labs_dt <- mixed_labs_dt[!is.na(sex_pair)]\n\npair_counts <- mixed_labs_dt[\n , .N, by = .(lab_id, action, sex_pair)\n][order(lab_id, action, sex_pair)]\n\npair_types <- pair_counts[\n , .(n_pair_types = uniqueN(sex_pair),\n total_episodes = sum(N)),\n by = .(lab_id, action)\n]\n\npair_counts[\n pair_types[n_pair_types == 1],\n on = .(lab_id, action), nomatch = 0\n][order(lab_id, -N)]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:51:13.038069Z","iopub.execute_input":"2025-12-16T07:51:13.039470Z","iopub.status.idle":"2025-12-16T07:51:13.117321Z","shell.execute_reply":"2025-12-16T07:51:13.115597Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"tile_data <- merge(\n pair_counts, pair_types, by = c(\"lab_id\", \"action\"), all.x = TRUE\n)\n\ntile_data[\n n_pair_types != 1, sex_pair := \"non-exclusive\"\n]\n\ntile_data[, sex_pair := factor(\n sex_pair, c(\"MM\", \"FF\", \"MF\", \"non-exclusive\"))\n]\n\nfig(23, 6)\nggplot(tile_data, aes(x = action, y = lab_id, fill = sex_pair)) +\n geom_tile() +\n scale_fill_manual(\n values = c(MM = \"#2166ac\", FF = \"#b2182b\",\n MF = \"#4d9221\", `non-exclusive` = \"grey85\")\n ) +\n theme_bw(base_size = 22) +\n labs(x = \"Action\", y = \"Lab ID\", fill = \"Exclusive in train\") +\n theme(\n axis.text.x = element_text(angle = 45, hjust = 1, vjust = 1)\n )","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:51:13.119919Z","iopub.execute_input":"2025-12-16T07:51:13.121341Z","iopub.status.idle":"2025-12-16T07:51:13.497987Z","shell.execute_reply":"2025-12-16T07:51:13.496084Z"},"_kg_hide-input":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"💡 Note: In videos with mice of mixed sex, mount, intromit, and attemptmount were detected only between male-female pairs","metadata":{}},{"cell_type":"markdown","source":"In rodents, including mice, mounting/submit behavior can actually occur in all sex combinations (dominance, territorial, social hierarchy). But Intromit is biologically constrained (real intromission can occur only in M–F pairs). But we saw in the plot above it was annotated in F- M pairs too, let's check this:","metadata":{}},{"cell_type":"code","source":"dt <- copy(combo_dt)\ndt[, ag := fcoalesce(ag_sex, \"(NA)\")]\ndt[, tg := fcoalesce(tg_sex, \"(NA)\")]\ndt[, sex_pair := paste(ag, tg, sep = \" → \")]\n\ndt[action == \"intromit\", .(n_videos = .N), by = .(action, sex_pair, lab_id)]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:51:13.500707Z","iopub.execute_input":"2025-12-16T07:51:13.502086Z","iopub.status.idle":"2025-12-16T07:51:13.548473Z","shell.execute_reply":"2025-12-16T07:51:13.545905Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"combo_dt[action == \"intromit\" & (ag_sex != \"male\" | tg_sex != \"female\")][\n , .(n_episodes = .N), by = .(lab_id, video_id, ag_sex, tg_sex, action)\n]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:51:13.552316Z","iopub.execute_input":"2025-12-16T07:51:13.553814Z","iopub.status.idle":"2025-12-16T07:51:13.594124Z","shell.execute_reply":"2025-12-16T07:51:13.591794Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"💡 Note: Biologically, actual intromission requires a male with a penis inserting into a female, not otherwise. I would say this is rather lab-specific bug, not a new mouse sex biology. ","metadata":{"execution":{"iopub.status.busy":"2025-11-13T11:05:51.396787Z","iopub.execute_input":"2025-11-13T11:05:51.543915Z","iopub.status.idle":"2025-11-13T11:05:51.574120Z","shell.execute_reply":"2025-11-13T11:05:51.571478Z"}}},{"cell_type":"markdown","source":"Finally, I perform the same sex-pair analysis on the whitelist (behaviors_labeled). This answers a distinct question: whether the intended\nlabel scope itself depends on sex pairing. If the whitelist includes implausible pairings (e.g. intromit for MM or FM), it supports the hypothesis that the whitelist may be generated mechanically (e.g. as a Cartesian product of mouse pairs and a per-video action list) rather than being biologically filtered.\n\nThis matters for modeling: sex-pair patterns may reflect annotation protocol and metadata construction, not the underlying behavior.","metadata":{}},{"cell_type":"code","source":"dt <- copy(wl_combo_dt[!action %in% self_actions])\ndt[, ag := fifelse(is.na(ag_sex), \"(NA)\", ag_sex)]\ndt[, tg := fifelse(is.na(tg_sex), \"(NA)\", tg_sex)]\ndt[, sex_pair := paste(ag, tg, sep = \"→\")]\n\ncounts <- dt[, .N, by = .(action, sex_pair)]\ncounts[, total := sum(N), by = action]\ncounts[, action := factor(action, levels = counts[\n , .(total = unique(total)), by = action][order(-total)]$action)\n]\n\nfig(16, 10)\nggplot(counts, aes(x = action, y = N, fill = sex_pair)) +\n geom_col(position = \"fill\") +\n coord_flip() +\n scale_y_continuous(labels = scales::percent_format(accuracy = 1)) +\n labs(\n x = NULL,\n y = \"Proportion of episodes (%)\",\n title = \"Paired actions by sex pair in whitelist\",\n fill = \"Sex pair\"\n ) +\n theme_bw(base_size = 22)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:51:13.597367Z","iopub.execute_input":"2025-12-16T07:51:13.598759Z","iopub.status.idle":"2025-12-16T07:51:14.084817Z","shell.execute_reply":"2025-12-16T07:51:14.082690Z"},"_kg_hide-input":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"dt <- copy(wl_combo_dt[action %in% self_actions])\ndt[, sex := fifelse(is.na(ag_sex), \"(NA)\", ag_sex)]\n\ncounts <- dt[, .N, by = .(action, sex)]\ncounts[, total := sum(N), by = action]\ncounts[, action := factor(action, levels = counts[\n , .(total = unique(total)), by = action][order(-total)]$action)\n]\n\nfig(16, 6)\nggplot(counts, aes(x = action, y = N, fill = sex)) +\n geom_col(position = \"fill\") +\n coord_flip() +\n scale_y_continuous(labels = scales::percent_format(accuracy = 1)) +\n labs(\n x = NULL,\n y = \"Proportion of episodes (%)\",\n title = \"Self actions by sex in whitelist\",\n fill = \"Sex\"\n ) +\n theme_bw(base_size = 22)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:51:14.087838Z","iopub.execute_input":"2025-12-16T07:51:14.089396Z","iopub.status.idle":"2025-12-16T07:51:14.461434Z","shell.execute_reply":"2025-12-16T07:51:14.459566Z"},"_kg_hide-input":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"💡 Note: \n- The whitelist specifies certain actions only for specific sex-pairings\n- However, again, some of combinations appear biologically implausible (for example, intromit in F–M pairs)\n- Possible explanations: the behaviors can truly occur only in certain sex pairs/in specified pairs, or the whitelist may simply be the Cartesian product of all mice pairs and the full action list for each video, without biological filtering","metadata":{}},{"cell_type":"code","source":"dt <- copy(wl_combo_dt)\ndt[, ag := fcoalesce(ag_sex, \"(NA)\")]\ndt[, tg := fcoalesce(tg_sex, \"(NA)\")]\ndt[, sex_pair := paste(ag, tg, sep = \" → \")]\n\ndt[action == \"intromit\", .(n_videos = .N), by = .(action, sex_pair, lab_id)]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:51:14.464089Z","iopub.execute_input":"2025-12-16T07:51:14.465538Z","iopub.status.idle":"2025-12-16T07:51:14.501565Z","shell.execute_reply":"2025-12-16T07:51:14.499145Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"wl_combo_dt[action == \"intromit\" & (ag_sex != \"male\" | tg_sex != \"female\")][\n , .(n_episodes = .N), by = .(lab_id, video_id, ag_sex, tg_sex, action)\n]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:51:14.505053Z","iopub.execute_input":"2025-12-16T07:51:14.506638Z","iopub.status.idle":"2025-12-16T07:51:14.549193Z","shell.execute_reply":"2025-12-16T07:51:14.546665Z"},"_kg_hide-input":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"💡 Note: So, intromit was intended to be detected between male-male or female-male pairs too! So the second option - the whitelist may simply be the Cartesian product of all mice pairs and the full action list for each video - is more likely.","metadata":{}},{"cell_type":"markdown","source":"## Condition","metadata":{}},{"cell_type":"markdown","source":"This section examines whether the presence of annotated actions depends on the experimental condition of the mice. As with sex, the goal is to\nidentify dataset-driven constraints that may introduce condition-specific patterns into the annotations.","metadata":{}},{"cell_type":"code","source":"dt_mice[, unique(condition)]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:51:14.553240Z","iopub.execute_input":"2025-12-16T07:51:14.554803Z","iopub.status.idle":"2025-12-16T07:51:14.572880Z","shell.execute_reply":"2025-12-16T07:51:14.570531Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"We begin by identifying laboratories and videos that contain mice with mixed conditions. If a lab records only a single condition,\nany apparent condition-specific action is uninformative and simply reflects the experimental design of that lab.","metadata":{}},{"cell_type":"code","source":"video_comp <- dt_mice[\n , .(n_cond = uniqueN(condition)),\n by = .(lab_id, video_id)\n]\nmixed_labs <- unique(video_comp[n_cond > 1, lab_id])\nmixed_labs","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:51:14.576934Z","iopub.execute_input":"2025-12-16T07:51:14.578518Z","iopub.status.idle":"2025-12-16T07:51:14.615358Z","shell.execute_reply":"2025-12-16T07:51:14.612897Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"mixed_labs_dt <- combo_dt[lab_id %chin% mixed_labs]\n\nmixed_labs_dt[, ag := fifelse(is.na(ag_condition), \"(NA)\", ag_condition)]\nmixed_labs_dt[, tg := fifelse(is.na(tg_condition), \"(NA)\", tg_condition)]\nmixed_labs_dt[, condition_pair := paste(ag, tg, sep = \" → \")]\n\npair_counts <- mixed_labs_dt[\n , .N, by = .(lab_id, action, condition_pair)\n][order(lab_id, action, condition_pair)]\n\npair_types <- pair_counts[\n , .(n_pair_types = uniqueN(condition_pair),\n total_episodes = sum(N)),\n by = .(lab_id, action)\n]\n\npair_counts[\n pair_types[n_pair_types == 1],\n on = .(lab_id, action), nomatch = 0\n][order(lab_id, -N)]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:51:14.619439Z","iopub.execute_input":"2025-12-16T07:51:14.621099Z","iopub.status.idle":"2025-12-16T07:51:14.678366Z","shell.execute_reply":"2025-12-16T07:51:14.675953Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"tile_data <- merge(\n pair_counts, pair_types, by = c(\"lab_id\", \"action\"), all.x = TRUE\n)\n\ntile_data[\n n_pair_types != 1, condition_pair := \"non-exclusive\"\n]\n\ntile_data[, condition_pair := factor(condition_pair)]\n\nfig(23, 6)\nggplot(tile_data, aes(x = action, y = lab_id, fill = condition_pair)) +\n geom_tile() +\n theme_bw(base_size = 22) +\n labs(x = \"action\", y = \"lab_id\", fill = \"exclusive in train\") +\n theme(\n axis.text.x = element_text(angle = 45, hjust = 1, vjust = 1)\n )","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:51:14.682290Z","iopub.execute_input":"2025-12-16T07:51:14.683849Z","iopub.status.idle":"2025-12-16T07:51:15.082514Z","shell.execute_reply":"2025-12-16T07:51:15.079694Z"},"_kg_hide-input":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"💡 Note:\n- In videos with mice with mixed condition, mount, intromit were detected only when target is \"hormone-primed\" or agent is \"estrus\"\n- Attemptmount, and allogroom (ElegantMink only) were detected only when target mice is hormone-primed\n- The rest seems lab specific","metadata":{}},{"cell_type":"markdown","source":"We then repeat the same analysis for the whitelist (behaviors_labeled), which describes intended annotation scope. This allows us to distinguish between constraints imposed by the recorded material and constraints imposed by the annotation protocol itself.","metadata":{}},{"cell_type":"code","source":"acts <- c(\"mount\", \"intromit\", \"attemptmount\")\nwl_combo_dt[\n action %in% acts & \n (\n (!is.na(tg_condition) & tg_condition != \"hormone primed\") |\n (!is.na(ag_condition) & ag_condition != \"estrus\")\n )\n]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-16T07:51:15.132913Z","iopub.execute_input":"2025-12-16T07:51:15.134326Z","iopub.status.idle":"2025-12-16T07:51:15.188171Z","shell.execute_reply":"2025-12-16T07:51:15.185706Z"},"_kg_hide-input":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"The whitelist analysis shows that these condition-specific actions were often intended to be detected even in videos with different or mixed conditions. This suggests that condition filtering is not strictly enforced at the whitelist level and that intended label space may again be constructed mechanically rather than biologically.\n\nFrom a modeling perspective, these findings imply that mouse condition should not be treated as a hard constraint on action availability.","metadata":{}},{"cell_type":"markdown","source":"

Final conclusions\n

","metadata":{}},{"cell_type":"markdown","source":"1. Pose tracking is heterogeneous across laboratories, with a large and varying set of tracked body parts. The lack of a common body-part schema motivates either lab-specific feature extraction or the use of strongly reduced, derived representations when training models across labs.\n\n2. Analysis of action distributions revealed a highly imbalanced label space. Self-actions are sparse and often lab-specific, while pair actions provide the majority of training signal. This justifies treating self and pair actions separately and motivates careful handling of rare behaviors.\n\n3. Actions listed in the whitelist may have zero positive examples in a lab, making them unidentifiable under purely per-lab supervised training. Non-trivial prediction of such actions requires cross-lab parameter sharing or additional sources of supervision.\n\n4. Arena configuration was shown to be effectively fixed within most labs and therefore largely redundant with the laboratory identifier.\n\n5. Finally, analyses of sex and experimental condition demonstrated that many apparent \"exclusive\" patterns arise from annotation scope, experimental design, or metadata construction, rather than from true biological restrictions. Both annotations and whitelists may include biologically implausible combinations, indicating that models should not encode hard biological rules based on these fields alone.","metadata":{}}]}