{ "fetched_at": "2026-10-19T19:10:47Z", "source_url": "https://www.kaggle.com/competitions/aacc-2023-helpwithhemolysis/data", "pages": [ { "name": "data-description", "content": "The dataset is a real, de-identified clinical dataset consisting of results of all hemolysis measurements produced at a large, academic, tertiary care center over one month. Additional features include the collector ID, basic patient demographics, metadata related to the care setting, the time of day, and the day of the month. \n\nThe training data contains the collector ID and relevance scores for a random subset of the collectors. The relevance scores were determined as the expected cost savings to be realized over the following year resulting from targeting each collector for hemolysis prevention training. Additional details related to the calculation of collector relevance can be found in the evaluation description under the overview tab.\n\nThe test data contains a randomly ordered list of the remaining collectors' collector IDs. Your task is to order this list of collector IDs from most to least relevant as a target for hemolysis prevention training.\n\nThe use of the de-identified data set for this competition has been reviewed and deemed as non-human research by the Human Research Protections Office at Washington University in St. Louis (IRB ID #202301172). \n\n## Files\n\n* **hemolysis_index_results.csv** - the de-identified clinical dataset of all hemolysis index results over one month\n* **redraw_thresholds.csv** - orderables and hemolysis index redraw thresholds\n* **redraw_costs.csv** - encounter types and costs (USD) of sample redraw\n* **train.csv** - the training set of collector IDs with labels of collector relevance\n* **test.csv** - the test set of collector IDs to be ranked by the competitors\n* **sample_submission.csv** - a sample submission file in the correct format\n\n## Columns\n\n###### hemolysis_index_results.csv\n* `ORDER_ID` - code uniquely identifying a single order\n* `ORDERABLE` - laboratory orderable\n* `PATIENT_ID` - code uniquely identifying the patient providing the sample\n* `PATIENT_AGE` - patient age in years at the time of draw\n* `PATIENT_SEX` - patient sex\n* `PATIENT_RACE` - patient race\n* `NURSING_UNIT` - code uniquely identifying the nursing unit where the sample was drawn\n* `ENCOUNTER_TYPE` - patient care setting where the sample was drawn\n* `MEDICAL_SERVICE` - medical service caring for the patient at the time of draw\n* `SPECIMEN_ID` - code uniquely identifying a single specimen\n* `SPECIMEN_TYPE` - specimen type\n* `COLLECTOR_ID` - code uniquely identifying a single specimen collector\n* `DRAW_SITE` - anatomic site of specimen collection\n* `COLLECTION_DAY` - day of the month the sample was drawn on\n* `COLLECTION_TIME` - time of day the sample was drawn\n* `HEMOLYSIS_INDEX` - hemolysis index (a quantitative measure of sample hemolysis)\n\n###### redraw_thresholds.csv\n* `ORDERABLE` - laboratory orderable\n* `HI_REDRAW_THRESHOLD` - the threshold for hemolysis index above which results are suppressed for at least one of the ordered analytes and redraw is required\n\n###### redraw_costs.csv\n* `ENCOUNTER_TYPE` - patient care setting where the sample was drawn\n* `REDRAW_COST` - costs (USD) assigned to sample redraw\n\n###### train.csv\n* `SPLIT` - indicates that each row is related to the training dataset. \n* `COLLECTOR_ID` - code uniquely identifying a single specimen collector\n* `RELEVANCE` - expected total preventable hemolysis costs (in thousands of USD) over the next year \n\n###### test.csv\n* `SPLIT` - indicates that each row is related to the test dataset. \n* `COLLECTOR_ID` - code uniquely identifying a single specimen collector\n\n###### sample_submission.csv\n* `SPLIT` - indicates that each row is related to the test dataset. **Leave all values as \"TEST\" in your submission**\n* `COLLECTOR_ID` - code identifying a single specimen collector in the test set. **Your job is to order these by relevance (greatest to least)!**\n\n", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "Description", "content": "Welcome to the second annual data analytics competition co-hosted by the Informatics Section of the [Department of Pathology and Immunology of Washington University in St Louis](https://pathology.wustl.edu/) and the [AACC Data Analytics Steering Committee](https://www.aacc.org/science-and-research/data-analytics-in-laboratory-medicine). **The winning team will receive a prestigious speakership at a [2023 AACC Annual Scientific Meeting ](https://meeting.aacc.org/) scientific session, with travel support and free meeting registration included.**\n\n## The challenge\n\nThis competition aims to rank blood specimen collectors based on how much money could be saved over the following year if they are trained on how to prevent sample hemolysis. To develop your ranking, you will have access to a real, de-identified clinical dataset representing the degree of hemolysis and associated metadata for specimens collected at a single institution during a single month. \n\n## Context\n\nHemolysis is the breaking open of red blood cells leading to the release of intracellular contents such as hemoglobin and potassium. Hemolysis during specimen collection (*in vitro* hemolysis) can lead to erroneous laboratory results by causing spurious analyte elevation or assay interference. In fact, *in vitro* hemolysis has been reported to be the most common preanalytical error affecting up to 3.3% of all routine samples. Sample hemolysis can be detected by visual inspection of the sample color or by quantitative measurement of free hemoglobin concentration. \n\nTo prevent the release of unreliable test results that may harm the patient, most laboratories will not report results for significantly hemolyzed samples, requiring a new specimen to be drawn. As a result, sample hemolysis is also the leading cause of sample rejection across practice settings. While necessary for patient safety, this practice may delay critical results, decrease patient satisfaction, and increase costs to the patient and institution.\n\nThe specimen collector's technique is an essential factor in preventing sample hemolysis. Several studies have demonstrated that hemolysis rates within a unit or institution can be meaningfully reduced by providing additional training to the specimen collectors on best practices for sample collection. However, such educational interventions are costly, and the effects are often temporary. Therefore, educational interventions for preventing sample hemolysis require significant and continued investment to be effective.\n\nThis leads us to the real-world problem at hand. We can provide additional one-on-one education on hemolysis prevention to a limited number of blood specimen collectors each month, far fewer than our institution's thousands of active specimen collectors. We would like you to help us maximize the financial impact of our educational intervention by providing a model that will prioritize which collectors should be educated because doing so would save the greatest amount of money over the following year. \n\n## Significance\n\nThe models developed in this competition would enable institutions to spend their educational resources most efficiently to maximally prevent *in vitro* hemolysis to improve patient care and decrease costs.\n\n## References\n\n1. Wan Azman WN, Omar J, Koon TS, Tuan Ismail TS. Hemolyzed Specimens: Major Challenge for Identifying and Rejecting Specimens in Clinical Laboratories. Oman Med J. 2019 Mar;34(2):94-98. https://doi.org/10.5001/omj.2019.19. PMID: 30918601; PMCID: PMC6425048.\n2. Lippi, Giuseppe, Blanckaert, Norbert, Bonini, Pierangelo, Green, Sol, Kitchen, Steve, Palicka, Vladimir, Vassault, Anne J. and Plebani, Mario. \"Haemolysis: an overview of the leading cause of unsuitable specimens in clinical laboratories\" Clinical Chemistry and Laboratory Medicine, vol. 46, no. 6, 2008, pp. 764-772. https://doi.org/10.1515/CCLM.2008.170\n3. Green SF. The cost of poor blood specimen quality and errors in preanalytical processes. Clin Biochem. 2013 Sep;46(13-14):1175-9. https://doi.org/10.1016/j.clinbiochem.2013.06.001. Epub 2013 Jun 14. PMID: 23769816.\n4. Heireman L, Van Geel P, Musger L, Heylen E, Uyttenbroeck W, Mahieu B. Causes, consequences and management of sample hemolysis in the clinical laboratory. Clin Biochem. 2017 Dec;50(18):1317-1322. https://doi.org/10.1016/j.clinbiochem.2017.09.013. Epub 2017 Sep 22. PMID: 28947321.\n\n## Acknowledgements\n\nThis competition was jointly developed and hosted by the Informatics Section of the [Department of Pathology and Immunology of Washington University in St Louis](https://pathology.wustl.edu/) and the [AACC Data Analytics Steering Committee](https://www.aacc.org/science-and-research/data-analytics-in-laboratory-medicine). The use of the de-identified clinical data set for this educational activity has been reviewed and deemed as non-human research by the Human Research Protections Office at Washington University in St. Louis (IRB ID #202301172).\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F9966537%2Fe0cd732b89d8caddcbb0a64c71119a9f%2FAknowledgements.png?generation=1676306354820837&alt=media)\n", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "Evaluation", "content": "

The metric to be used in the challenge is the Normalized Discounted Cumulative Gain (NDCG) at k, with k=50.

\n\n

NCDG@50 is a metric of 'goodness of ranking'. For the Help with Hemolysis challenge, NCDG@50 measures if the 50 most relevant collectors to target with our educational intervention occupy the first 50 spots in a ranked list and if they are ordered appropriately from most relevant to least relevant. A perfect ranking will have an NCDG@50 of 1. The worst rankings will have an NCDG@50 of zero.

\n\n

Assignment of collector relevance

\n

To assign the relevance of each collector for the Help with Hemolysis task, we have modeled how much money could be saved over one year by targeting that collector with our educational intervention. The development of this cost-savings model is described below:

\n\n
Benchmarking data collection
\n

We collected a benchmark dataset spanning the full year immediately following the one-month training epoch.

\n\n
Estimating 'untrained' hemolysis costs per collector
\n

To estimate the annual costs attributable to sample hemolysis by each collector without additional training, we added up the costs of all their hemolyzed samples within the benchmark. To do so, we assume that:

\n\n
    \n
  1. All samples with a hemolysis index above the threshold value for at least one orderable ordered on that sample must be redrawn and will incur additional costs. See redraw_thresholds.csv on the data tab for the thresholds for the different orderables in the dataset.
    2. \n
  2. Redraw costs depend on the patient encounter type (Outpatient vs Inpatient vs Emergency). Please look at the table below or redraw_costs.csv on the data tab for the redraw costs of these different encounter types.
    3. \n
\n\n
\n\n \n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
SettingCost per hemolyzed sampleReference
Inpatient (general)$357.15Green 2013 Clin Biochem
Inpatient (critical care)$162.18Green 2013 Clin Biochem
Outpatient$337.05Green 2013 Clin Biochem
Emergency$600.00Phelan 2021 J Appl Lab Med
\n
\n\n
\n\n
Estimating 'trained' hemolysis costs per collector
\n

To estimate what the annual hemolysis costs per collector could have been had that collector been trained, we assumed that underperforming collectors would perform similarly to their nurse unit peers after retraining.

\n\n

To implement this model, we simulate similar-to-peer performance for each collector by randomly sampling, without replacement, the set of all specimens obtained on each nursing unit n times, where n represented the number of samples obtained by the collector of interest on that nursing unit in the benchmark. This process was repeated for all nursing units to generate a simulated performance after training for the collector. The simulated performance is converted into a cost in US dollars using the cost model described above. Finally, the simulation process was repeated ten times, and the median cost over these random trials was used to estimate the collector's 'trained' hemolysis costs.

\n\n
Assigning collector relevance
\n

Finally, the annual cost saving achieved by training a collector is taken as the difference between the untrained and trained costs for that collector.

\n\n

  Annual Cost Savings (thousands of USD) = Untrained Costs (thousands of USD) - Trained Costs (thousands of USD)

\n\n

For any collector with positive cost savings, the annual cost savings was taken as their relevance for ranking. Any collector with negative cost savings was assigned zero relevance, i.e., we assumed that training is unlikely to impact performance negatively.

\n\n

In summary, the most relevant collectors for our ranking task are those for whom training on hemolysis prevention is expected to produce the greatest amount of cost savings over a one-year term.

\n", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "rules", "content": " - Don't cheat!\n - Publish your code to be eligible to win the prize!\n - Apply yourself!\n - Have fun!", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "Timeline", "content": "
\n\n\nAll deadlines are at 11:59 PM UTC on the indicated day. The competition organizers reserve the right to update the contest timeline if necessary.\n
", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "Code Requirements", "content": "

Let's learn from each other

\n

Publishing your code openly will enable others to learn from your approach to applying data analytics in laboratory practice.

\n

To encourage competitors to share code, we require that teams publish their source code to the Kaggle page by March 22, 2023, for eligibility to win the travel support and AACC 2023 ASM speakership.

", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "Prizes", "content": "The winning team will get to send a representative to present at the [2023 Annual Scientific Meeting & Clinical Expo](https://meeting.aacc.org/), July 23-27, 2023. \n\n##### Support from AACC will include:\n- Speakership at a Scientific Session\n- Financial support for travel to Anaheim, California, as described in the [Speaker Benefits](https://meeting.aacc.org/call-for-proposals/submitting-a-proposal#benefits) section of the proposal site\n- Free meeting registration\n\nTo be eligible for this support, you must:\n1. Publish your source code to the Kaggle page by March 22, 2023\n2. Provide a completed conflict of interest disclosure statement to AACC by March 31, 2023\n\n", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false } ] }