{ "fetched_at": "2026-10-19T07:44:16Z", "source_url": "https://www.kaggle.com/competitions/ai-serving-grid-stability/data", "pages": [ { "name": "data-description", "content": "## Files\n\n* **train.csv** - the training set. Use this to develop and/or train your algorithm. Note that the training set does not contain any information on when anomalies are present. Nevertheless, we can assure you that there are anomalies. This is an unsupervised learning challenge🚨. \n* **test.csv** - the test set consists of 21 disjunct data segments that all have the same length (1800 time steps). Note that data segments with the same `test_data_segment_id` belong together. All but one data segments contain at least one anomaly. You are asked to spot the anomalies🔎. Your submission should only contain information if an anomaly is present at a certain time step or not. Additional Information: The test set contains the same features as the training set (except the additional `test_data_segment_id`).\n* **sample_submission.csv** - a sample submission file in the correct format.Your submission should look like this. Except that the predictions may be better 😉. Note that the `id` column of your submission matches the `id` column of the test set.\n\nA visual representation of the structure of the `train.csv` and `test.csv` as it was explained during the kick-off can be found [here](https://miro.com/app/board/uXjVND-YN_Y=/?share_link_id=813876208475). Additionally, you can find further explanations of the features in the [recording of the kick-off](https://tu-darmstadt.zoom.us/rec/play/bioWA7CnZFIXCWuUgrKvRe18gtM31VJPyIVSnBuPMvPDFfxs37OBNXjDX8uk2KHY6rknE0o5CV8jMo6J.3cr37bYPyXI3ngKK?canPlayFromShare=true&from=my_recording&continueMode=true&componentName=rec-play&originRequestUrl=https%3A%2F%2Ftu-darmstadt.zoom.us%2Frec%2Fshare%2Fpyuas5ZK8xA6v68P03OCdEr5PLyJR307We7uZWSNL17zJpFpcOqGz25yT6n_qMYS.3QusCIA4t1A74Epu%20Kenncode%3A%20a%23OS8EVq) from the minute 19:00.\n\n## Task\n\nCan you spot all time steps in which anomalies are present in the `test.csv` file? Since this is an unsupervised challenge and you have no information about what anomalies are and what they are not, we describe some examplary anomaly types in [the challenge overview](overview/example-anomalies). You can use those descriptions to double-check if the anomalies you discovered are valid.\n\n## Features\n\n| Parameter | Description | Unit |\n|------- |----------------|------- |\n| `aFRRactivation` | The automatic Frequency Restoration Reserve (aFRR) **activation** is the actual reserve capacity that has been brought online and is actively contributing to the grid's frequency restoration. | MW |\n| `correctedDemand` / `correctedAFRR` | CorrectedAFFR refers to the actual amount of Frequency Restoration Reserve that is required to maintain the grid's frequency stability. It can be interpreted as the target value of the automatic Frequency Restoration Reserve. | MW |\n|`aFRRrequest`|The automatic Frequency Restoration Reserve (aFRR) **request** is the actual requested reserve capacity that has been requested from the PICASSO system.|MW|\n|`controlBandPos` and `controlBandNeg`| Maximum or minimum of the Frequency Restoration Reserve. |MW|\n|`LFCInput`| LFCInput is the control variable of frequency restoration controllers, reflecting the imbalance of the respective control area. |MW|\n|`FRCE`| FRCE is the \"Frequency Restoration Control Error\", reflecting the system imbalance that is caused within the respective control area. |MW|\n|`Demand`| `Demand = correctedDemand - correction` |MW|\n|`correction`| `correction = correctedDemand - Demand` |MW|\n|`correctionEcho`| `correction` 4-8s shifted |MW|\n|`participationCMO`| Represents a preliminary stage of participation |Bool|\n|`participationIN`| Represents full participation |Bool|\n|`BandLimitedCorrectedDemand`| Corrected demand limited by the control bands |MW|\n|`controlArea`| ID or the control area |int|", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "Description", "content": "# Introduction\n\nEuropeans are used to a constant electricity supply and rarely experience blackouts. TransnetBW’s control room is one of the places where people are working 24/7 to maintain this seemingly inexhaustible flow. The company is one of four transmission system operators in Germany and ensures the stability of our electricity grid. To do this, the same amount of electricity must be fed into the grid as is taken out at any point in time to maintain a frequency of 50 Hertz. To ensure this, TransnetBW uses automated control signals to instruct power plant operators to ramp up or down their infeed. In this challenge, we look at the secondary reserve (a primary and a tertiary version also exists) that must be active after a maximum of 5 minutes.\n\nUntil recently, every European state used power plants in their own country to maintain the system balance. This has regularly led to the following situations: In Germany, for example, a power plant is ramped up and a few kilometers further across the border another power plant is ramped down. The costs that are incurred by this inefficient use of reserves are ultimately paid by all electricity customers. Since 2022, TransnetBW has operated the PICASSO platform that coordinates the deployment of secondary reserves in real-time, ensuring optimal usage of reserves in a European domestic balancing market. This makes the PICASSO platform the world`s largest system for real-time optimization of electricity flows. It leads to macroeconomic savings of several hundred million euros per year and contributes to the decarbonization of the balancing market.\n\nTo ensure the correct functionality of the platform, information on imbalances, reserves, transmission capacities, and many other time series are constantly monitored. These monitored features sometimes behave strangely, indicating to the control room engineers that it might be necessary to take action. Algorithms that recognize anomalies early and precisely may be of great help here. That’s why TransnetBW, in cooperation with Fraunhofer IEE, provides a previously unpublished data set consisting of various PICASSO platform time series to participants. \n\n# Challenge\n\nThis is an unsupervised learning challenge where we ask you to develop an (AI) algorithm to detect anomalies in the operation of the PICASSO platform. If this is successful, AI-based solutions could support TransnetBW and other transmission system operators in recognizing deviations from the regular operation. The challenge aims at evaluating different solution methodologies in a short period of time and establishing personal contacts between industry and AI developers. You can find a more technical problem description and the data itself in the [data tab](https://www.kaggle.com/competitions/transnetbw-test-challenge/data).\n\n# Important Links \n\n- If you are not yet registered, please register on the [registration page](https://eveeno.com/anomalydetectionineuropeanenergy) (even if the challenge has already started). \n- Get an even more detailed description of the challenge and information about the final event on our [website](https://hessian.ai/industry-challenge/).\n- View the [recording](https://tu-darmstadt.zoom.us/rec/play/bioWA7CnZFIXCWuUgrKvRe18gtM31VJPyIVSnBuPMvPDFfxs37OBNXjDX8uk2KHY6rknE0o5CV8jMo6J.3cr37bYPyXI3ngKK?canPlayFromShare=true&from=my_recording&continueMode=true&componentName=rec-play&originRequestUrl=https%3A%2F%2Ftu-darmstadt.zoom.us%2Frec%2Fshare%2Fpyuas5ZK8xA6v68P03OCdEr5PLyJR307We7uZWSNL17zJpFpcOqGz25yT6n_qMYS.3QusCIA4t1A74Epu%20Kenncode%3A%20a%23OS8EVq) of the kickoff presentations and the [documentation](https://app.mural.co/t/quantchengluckgmbhcokg7492/m/quantchengluckgmbhcokg7492/1699602119184/d836574a2883a4aeb50b480a128b16da5e299882?sender=u6ee4d51e9cc36f12620a7561) of the kickoff event.\n- Join us at [discord](https://discord.gg/Q9ZPhTQbS3) to stay up to date and to get in contact with other participants.\n- Link to Q&A session on 11.01.2024 at 4pm [here](https://tu-darmstadt.zoom-x.de/j/64122381186?pwd=TXVOMGZhUks0VlFHeDY1WXhGUnB1Zz09)\n- Link to final event on 14.02.2024 at 1-3:30pm [here](https://tu-darmstadt.zoom-x.de/j/68385147444?pwd=UmpsRUw1bHI2Z0I2WU16ZUNWM0NjQT09)\n", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "Organizers", "content": "| | | | | |\n| --- | --- |--- | --- | --- | \n| \"drawing\" | \"drawing\" | \"drawing\" | \"drawing\" | \"drawing\" |\n| \"drawing\" | \"drawing\" | \"drawing\" | \"drawing\" | | \n", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "rules", "content": "Don't cheat, be fair, be friendly, \n\nand most importantly have:\n![Alt Text](https://media.giphy.com/media/lmv5aDvwOgTmby3a13/giphy.gif)", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "abstract", "content": " ", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "Evaluation", "content": "The evaluation of the challenge will consist of 2 parts:\n\n### 1. Detection Score\nWe use the F-beta-score with a beta value set to 1.75 as the error metric. You can find more information about the F-beta-score [here](https://medium.com/@douglaspsteen/beyond-the-f-1-score-a-look-at-the-f-beta-score-3743ac2ef6e3). You can view the score of your submission based on the error metric on the leaderboard at any time. Here you can also see how well your algorithm performs compared to other teams. It is important to note that the leaderboard does not show the final score as it is only determined based on around 37% of the test data. We will use the other 63% of the test data only at the end of the challenge for a final evaluation.\n\n### 2. Best Complete Package\nWe are not only interested in the pure error score but also in your ideas. Participants are invited to present their solutions at the final event. There we will crown the winner for the best overall solution approach. The price for this category will be awarded by the participants themselves as a community decision.\n", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "Example Anomalies", "content": "Since this is an unsupervised challenge and you have no information about what anomalies are and what they are not, we will describe some examples of anomaly types. Once you have developed your algorithm and discovered the first potential anomalies, you can use the descriptions to check if your algorithm detects valid anomalies. But beware, we do not list all anomalies that are present in the data here ;) \n\nNote that you can find additional information on example anomalies in the [recording of the kick-off](https://tu-darmstadt.zoom.us/rec/play/bioWA7CnZFIXCWuUgrKvRe18gtM31VJPyIVSnBuPMvPDFfxs37OBNXjDX8uk2KHY6rknE0o5CV8jMo6J.3cr37bYPyXI3ngKK?canPlayFromShare=true&from=my_recording&continueMode=true&componentName=rec-play&originRequestUrl=https%3A%2F%2Ftu-darmstadt.zoom.us%2Frec%2Fshare%2Fpyuas5ZK8xA6v68P03OCdEr5PLyJR307We7uZWSNL17zJpFpcOqGz25yT6n_qMYS.3QusCIA4t1A74Epu%20Kenncode%3A%20a%23OS8EVq) around minute 14:00 and 23:00.\n\n# Example Anomaly 1\n\nThe values of `aFRRrequest` and `aFRRactivation`usually follow a time-delayed relationship. In this example, you can see that the values are even partially asynchronous.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1811734%2Ff853ce426a65698240c155c19cd5049a%2Fexample_anomaly1.png?generation=1702454220906675&alt=media)\n\n# Example Anomaly 2\n\nDue to transmission errors, freezes of signals can occur in some of the features.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1811734%2Fb7e732ddced53655398d85335376db9d%2Fexample_anomaly2.png?generation=1702454513763275&alt=media)\n\n# Example Anomaly 3\n\nAnomalies can also occur in the other variables, as can be seen here for `FRCE` and `LFCInput`. Yet the relationship of these two variables is unknown.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1811734%2Fe32af38e1185d1e46ca70dbc7e72ed6e%2Fexample_anomaly3.png?generation=1702454739170118&alt=media)", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "Frequently Asked Questions", "content": "## Participate as a Team or Individual?\nIt's up to you 😊 During our Kick-Off event, we will provide the possibility to create teams with other participants (if you don't have a team yet). At our Discord server, you can also search for buddies to team up with.\n\n## I Have Trouble Understanding the Data\nPlease see the detailed description of the data in the Data tab. If that is not sufficient, please reach out to us via Discord.\n\n## Is the Leaderboard Showing the Final Score?\nNo. We don't want you to overfit your solution to the leaderboard score. The leaderboard score which you see until the end of the challenge, is only determined based on 37% of the data. We will use the other 63% of the data only at the end of the challenge for a final evaluation.\n\n## I am a Student. Can I Somehow Get Credit Points for this?\nIn some studies, there is an optional module like \"research and development\". Sometimes those modules provide the possibility to work on interesting research projects. You can contact a Prof. at your uni to set this up. You can also contact us for any help or questions on this.\n\n## I am new to this Topic. Any hints on where to start?\nAnomaly Detection, also known as Outlier Detection or Novelty Detection, is a vital research field. There are plenty of books and surveys on this topic from renowned researchers available. As a starting point, you can read the following publications.\n\n## I am Hyped on the Topic and want to Continue Working on It\nGreat. That was one of our goals. Just reach out to us and we will try to find a way :)", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "Further Readings", "content": "\n###Foundations:\n- V. Chandola, A. Banerjee, and V. Kumar, \"Anomaly detection,\" ACM Comput. Surv., vol. 41, no. 3, pp. 1–58, 2009, https://doi.org/10.1145/1541880.1541882.\n- C. C. Aggarwal, Outlier Analysis. Cham: Springer International Publishing, 2017. https://doi.org/10.1007/978-3-319-47578-3\n- K. G. Mehrotra, C. K. Mohan, and H. Huang, Anomaly Detection Principles and Algorithms. Cham: Springer International Publishing, 2017. https://doi.org/10.1007/978-3-319-67526-8\n- D. Samariya and A. Thakkar, \"A Comprehensive Survey of Anomaly Detection Algorithms,\" Ann. Data. Sci., vol. 10, no. 3, pp. 829–850, 2023. https://doi.org/10.1007/s40745-021-00362-9.\n\n###AI/ML:\n- C. C. Aggarwal, Neural Networks and Deep Learning. Cham: Springer International Publishing, 2018. https://doi.org/10.1007/978-3-319-94463-0\n- Z. Z. Darban, G. I. Webb, S. Pan, C. C. Aggarwal, and M. Salehi, \"Deep Learning for Time Series Anomaly Detection: A Survey,\" Nov. 2022. https://doi.org/10.48550/arXiv.2211.05244", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "How to cite this challenge:", "content": "Please ignore the autogenerated Citation section which follows this one and **use the following Bibtex for your publications**. \n\n```\n@misc{Remppis2023,\nauthor = {Remppis, Simon and Rehwald, Florian and Rohrer, Tobias and Kaupp, Lukas},\nyear = {2023},\ntitle = {Industry Challenge: AI Serving Grid Stability},\npublisher = {Kaggle},\nurl = {https://kaggle.com/competitions/ai-serving-grid-stability}\n}\n```\n\nWe encourage all scholars to present and publish their developed methods at scientific conferences or journals.", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false } ] }