{
  "id": 438862,
  "title": "Use polars to speed up by nearly 5x while reading parquet ⚡",
  "url": "/competitions/child-mind-institute-detect-sleep-states/discussion/438862",
  "author_name": "",
  "post_date": "2023-09-12T19:59:18.143927700Z",
  "votes": 10,
  "comment_count": 3,
  "views": 0,
  "content": "<p>As we already know Polars is a new data reading and manipulation. So, I thought of trying polars as pandas was taking 10-15 seconds to process and it was 5 times faster than pandas.</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5324499%2F4e19e0fc5938da8260f3eeb1c041fe4b%2FScreenshot%202023-09-13%20012216.png?generation=1694548681206038&amp;alt=media\" alt=\"\"></p>",
  "messages": [
    {
      "id": "2435240",
      "postDate": "09/12/2023 19:59:18",
      "content": "<p>As we already know Polars is a new data reading and manipulation. So, I thought of trying polars as pandas was taking 10-15 seconds to process and it was 5 times faster than pandas.</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5324499%2F4e19e0fc5938da8260f3eeb1c041fe4b%2FScreenshot%202023-09-13%20012216.png?generation=1694548681206038&amp;alt=media\" alt=\"\"></p>",
      "rawMarkdown": "As we already know Polars is a new data reading and manipulation. So, I thought of trying polars as pandas was taking 10-15 seconds to process and it was 5 times faster than pandas.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5324499%2F4e19e0fc5938da8260f3eeb1c041fe4b%2FScreenshot%202023-09-13%20012216.png?generation=1694548681206038&alt=media)",
      "votes": null
    },
    {
      "id": "2435414",
      "postDate": "09/13/2023 00:41:23",
      "content": "<p>Thank you for sharing <a href=\"https://www.kaggle.com/warcoder\" target=\"_blank\">@warcoder</a>. Why is the memory of polars about twice that of pandas?</p>",
      "rawMarkdown": "Thank you for sharing @warcoder. Why is the memory of polars about twice that of pandas?",
      "votes": null
    },
    {
      "id": "2438375",
      "postDate": "09/14/2023 08:14:08",
      "content": "<p><a href=\"https://www.kaggle.com/warcoder\" target=\"_blank\">@warcoder</a> please peruse my baseline EDA work to implement this and much more with polars. </p>",
      "rawMarkdown": "warcoder please peruse my baseline EDA work to implement this and much more with polars.",
      "votes": null
    },
    {
      "id": "2447944",
      "postDate": "09/20/2023 10:28:17",
      "content": "<p>Cool insights, <a href=\"https://www.kaggle.com/warcoder\" target=\"_blank\">@warcoder</a> ! Have you done benchmarking of reading with Polars <strong>+ convert it into Pandas DataFrame</strong>? Is it the same 2s-ish if we do that?</p>",
      "rawMarkdown": "Cool insights, @warcoder ! Have you done benchmarking of reading with Polars **+ convert it into Pandas DataFrame**? Is it the same 2s-ish if we do that?",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2435414,
      "author_name": "mitsuyasuhoshino",
      "author_url": "",
      "post_date": "09/13/2023 00:41:23",
      "content": "<p>Thank you for sharing <a href=\"https://www.kaggle.com/warcoder\" target=\"_blank\">@warcoder</a>. Why is the memory of polars about twice that of pandas?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2438375,
      "author_name": "ravi20076",
      "author_url": "",
      "post_date": "09/14/2023 08:14:08",
      "content": "<p><a href=\"https://www.kaggle.com/warcoder\" target=\"_blank\">@warcoder</a> please peruse my baseline EDA work to implement this and much more with polars. </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2447944,
      "author_name": "rimbax",
      "author_url": "",
      "post_date": "09/20/2023 10:28:17",
      "content": "<p>Cool insights, <a href=\"https://www.kaggle.com/warcoder\" target=\"_blank\">@warcoder</a> ! Have you done benchmarking of reading with Polars <strong>+ convert it into Pandas DataFrame</strong>? Is it the same 2s-ish if we do that?</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2435240": "As we already know Polars is a new data reading and manipulation. So, I thought of trying polars as pandas was taking 10-15 seconds to process and it was 5 times faster than pandas.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5324499%2F4e19e0fc5938da8260f3eeb1c041fe4b%2FScreenshot%202023-09-13%20012216.png?generation=1694548681206038&alt=media)",
    "2435414": "Thank you for sharing @warcoder. Why is the memory of polars about twice that of pandas?",
    "2438375": "warcoder please peruse my baseline EDA work to implement this and much more with polars.",
    "2447944": "Cool insights, @warcoder ! Have you done benchmarking of reading with Polars **+ convert it into Pandas DataFrame**? Is it the same 2s-ish if we do that?"
  },
  "source": "meta"
}