This competition often requires the latest version of Transformer, which is installed with the following command.
\n!pip install git+https://github.com/huggingface/transformers\nHowever, the inference notebook cannot pip install because the internet is turned off.
\nAt that time, you can install Transformer from Dataset by executing the following command.
1 Do the following commands at your local env for making wheel file.
\ngit https://github.com/huggingface/transformers.git\n transformers\npip install --upgrade pip setuptools wheel\npython setup.py bdist_wheel\n2 You get a file named dist/transformers-4.XX.0.dev0-py3-none-any.whl, upload it to the Kaggle Dataset.
3 Execute the following commands on the Notebook.
\n!pip install /kaggle/input/YOUR_DATASET_NAME/transformers-4.XX.0.dev0-py3-none-any.whl \nPlease let me know if there is an easier way.
", "messages": [ { "id": "2258238", "postDate": "05/14/2023 03:21:17", "content": "This competition often requires the latest version of Transformer, which is installed with the following command.
\n!pip install git+https://github.com/huggingface/transformers\nHowever, the inference notebook cannot pip install because the internet is turned off.
\nAt that time, you can install Transformer from Dataset by executing the following command.
1 Do the following commands at your local env for making wheel file.
\ngit https://github.com/huggingface/transformers.git\n transformers\npip install --upgrade pip setuptools wheel\npython setup.py bdist_wheel\n2 You get a file named dist/transformers-4.XX.0.dev0-py3-none-any.whl, upload it to the Kaggle Dataset.
3 Execute the following commands on the Notebook.
\n!pip install /kaggle/input/YOUR_DATASET_NAME/transformers-4.XX.0.dev0-py3-none-any.whl \nPlease let me know if there is an easier way.
", "rawMarkdown": "This competition often requires the latest version of Transformer, which is installed with the following command.\n```sh\n!pip install git+https://github.com/huggingface/transformers\n```\n\nHowever, the inference notebook cannot pip install because the internet is turned off. \nAt that time, you can install Transformer from Dataset by executing the following command.\n\n---\n1 Do the following commands at your local env for making wheel file.\n\n```sh\ngit clone https://github.com/huggingface/transformers.git\ncd transformers\npip install --upgrade pip setuptools wheel\npython setup.py bdist_wheel\n```\n\n2 You get a file named `dist/transformers-4.XX.0.dev0-py3-none-any.whl`, upload it to the Kaggle Dataset.\n\n3 Execute the following commands on the Notebook.\n```sh\n!pip install /kaggle/input/YOUR_DATASET_NAME/transformers-4.XX.0.dev0-py3-none-any.whl \n```\n\n---\n\nPlease let me know if there is an easier way.", "votes": null }, { "id": "2258335", "postDate": "05/14/2023 05:57:45", "content": "You can use Notebook output as well.
\nXXX) and run !pip download git+https://github.com/huggingface/transformers!pip download wheel and !pip download setuptoolsAdd Data button and run !pip uninstall transformers -y and !python -m pip install --no-index --find-links=/kaggle/input/XXX transformers@tikutiku
\nI didn't know about that method, thanks!
Note that when using the Transformer Repository in an internet-off notebook, you will need to update the file paths to point to the correct locations in the dataset.
", "rawMarkdown": "Note that when using the Transformer Repository in an internet-off notebook, you will need to update the file paths to point to the correct locations in the dataset.", "votes": null }, { "id": "2264664", "postDate": "05/18/2023 16:06:16", "content": "This method does not work for me.. When I try to load the files into a notebook I get the following error.
\n Installing build dependencies ... error\n error: subprocess-exited-with-error\n\n × pip subprocess to install build dependencies did not run successfully.\n │ exit code: 1\n ╰─> [4 lines of output]\n Looking in links: /kaggle/input/XXX/\n ERROR: Could not find a version that satisfies the requirement setuptools>=40.8.0 (from versions: none)\n ERROR: No matching distribution found for setuptools>=40.8.0\n\n [end of output]\nGood catch. I found that wheel and setuptools should be in the Notebook output as well (usually these are automatically downloaded by !pip download your-package).
I created Notebooks for demo:
\nhttps://www.kaggle.com/code/tikutiku/benetech-pip
\nhttps://www.kaggle.com/code/tikutiku/benetech-install-transformers-w-o-internet
Hope it helps.
", "rawMarkdown": "Good catch. I found that `wheel` and `setuptools` should be in the Notebook output as well (usually these are automatically downloaded by `!pip download your-package`).\n\nI created Notebooks for demo:\nhttps://www.kaggle.com/code/tikutiku/benetech-pip\nhttps://www.kaggle.com/code/tikutiku/benetech-install-transformers-w-o-internet\n\nHope it helps.", "votes": null }, { "id": "2265815", "postDate": "05/19/2023 14:28:05", "content": "Thanks for sharing this Tom. Very helpful :)
", "rawMarkdown": "Thanks for sharing this Tom. Very helpful :)", "votes": null }, { "id": "2267594", "postDate": "05/21/2023 05:36:03", "content": "Hi Bartley, I have done it many times. Why am I still getting the wrong transformers version?
\nimport transformers
\nprint(\"transformers v\" + transformers.version)
\n------output---------------
\ntransformers v4.28.1
!pip uninstall transformers -y
\n------output---------------
\nSuccessfully uninstalled transformers-4.28.1
Question 1: Why I am still getting following result?**
\nimport transformers
\nprint(\"transformers v\" + transformers.version)
\n------output---------------
\ntransformers v4.28.1
===========================================
\nQuestion 2: Why I am still getting the following result transformers v4.28.1 after installing 4.30?
\n!python -m pip install --no-index --find-links=/kaggle/input/benetech-pip transformers
\nSuccessfully installed huggingface-hub-0.14.1 transformers-4.30.0.dev0
import transformers
\nprint(\"transformers v\" + transformers.version)
\n------output---------------
\ntransformers v4.28.1
Are these due to following warning. How to avoid this?
\nWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv class=\"ansi-yellow-fg\">
@pksamantaray
\nI think you need to avoid import transformers before you install your preferred version of transformers.
Great content in here! Thanks for sharing @sinchir0 🔥💪
", "rawMarkdown": "Great content in here! Thanks for sharing @sinchir0 🔥💪", "votes": null } ], "comments": [ { "id": 2258335, "author_name": "tikutiku", "author_url": "", "post_date": "05/14/2023 05:57:45", "content": "You can use Notebook output as well.
\nXXX) and run !pip download git+https://github.com/huggingface/transformers!pip download wheel and !pip download setuptoolsAdd Data button and run !pip uninstall transformers -y and !python -m pip install --no-index --find-links=/kaggle/input/XXX transformers@tikutiku
\nI didn't know about that method, thanks!
This method does not work for me.. When I try to load the files into a notebook I get the following error.
\n Installing build dependencies ... error\n error: subprocess-exited-with-error\n\n × pip subprocess to install build dependencies did not run successfully.\n │ exit code: 1\n ╰─> [4 lines of output]\n Looking in links: /kaggle/input/XXX/\n ERROR: Could not find a version that satisfies the requirement setuptools>=40.8.0 (from versions: none)\n ERROR: No matching distribution found for setuptools>=40.8.0\n\n [end of output]\nGood catch. I found that wheel and setuptools should be in the Notebook output as well (usually these are automatically downloaded by !pip download your-package).
I created Notebooks for demo:
\nhttps://www.kaggle.com/code/tikutiku/benetech-pip
\nhttps://www.kaggle.com/code/tikutiku/benetech-install-transformers-w-o-internet
Hope it helps.
", "votes": null, "replies": [ { "id": 2265815, "author_name": "brendanartley", "author_url": "", "post_date": "05/19/2023 14:28:05", "content": "Thanks for sharing this Tom. Very helpful :)
", "votes": null, "replies": [ { "id": 2267594, "author_name": "pksamantaray", "author_url": "", "post_date": "05/21/2023 05:36:03", "content": "Hi Bartley, I have done it many times. Why am I still getting the wrong transformers version?
\nimport transformers
\nprint(\"transformers v\" + transformers.version)
\n------output---------------
\ntransformers v4.28.1
!pip uninstall transformers -y
\n------output---------------
\nSuccessfully uninstalled transformers-4.28.1
Question 1: Why I am still getting following result?**
\nimport transformers
\nprint(\"transformers v\" + transformers.version)
\n------output---------------
\ntransformers v4.28.1
===========================================
\nQuestion 2: Why I am still getting the following result transformers v4.28.1 after installing 4.30?
\n!python -m pip install --no-index --find-links=/kaggle/input/benetech-pip transformers
\nSuccessfully installed huggingface-hub-0.14.1 transformers-4.30.0.dev0
import transformers
\nprint(\"transformers v\" + transformers.version)
\n------output---------------
\ntransformers v4.28.1
Are these due to following warning. How to avoid this?
\nWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv class=\"ansi-yellow-fg\">
@pksamantaray
\nI think you need to avoid import transformers before you install your preferred version of transformers.
Note that when using the Transformer Repository in an internet-off notebook, you will need to update the file paths to point to the correct locations in the dataset.
", "votes": null, "replies": [] }, { "id": 2300941, "author_name": "vladiluzjr", "author_url": "", "post_date": "06/13/2023 14:23:06", "content": "Great content in here! Thanks for sharing @sinchir0 🔥💪
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "2258238": "This competition often requires the latest version of Transformer, which is installed with the following command.\n```sh\n!pip install git+https://github.com/huggingface/transformers\n```\n\nHowever, the inference notebook cannot pip install because the internet is turned off. \nAt that time, you can install Transformer from Dataset by executing the following command.\n\n---\n1 Do the following commands at your local env for making wheel file.\n\n```sh\ngit clone https://github.com/huggingface/transformers.git\ncd transformers\npip install --upgrade pip setuptools wheel\npython setup.py bdist_wheel\n```\n\n2 You get a file named `dist/transformers-4.XX.0.dev0-py3-none-any.whl`, upload it to the Kaggle Dataset.\n\n3 Execute the following commands on the Notebook.\n```sh\n!pip install /kaggle/input/YOUR_DATASET_NAME/transformers-4.XX.0.dev0-py3-none-any.whl \n```\n\n---\n\nPlease let me know if there is an easier way.", "2258335": "You can use Notebook output as well.\n\n1. Create a Notebook (named `XXX`) and run `!pip download git+https://github.com/huggingface/transformers`\nEDIT : Also run `!pip download wheel` and `!pip download setuptools`\n2. Save this Notebook\n3. In your inference Notebook, add the output of the above Notebook by `Add Data` button and run `!pip uninstall transformers -y` and `!python -m pip install --no-index --find-links=/kaggle/input/XXX transformers`", "2258384": "tikutiku \nI didn't know about that method, thanks!", "2259116": "Note that when using the Transformer Repository in an internet-off notebook, you will need to update the file paths to point to the correct locations in the dataset.", "2264664": "This method does not work for me.. When I try to load the files into a notebook I get the following error.\n```\n Installing build dependencies ... error\n error: subprocess-exited-with-error\n \n × pip subprocess to install build dependencies did not run successfully.\n │ exit code: 1\n ╰─> [4 lines of output]\n Looking in links: /kaggle/input/XXX/\n ERROR: Could not find a version that satisfies the requirement setuptools>=40.8.0 (from versions: none)\n ERROR: No matching distribution found for setuptools>=40.8.0\n \n [end of output]\n```", "2265047": "Good catch. I found that `wheel` and `setuptools` should be in the Notebook output as well (usually these are automatically downloaded by `!pip download your-package`).\n\nI created Notebooks for demo:\nhttps://www.kaggle.com/code/tikutiku/benetech-pip\nhttps://www.kaggle.com/code/tikutiku/benetech-install-transformers-w-o-internet\n\nHope it helps.", "2265815": "Thanks for sharing this Tom. Very helpful :)", "2267594": "Hi Bartley, I have done it many times. Why am I still getting the wrong transformers version?\n\n**import transformers**\n**print(\"transformers v\" + transformers.__version__)**\n------output---------------\n*transformers v4.28.1*\n\n\n**!pip uninstall transformers -y**\n------output---------------\n*Successfully uninstalled transformers-4.28.1*\n\n**Question 1:** Why I am still getting following result?**\n**import transformers**\n**print(\"transformers v\" + transformers.__version__)**\n------output---------------\n*transformers v4.28.1*\n\n===========================================\n**Question 2:** Why I am still getting the following result transformers v4.28.1 after installing 4.30?\n!python -m pip install --no-index --find-links=/kaggle/input/benetech-pip transformers\nSuccessfully installed huggingface-hub-0.14.1 transformers-4.30.0.dev0\n\n**import transformers**\n**print(\"transformers v\" + transformers.__version__)**\n------output---------------\n*transformers v4.28.1*\n\nAre these due to following warning. How to avoid this?\n*WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv class=\"ansi-yellow-fg\">*", "2267616": "pksamantaray \nI think you need to avoid `import transformers` before you install your preferred version of transformers.", "2300941": "Great content in here! Thanks for sharing @sinchir0 🔥💪" }, "source": "meta" }