Hi, I went through some notebooks after getting lost on the fine-tuning part. I was fine-tuning on my local machine with a 3060 12GB GPU, using a batch size of 4, and it took an enormous amount of time even for just 3 epochs (around 6–8 hours).
\nMy questions are: How are you all fine-tuning your models? Are you using any techniques like LoRA or adapters (freezing weights)? By the way, I fine-tuned on ByT5-small (300M).
\nAlso, were you guys fine-tuning the model just on the train.csv data?
I’m not aiming to win, I’m here to learn how things are done. Your help and guidance would be deeply appreciated.
", "messages": [ { "id": 3394031, "postDate": "2026-01-20T10:11:42.883Z", "content": "actually you can even finetune byt5 base with 12 gb gpu without OOM issues using huggingface pipelines. Some of training argument tricks for GPU memory efficency. gradient_checkpointing=True, torch_empty_cache_steps=2 (use this one only if oom issue persist otherwise overkill will slow down training), prefer bf16=True nvidia 3000-4000 series consumer gpus are compatible it, it will be better then fp16 at preventing numeric overflow. Also some speed up for validation during training, dataloader_num_workers=8 (or 4 if your CPU don't have many core check out cpu utilization 8 kind of guarantees your gpu never starve but can be a bit over kill), dataloader_pin_memory=True, dataloader_persistent_workers=True. Most importantly incease your real memory with gradient accumulation do not directly increase batch size exp (per_device_train_batch_size=2, gradient_accumulation_steps=8, per_device_eval_batch_size=4 ) you can double per_device_eval_batch_size, gradient_accumulation_steps if you don't see OOM issues. in general validation usually takes 1/3 of training memory thnx to forward pass only nature so it can be safe to increase per_device_eval_batch_size=8.
", "rawMarkdown": "actually you can even finetune byt5 base with 12 gb gpu without OOM issues using huggingface pipelines. Some of training argument tricks for GPU memory efficency. gradient_checkpointing=True, torch_empty_cache_steps=2 (use this one only if oom issue persist otherwise overkill will slow down training), prefer bf16=True nvidia 3000-4000 series consumer gpus are compatible it, it will be better then fp16 at preventing numeric overflow. Also some speed up for validation during training, dataloader_num_workers=8 (or 4 if your CPU don't have many core check out cpu utilization 8 kind of guarantees your gpu never starve but can be a bit over kill), dataloader_pin_memory=True, dataloader_persistent_workers=True. Most importantly incease your real memory with gradient accumulation do not directly increase batch size exp (per_device_train_batch_size=2, gradient_accumulation_steps=8, per_device_eval_batch_size=4 ) you can double per_device_eval_batch_size, gradient_accumulation_steps if you don't see OOM issues. in general validation usually takes 1/3 of training memory thnx to forward pass only nature so it can be safe to increase per_device_eval_batch_size=8. ", "votes": 1 }, { "id": 3393930, "postDate": "2026-01-20T06:49:45.207Z", "content": "Hi, I went through some notebooks after getting lost on the fine-tuning part. I was fine-tuning on my local machine with a 3060 12GB GPU, using a batch size of 4, and it took an enormous amount of time even for just 3 epochs (around 6–8 hours).
\nMy questions are: How are you all fine-tuning your models? Are you using any techniques like LoRA or adapters (freezing weights)? By the way, I fine-tuned on ByT5-small (300M).
\nAlso, were you guys fine-tuning the model just on the train.csv data?
I’m not aiming to win, I’m here to learn how things are done. Your help and guidance would be deeply appreciated.
", "rawMarkdown": "Hi, I went through some notebooks after getting lost on the fine-tuning part. I was fine-tuning on my local machine with a 3060 12GB GPU, using a batch size of 4, and it took an enormous amount of time even for just 3 epochs (around 6–8 hours).\n\nMy questions are: How are you all fine-tuning your models? Are you using any techniques like LoRA or adapters (freezing weights)? By the way, I fine-tuned on ByT5-small (300M).\n\nAlso, were you guys fine-tuning the model just on the `train.csv` data?\n\nI’m not aiming to win, I’m here to learn how things are done. Your help and guidance would be deeply appreciated.\n\n" }, { "id": 3394360, "postDate": "2026-01-20T22:12:19.070Z", "content": "For me it takes 90 Minutes to train full fine tuning. (8 GB VRAM, but 50xx)
", "rawMarkdown": "For me it takes 90 Minutes to train full fine tuning. (8 GB VRAM, but 50xx)" }, { "id": 3393943, "postDate": "2026-01-20T07:05:01.467Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 3394031, "author_name": "Pelin Keskin", "author_url": "", "post_date": "2026-01-20T10:11:42.883000", "content": "actually you can even finetune byt5 base with 12 gb gpu without OOM issues using huggingface pipelines. Some of training argument tricks for GPU memory efficency. gradient_checkpointing=True, torch_empty_cache_steps=2 (use this one only if oom issue persist otherwise overkill will slow down training), prefer bf16=True nvidia 3000-4000 series consumer gpus are compatible it, it will be better then fp16 at preventing numeric overflow. Also some speed up for validation during training, dataloader_num_workers=8 (or 4 if your CPU don't have many core check out cpu utilization 8 kind of guarantees your gpu never starve but can be a bit over kill), dataloader_pin_memory=True, dataloader_persistent_workers=True. Most importantly incease your real memory with gradient accumulation do not directly increase batch size exp (per_device_train_batch_size=2, gradient_accumulation_steps=8, per_device_eval_batch_size=4 ) you can double per_device_eval_batch_size, gradient_accumulation_steps if you don't see OOM issues. in general validation usually takes 1/3 of training memory thnx to forward pass only nature so it can be safe to increase per_device_eval_batch_size=8.
", "votes": 1, "replies": [] }, { "id": 3394360, "author_name": "Aristeides", "author_url": "", "post_date": "2026-01-20T22:12:19.070000", "content": "For me it takes 90 Minutes to train full fine tuning. (8 GB VRAM, but 50xx)
", "votes": 0, "replies": [] }, { "id": 3393943, "author_name": "", "author_url": "", "post_date": "2026-01-20T07:05:01.467000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "3394031": "actually you can even finetune byt5 base with 12 gb gpu without OOM issues using huggingface pipelines. Some of training argument tricks for GPU memory efficency. gradient_checkpointing=True, torch_empty_cache_steps=2 (use this one only if oom issue persist otherwise overkill will slow down training), prefer bf16=True nvidia 3000-4000 series consumer gpus are compatible it, it will be better then fp16 at preventing numeric overflow. Also some speed up for validation during training, dataloader_num_workers=8 (or 4 if your CPU don't have many core check out cpu utilization 8 kind of guarantees your gpu never starve but can be a bit over kill), dataloader_pin_memory=True, dataloader_persistent_workers=True. Most importantly incease your real memory with gradient accumulation do not directly increase batch size exp (per_device_train_batch_size=2, gradient_accumulation_steps=8, per_device_eval_batch_size=4 ) you can double per_device_eval_batch_size, gradient_accumulation_steps if you don't see OOM issues. in general validation usually takes 1/3 of training memory thnx to forward pass only nature so it can be safe to increase per_device_eval_batch_size=8. ", "3393930": "Hi, I went through some notebooks after getting lost on the fine-tuning part. I was fine-tuning on my local machine with a 3060 12GB GPU, using a batch size of 4, and it took an enormous amount of time even for just 3 epochs (around 6–8 hours).\n\nMy questions are: How are you all fine-tuning your models? Are you using any techniques like LoRA or adapters (freezing weights)? By the way, I fine-tuned on ByT5-small (300M).\n\nAlso, were you guys fine-tuning the model just on the `train.csv` data?\n\nI’m not aiming to win, I’m here to learn how things are done. Your help and guidance would be deeply appreciated.\n\n", "3394360": "For me it takes 90 Minutes to train full fine tuning. (8 GB VRAM, but 50xx)", "3393943": "" } }