{
  "id": 150526,
  "title": "How train Fast rcnn on Multi GPUs?",
  "url": "/competitions/global-wheat-detection/discussion/150526",
  "author_name": "",
  "post_date": "2020-05-12T13:47:45.353068800Z",
  "votes": 3,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Newer come to the Detection competition.Anyone can help me to train fastrcnn on Multi GPUs? I failed to use the torch.nn.DataParallel/torch.nn.parallel.DistributedDataParallel. I have tried following code and failed :&lt;\nmodel create code :\nmodel = torchvision.models.detection.fasterrcnn_resnet50_fpn(pretrained=True)\nnum_classes = 2 <br>\nin_features = model.roi_heads.box_predictor.cls_score.in_features\nmodel.roi_heads.box_predictor = FastRCNNPredictor(in_features, num_classes)</p>\n\n<p>1.Multi GPUs code:\nmodel.to(device)\ntorch.distributed.init_process_group('nccl',init_method='file:///home/.../my_file',world_size=1,rank=0)\nmodel = torch.nn.parallel.DistributedDataParallel(model,device_ids=[0,1])\nloss_dict = model(images, targets)</p>\n\n<p>error: keep running...without any output</p>\n\n<p>2.Multi GPUs code:\nmodel = torch.nn.DataParallel(model,device_ids = device_ids)\nloss_dict = model(images, targets)</p>\n\n<p>error:  RuntimeError: The size of tensor a (2) must match the size of tensor b (3) at non-singleton dimension 0</p>\n\n<p>3.Multi GPUs code:\nfrom torch.nn.parallel.data_parallel import data_parallel\npred = data_parallel(classifier, [image,label])</p>",
  "messages": [
    {
      "id": "844173",
      "postDate": "05/12/2020 13:47:45",
      "content": "<p>Newer come to the Detection competition.Anyone can help me to train fastrcnn on Multi GPUs? I failed to use the torch.nn.DataParallel/torch.nn.parallel.DistributedDataParallel. I have tried following code and failed :&lt;\nmodel create code :\nmodel = torchvision.models.detection.fasterrcnn_resnet50_fpn(pretrained=True)\nnum_classes = 2 <br>\nin_features = model.roi_heads.box_predictor.cls_score.in_features\nmodel.roi_heads.box_predictor = FastRCNNPredictor(in_features, num_classes)</p>\n\n<p>1.Multi GPUs code:\nmodel.to(device)\ntorch.distributed.init_process_group('nccl',init_method='file:///home/.../my_file',world_size=1,rank=0)\nmodel = torch.nn.parallel.DistributedDataParallel(model,device_ids=[0,1])\nloss_dict = model(images, targets)</p>\n\n<p>error: keep running...without any output</p>\n\n<p>2.Multi GPUs code:\nmodel = torch.nn.DataParallel(model,device_ids = device_ids)\nloss_dict = model(images, targets)</p>\n\n<p>error:  RuntimeError: The size of tensor a (2) must match the size of tensor b (3) at non-singleton dimension 0</p>\n\n<p>3.Multi GPUs code:\nfrom torch.nn.parallel.data_parallel import data_parallel\npred = data_parallel(classifier, [image,label])</p>",
      "rawMarkdown": "Newer come to the Detection competition.Anyone can help me to train fastrcnn on Multi GPUs? I failed to use the torch.nn.DataParallel/torch.nn.parallel.DistributedDataParallel. I have tried following code and failed :&lt;\nmodel create code :\nmodel = torchvision.models.detection.fasterrcnn_resnet50_fpn(pretrained=True)\nnum_classes = 2  \nin_features = model.roi_heads.box_predictor.cls_score.in_features\nmodel.roi_heads.box_predictor = FastRCNNPredictor(in_features, num_classes)\n\n1.Multi GPUs code:\nmodel.to(device)\ntorch.distributed.init_process_group('nccl',init_method='file:///home/.../my_file',world_size=1,rank=0)\nmodel = torch.nn.parallel.DistributedDataParallel(model,device_ids=[0,1])\nloss_dict = model(images, targets)\n\nerror: keep running...without any output\n\n2.Multi GPUs code:\nmodel = torch.nn.DataParallel(model,device_ids = device_ids)\nloss_dict = model(images, targets)\n\nerror:  RuntimeError: The size of tensor a (2) must match the size of tensor b (3) at non-singleton dimension 0\n\n3.Multi GPUs code:\nfrom torch.nn.parallel.data_parallel import data_parallel\npred = data_parallel(classifier, [image,label])",
      "votes": null
    },
    {
      "id": "844209",
      "postDate": "05/12/2020 14:05:58",
      "content": "<p>Switch to <a href=\"https://github.com/PyTorchLightning/pytorch-lightning\">Pytorch Lightning</a> :)</p>",
      "rawMarkdown": "Switch to [Pytorch Lightning](https://github.com/PyTorchLightning/pytorch-lightning) :)",
      "votes": null
    },
    {
      "id": "844222",
      "postDate": "05/12/2020 14:14:21",
      "content": "<p>😂  There is public kernel on Pytorch Lightning.I will take a look.Thank you for answer :D</p>",
      "rawMarkdown": "😂  There is public kernel on Pytorch Lightning.I will take a look.Thank you for answer :D",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 844209,
      "author_name": "salilm23",
      "author_url": "",
      "post_date": "05/12/2020 14:05:58",
      "content": "<p>Switch to <a href=\"https://github.com/PyTorchLightning/pytorch-lightning\">Pytorch Lightning</a> :)</p>",
      "votes": null,
      "replies": [
        {
          "id": 844222,
          "author_name": "raininbox",
          "author_url": "",
          "post_date": "05/12/2020 14:14:21",
          "content": "<p>😂  There is public kernel on Pytorch Lightning.I will take a look.Thank you for answer :D</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "844173": "Newer come to the Detection competition.Anyone can help me to train fastrcnn on Multi GPUs? I failed to use the torch.nn.DataParallel/torch.nn.parallel.DistributedDataParallel. I have tried following code and failed :&lt;\nmodel create code :\nmodel = torchvision.models.detection.fasterrcnn_resnet50_fpn(pretrained=True)\nnum_classes = 2  \nin_features = model.roi_heads.box_predictor.cls_score.in_features\nmodel.roi_heads.box_predictor = FastRCNNPredictor(in_features, num_classes)\n\n1.Multi GPUs code:\nmodel.to(device)\ntorch.distributed.init_process_group('nccl',init_method='file:///home/.../my_file',world_size=1,rank=0)\nmodel = torch.nn.parallel.DistributedDataParallel(model,device_ids=[0,1])\nloss_dict = model(images, targets)\n\nerror: keep running...without any output\n\n2.Multi GPUs code:\nmodel = torch.nn.DataParallel(model,device_ids = device_ids)\nloss_dict = model(images, targets)\n\nerror:  RuntimeError: The size of tensor a (2) must match the size of tensor b (3) at non-singleton dimension 0\n\n3.Multi GPUs code:\nfrom torch.nn.parallel.data_parallel import data_parallel\npred = data_parallel(classifier, [image,label])",
    "844209": "Switch to [Pytorch Lightning](https://github.com/PyTorchLightning/pytorch-lightning) :)",
    "844222": "😂  There is public kernel on Pytorch Lightning.I will take a look.Thank you for answer :D"
  },
  "source": "meta"
}