Generating a description of the images to augment the existing description can improve the list of context-filtered synonyms you use to match descriptions with tags.
\nThere are tons of solutions out there to generate a caption for an image. However, a risk is the variety of photo content- both in verb tense and whether the nouns are executing the same verb- will lead to captions that are not nearly as accurate or normalized as we have now: https://www.ibm.com/blogs/research/2019/06/image-captioning .
\nBut the beauty is with existing correct captions, we can ensure the auto-descriptions are both correct and provide new synonyms.
\nAuto-generated captions have sentence structure consistent enough to apply text similarity technologies that rely on structure, and word order to identify the right synonyms:
\nhttps://ieeexplore.ieee.org/abstract/document/8630924
However, we are getting inaccurate similarity scores between obvious similar sentences. The following as been attempted:
\ni. using max_similarity as noted in the paper instead of default
\ndisambiguation as https://github.com/chandu8542/semantic-similarity has
\nimplemented. The GitHub author points out in the ReadMe he took that
\ndifferent route in his implementation, but doesn't give a reason for it
\nii. dependency checks. Using the right versions of everything
\niii. adjusting hyperparameter alpha
\niv. inconsistent vector coefficients. This is a problem, but haven’t fixed it
When using more basic similarity tests such as cosine, the results make sense so implementing this paper is the issue. We believe this paper would be useful in finding almost guaranteed-correct synonyms to then search for in the image tag files, and any advice on hyperparameter adjustment or proper implementation would be welcome.
", "messages": [ { "id": "1615055", "postDate": "12/11/2021 18:47:55", "content": "Generating a description of the images to augment the existing description can improve the list of context-filtered synonyms you use to match descriptions with tags.
\nThere are tons of solutions out there to generate a caption for an image. However, a risk is the variety of photo content- both in verb tense and whether the nouns are executing the same verb- will lead to captions that are not nearly as accurate or normalized as we have now: https://www.ibm.com/blogs/research/2019/06/image-captioning .
\nBut the beauty is with existing correct captions, we can ensure the auto-descriptions are both correct and provide new synonyms.
\nAuto-generated captions have sentence structure consistent enough to apply text similarity technologies that rely on structure, and word order to identify the right synonyms:
\nhttps://ieeexplore.ieee.org/abstract/document/8630924
However, we are getting inaccurate similarity scores between obvious similar sentences. The following as been attempted:
\ni. using max_similarity as noted in the paper instead of default
\ndisambiguation as https://github.com/chandu8542/semantic-similarity has
\nimplemented. The GitHub author points out in the ReadMe he took that
\ndifferent route in his implementation, but doesn't give a reason for it
\nii. dependency checks. Using the right versions of everything
\niii. adjusting hyperparameter alpha
\niv. inconsistent vector coefficients. This is a problem, but haven’t fixed it
When using more basic similarity tests such as cosine, the results make sense so implementing this paper is the issue. We believe this paper would be useful in finding almost guaranteed-correct synonyms to then search for in the image tag files, and any advice on hyperparameter adjustment or proper implementation would be welcome.
", "rawMarkdown": "Generating a description of the images to augment the existing description can improve the list of context-filtered synonyms you use to match descriptions with tags. \n\n There are tons of solutions out there to generate a caption for an image. However, a risk is the variety of photo content- both in verb tense and whether the nouns are executing the same verb- will lead to captions that are not nearly as accurate or normalized as we have now: https://www.ibm.com/blogs/research/2019/06/image-captioning . \n\nBut the beauty is with existing correct captions, we can ensure the auto-descriptions are both correct and provide new synonyms. \n\nAuto-generated captions have sentence structure consistent enough to apply text similarity technologies that rely on structure, and word order to identify the right synonyms:\nhttps://ieeexplore.ieee.org/abstract/document/8630924\n\nHowever, we are getting inaccurate similarity scores between obvious similar sentences. The following as been attempted:\n\n i. using max_similarity as noted in the paper instead of default\ndisambiguation as https://github.com/chandu8542/semantic-similarity has\nimplemented. The GitHub author points out in the ReadMe he took that\ndifferent route in his implementation, but doesn't give a reason for it\nii. dependency checks. Using the right versions of everything\niii. adjusting hyperparameter alpha\niv. inconsistent vector coefficients. This is a problem, but haven’t fixed it\n\nWhen using more basic similarity tests such as cosine, the results make sense so implementing this paper is the issue. We believe this paper would be useful in finding almost guaranteed-correct synonyms to then search for in the image tag files, and any advice on hyperparameter adjustment or proper implementation would be welcome.", "votes": null }, { "id": "1615066", "postDate": "12/11/2021 19:07:53", "content": "How are you generating captions? I played around with the idea of trying to generate captions for images and then using those captions as additional data in some way, but found it too inconsistent to incorporate into my approach. Depending on your generation method it could be that the captions are just being generated in an unexpected way. I would definitely check how whatever tokenizer you're using is handling those captions.
\nOn the other side of things you might want to check how you're inputting the \"true\" descriptions. Given we have multiple for each image it might be that there is some weird formatting going on in comparing a single caption to a series of them.
\nIf nothing else I wonder if you could just use cosine similarity to detect and discard bad captions since you mentioned that is producing reasonable results.
", "rawMarkdown": "How are you generating captions? I played around with the idea of trying to generate captions for images and then using those captions as additional data in some way, but found it too inconsistent to incorporate into my approach. Depending on your generation method it could be that the captions are just being generated in an unexpected way. I would definitely check how whatever tokenizer you're using is handling those captions. \n\nOn the other side of things you might want to check how you're inputting the \"true\" descriptions. Given we have multiple for each image it might be that there is some weird formatting going on in comparing a single caption to a series of them.\n\nIf nothing else I wonder if you could just use cosine similarity to detect and discard bad captions since you mentioned that is producing reasonable results.", "votes": null }, { "id": "1615141", "postDate": "12/11/2021 21:56:12", "content": "Thanks man.
\nGenerating captions a few different OSS ways, but agreed its far from perfect. most attempted OSS is with per-trained models since this is computationally heavy.* we may ignore the need to measure description accuracy and just focus on noun synonyms since the Named Entities are usually right even if the verb isnt. **
The \"true\" descriptions are formatted in that we're not comparing 5 captions against 1. We acknowledge normalizing the two ends in many ways would help.
\nthat linked IEEE paper fails on its own in our attempts.
\n Ex 63 in linked paper (\"An automobile is a car.\" , \"A car is a motor vehicle with room for a small number of passengers.\") is .065 programmatically, yet the paper output is .55
\nOther examples we get close to their score, and others like the 'woodland is land' example I'm >1 similarity which should be possible
`
\n\"\"\"
\nEdite Code from:
It did not work as it was.
\n\"\"\"
\nfrom pywsd import disambiguate
\nfrom pywsd.similarity import max_similarity
\nimport nltk , pprint
\nfrom nltk.corpus import wordnet as wn
import numpy as np
\nfrom collections import defaultdict
\npp = pprint.PrettyPrinter(indent=4 , sort_dicts=False)
alpha = 0.2
\nbeta = 0.45
\nbenchmark_similarity = 0.8025
\ngamma = 1.8
\n\"\"\"
\nSemantic similarity based on the paper:
\n Calculating the similarity between words and sentences using a lexical database and corpus statistics
\nTKDE, 2018
\n\"\"\"
def _synset_similarity(s1,s2):
\n L1 =dict()
\n L2 =defaultdict(list)
for syn1 in s1:\n #s1_word = syn1.split('.')[0]\n L1[syn1.split('.')[0]] =list()\n\n for syn2 in s2:\n #s2_word = syn1.split('.')[0]\n subsumer = wn.synset(syn1).lowest_common_hypernyms(wn.synset(syn2), simulate_root=True)[0]\n\n h =subsumer.max_depth() + 1 # as done on NLTK wordnet \n syn1_dist_subsumer = wn.synset(syn1).shortest_path_distance(subsumer,simulate_root =True)\n syn2_dist_subsumer = wn.synset(syn2).shortest_path_distance(subsumer,simulate_root =True)\n l =syn1_dist_subsumer + syn2_dist_subsumer\n f1 = np.exp(-alpha*l)\n a = np.exp(beta*h)\n b = np.exp(-beta*h)\n f2 = (a-b) /(a+b)\n sim = f1*f2\n L1[syn1.split('.')[0]].append(sim) \n L2[syn2.split('.')[0]].append(sim)\nreturn L1, L2\ndef getSimilarity(s1,s2):
\ns1_wsd = disambiguate(s1, algorithm=max_similarity) # using max similarity\ns2_wsd = disambiguate(s2, algorithm=max_similarity)\n\n# remove None synsets\ns1_wsd = [syn for syn in s1_wsd if syn[1]]\ns2_wsd = [syn for syn in s2_wsd if syn[1]]\n\ns1_wsd_wn = [_sys[1].name() for _sys in s1_wsd]\ns2_wsd_wn = [_sys[1].name() for _sys in s2_wsd]\npp.pprint(s1_wsd)\npp.pprint(s2_wsd)\nprint('s1 words to go in sysnet sim func: {}'.format(s1_wsd_wn))\nprint('s2 words to go in sysnet sim func: {}'.format(s2_wsd_wn))\n#vector_length = max(len(s1_wsd), len(s2_wsd))\nL1,L2 = _synset_similarity(s1_wsd_wn,s2_wsd_wn)\n\n\n#L1,L2 = _synset_similarity(s1_wsd_wn,s2_wsd_wn)\nV1 =np.array( [max(L1[key]) for key in L1.keys()])\nV2 = np.array([max(L2[key]) for key in L2.keys()])\nS = np.linalg.norm(V1)*np.linalg.norm(V2)\nC1 = sum(V1>=benchmark_similarity)\nC2 = sum(V2>=benchmark_similarity)\n\nXi = (C1+C2) / gamma\n\nif C1+C2 == 0:\n Xi = max(V1.size, V2.size) / 2\nreturn S/Xi\ns1 = 'An automobile is a car.'
\ns2 = 'A car is a motor vehicle with room for a small number of passengers.'
print('similarity between '+'\\\"'+s1+'\\\"'+' and ' +'\\\"'+ s2+'\\\"'+ 'is: '+str(getSimilarity(s1,s2)))
\n`
Interesting. I'm not sure what's going on given you seem to be using nearly the exact same code as was used in the original implementation. Have you check to make sure your vocab dictionary is setup correctly? That's the only thing I can think of at the moment. Other than that maybe try to do some additional print outs of V1 and V2 to make sure those values are consistent with what you expect.
\nIf it doesn't work maybe you could consider using the tags to validate the captions i.e. do the captions relate to the tags at all or are they giving something completely different.
", "rawMarkdown": "Interesting. I'm not sure what's going on given you seem to be using nearly the exact same code as was used in the original implementation. Have you check to make sure your vocab dictionary is setup correctly? That's the only thing I can think of at the moment. Other than that maybe try to do some additional print outs of V1 and V2 to make sure those values are consistent with what you expect. \n\nIf it doesn't work maybe you could consider using the tags to validate the captions i.e. do the captions relate to the tags at all or are they giving something completely different.", "votes": null }, { "id": "1615879", "postDate": "12/12/2021 20:46:11", "content": "The vocab dict uses the version they have for pywsd which online was an issue for others.
\nRegarding the image caption generator, if we are using this approach (https://machinelearningmastery.com/develop-a-deep-learning-caption-generation-model-in-python/)
am i correct in assuming reducing the number of epochs to quickly train the data has no impact outside of under/over fitting?
", "rawMarkdown": "The vocab dict uses the version they have for pywsd which online was an issue for others. \nRegarding the image caption generator, if we are using this approach (https://machinelearningmastery.com/develop-a-deep-learning-caption-generation-model-in-python/)\n\nam i correct in assuming reducing the number of epochs to quickly train the data has no impact outside of under/over fitting?", "votes": null }, { "id": "1615936", "postDate": "12/13/2021 00:24:19", "content": "If you reduce the number of epochs too far you'll end up severely under-fitting. For quick training what I would suggest is saving a copy of the model after each epoch if it improves the loss on the validation set. That way you can cut the training after you see things no longer improving without discarding your results. Once you've done this you can load the last best checkpoint and then use that model from there.
", "rawMarkdown": "If you reduce the number of epochs too far you'll end up severely under-fitting. For quick training what I would suggest is saving a copy of the model after each epoch if it improves the loss on the validation set. That way you can cut the training after you see things no longer improving without discarding your results. Once you've done this you can load the last best checkpoint and then use that model from there.", "votes": null } ], "comments": [ { "id": 1615066, "author_name": "epickens", "author_url": "", "post_date": "12/11/2021 19:07:53", "content": "How are you generating captions? I played around with the idea of trying to generate captions for images and then using those captions as additional data in some way, but found it too inconsistent to incorporate into my approach. Depending on your generation method it could be that the captions are just being generated in an unexpected way. I would definitely check how whatever tokenizer you're using is handling those captions.
\nOn the other side of things you might want to check how you're inputting the \"true\" descriptions. Given we have multiple for each image it might be that there is some weird formatting going on in comparing a single caption to a series of them.
\nIf nothing else I wonder if you could just use cosine similarity to detect and discard bad captions since you mentioned that is producing reasonable results.
", "votes": null, "replies": [] }, { "id": 1615141, "author_name": "achalamin", "author_url": "", "post_date": "12/11/2021 21:56:12", "content": "Thanks man.
\nGenerating captions a few different OSS ways, but agreed its far from perfect. most attempted OSS is with per-trained models since this is computationally heavy.* we may ignore the need to measure description accuracy and just focus on noun synonyms since the Named Entities are usually right even if the verb isnt. **
The \"true\" descriptions are formatted in that we're not comparing 5 captions against 1. We acknowledge normalizing the two ends in many ways would help.
\nthat linked IEEE paper fails on its own in our attempts.
\n Ex 63 in linked paper (\"An automobile is a car.\" , \"A car is a motor vehicle with room for a small number of passengers.\") is .065 programmatically, yet the paper output is .55
\nOther examples we get close to their score, and others like the 'woodland is land' example I'm >1 similarity which should be possible
`
\n\"\"\"
\nEdite Code from:
It did not work as it was.
\n\"\"\"
\nfrom pywsd import disambiguate
\nfrom pywsd.similarity import max_similarity
\nimport nltk , pprint
\nfrom nltk.corpus import wordnet as wn
import numpy as np
\nfrom collections import defaultdict
\npp = pprint.PrettyPrinter(indent=4 , sort_dicts=False)
alpha = 0.2
\nbeta = 0.45
\nbenchmark_similarity = 0.8025
\ngamma = 1.8
\n\"\"\"
\nSemantic similarity based on the paper:
\n Calculating the similarity between words and sentences using a lexical database and corpus statistics
\nTKDE, 2018
\n\"\"\"
def _synset_similarity(s1,s2):
\n L1 =dict()
\n L2 =defaultdict(list)
for syn1 in s1:\n #s1_word = syn1.split('.')[0]\n L1[syn1.split('.')[0]] =list()\n\n for syn2 in s2:\n #s2_word = syn1.split('.')[0]\n subsumer = wn.synset(syn1).lowest_common_hypernyms(wn.synset(syn2), simulate_root=True)[0]\n\n h =subsumer.max_depth() + 1 # as done on NLTK wordnet \n syn1_dist_subsumer = wn.synset(syn1).shortest_path_distance(subsumer,simulate_root =True)\n syn2_dist_subsumer = wn.synset(syn2).shortest_path_distance(subsumer,simulate_root =True)\n l =syn1_dist_subsumer + syn2_dist_subsumer\n f1 = np.exp(-alpha*l)\n a = np.exp(beta*h)\n b = np.exp(-beta*h)\n f2 = (a-b) /(a+b)\n sim = f1*f2\n L1[syn1.split('.')[0]].append(sim) \n L2[syn2.split('.')[0]].append(sim)\nreturn L1, L2\ndef getSimilarity(s1,s2):
\ns1_wsd = disambiguate(s1, algorithm=max_similarity) # using max similarity\ns2_wsd = disambiguate(s2, algorithm=max_similarity)\n\n# remove None synsets\ns1_wsd = [syn for syn in s1_wsd if syn[1]]\ns2_wsd = [syn for syn in s2_wsd if syn[1]]\n\ns1_wsd_wn = [_sys[1].name() for _sys in s1_wsd]\ns2_wsd_wn = [_sys[1].name() for _sys in s2_wsd]\npp.pprint(s1_wsd)\npp.pprint(s2_wsd)\nprint('s1 words to go in sysnet sim func: {}'.format(s1_wsd_wn))\nprint('s2 words to go in sysnet sim func: {}'.format(s2_wsd_wn))\n#vector_length = max(len(s1_wsd), len(s2_wsd))\nL1,L2 = _synset_similarity(s1_wsd_wn,s2_wsd_wn)\n\n\n#L1,L2 = _synset_similarity(s1_wsd_wn,s2_wsd_wn)\nV1 =np.array( [max(L1[key]) for key in L1.keys()])\nV2 = np.array([max(L2[key]) for key in L2.keys()])\nS = np.linalg.norm(V1)*np.linalg.norm(V2)\nC1 = sum(V1>=benchmark_similarity)\nC2 = sum(V2>=benchmark_similarity)\n\nXi = (C1+C2) / gamma\n\nif C1+C2 == 0:\n Xi = max(V1.size, V2.size) / 2\nreturn S/Xi\ns1 = 'An automobile is a car.'
\ns2 = 'A car is a motor vehicle with room for a small number of passengers.'
print('similarity between '+'\\\"'+s1+'\\\"'+' and ' +'\\\"'+ s2+'\\\"'+ 'is: '+str(getSimilarity(s1,s2)))
\n`
Interesting. I'm not sure what's going on given you seem to be using nearly the exact same code as was used in the original implementation. Have you check to make sure your vocab dictionary is setup correctly? That's the only thing I can think of at the moment. Other than that maybe try to do some additional print outs of V1 and V2 to make sure those values are consistent with what you expect.
\nIf it doesn't work maybe you could consider using the tags to validate the captions i.e. do the captions relate to the tags at all or are they giving something completely different.
", "votes": null, "replies": [] }, { "id": 1615879, "author_name": "achalamin", "author_url": "", "post_date": "12/12/2021 20:46:11", "content": "The vocab dict uses the version they have for pywsd which online was an issue for others.
\nRegarding the image caption generator, if we are using this approach (https://machinelearningmastery.com/develop-a-deep-learning-caption-generation-model-in-python/)
am i correct in assuming reducing the number of epochs to quickly train the data has no impact outside of under/over fitting?
", "votes": null, "replies": [] }, { "id": 1615936, "author_name": "epickens", "author_url": "", "post_date": "12/13/2021 00:24:19", "content": "If you reduce the number of epochs too far you'll end up severely under-fitting. For quick training what I would suggest is saving a copy of the model after each epoch if it improves the loss on the validation set. That way you can cut the training after you see things no longer improving without discarding your results. Once you've done this you can load the last best checkpoint and then use that model from there.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1615055": "Generating a description of the images to augment the existing description can improve the list of context-filtered synonyms you use to match descriptions with tags. \n\n There are tons of solutions out there to generate a caption for an image. However, a risk is the variety of photo content- both in verb tense and whether the nouns are executing the same verb- will lead to captions that are not nearly as accurate or normalized as we have now: https://www.ibm.com/blogs/research/2019/06/image-captioning . \n\nBut the beauty is with existing correct captions, we can ensure the auto-descriptions are both correct and provide new synonyms. \n\nAuto-generated captions have sentence structure consistent enough to apply text similarity technologies that rely on structure, and word order to identify the right synonyms:\nhttps://ieeexplore.ieee.org/abstract/document/8630924\n\nHowever, we are getting inaccurate similarity scores between obvious similar sentences. The following as been attempted:\n\n i. using max_similarity as noted in the paper instead of default\ndisambiguation as https://github.com/chandu8542/semantic-similarity has\nimplemented. The GitHub author points out in the ReadMe he took that\ndifferent route in his implementation, but doesn't give a reason for it\nii. dependency checks. Using the right versions of everything\niii. adjusting hyperparameter alpha\niv. inconsistent vector coefficients. This is a problem, but haven’t fixed it\n\nWhen using more basic similarity tests such as cosine, the results make sense so implementing this paper is the issue. We believe this paper would be useful in finding almost guaranteed-correct synonyms to then search for in the image tag files, and any advice on hyperparameter adjustment or proper implementation would be welcome.", "1615066": "How are you generating captions? I played around with the idea of trying to generate captions for images and then using those captions as additional data in some way, but found it too inconsistent to incorporate into my approach. Depending on your generation method it could be that the captions are just being generated in an unexpected way. I would definitely check how whatever tokenizer you're using is handling those captions. \n\nOn the other side of things you might want to check how you're inputting the \"true\" descriptions. Given we have multiple for each image it might be that there is some weird formatting going on in comparing a single caption to a series of them.\n\nIf nothing else I wonder if you could just use cosine similarity to detect and discard bad captions since you mentioned that is producing reasonable results.", "1615141": "Thanks man. \nGenerating captions a few different OSS ways, but agreed its far from perfect. most attempted OSS is with per-trained models since this is computationally heavy.* we may ignore the need to measure description accuracy and just focus on noun synonyms since the Named Entities are usually right even if the verb isnt. **\n\nThe \"true\" descriptions are formatted in that we're not comparing 5 captions against 1. We acknowledge normalizing the two ends in many ways would help. \n\nthat linked IEEE paper fails on its own in our attempts. \n Ex 63 in linked paper (\"An automobile is a car.\" , \"A car is a motor vehicle with room for a small number of passengers.\") is .065 programmatically, yet the paper output is .55\nOther examples we get close to their score, and others like the 'woodland is land' example I'm >1 similarity which should be possible\n\n`\n\"\"\"\nEdite Code from:\n#https://github.com/chandu8542/semantic-similarity/blob/master/semantic_similarity.py\nIt did not work as it was. \n\"\"\"\nfrom pywsd import disambiguate\nfrom pywsd.similarity import max_similarity\nimport nltk , pprint\nfrom nltk.corpus import wordnet as wn\n#nltk.download('averaged_perceptron_tagger')\n#nltk.download('punkt')\nimport numpy as np\nfrom collections import defaultdict\npp = pprint.PrettyPrinter(indent=4 , sort_dicts=False)\n\n# initiaalize constaants\nalpha = 0.2\nbeta = 0.45\nbenchmark_similarity = 0.8025\ngamma = 1.8\n\"\"\"\nSemantic similarity based on the paper:\n Calculating the similarity between words and sentences using a lexical database and corpus statistics\nTKDE, 2018\n\"\"\"\n \n\n\ndef _synset_similarity(s1,s2):\n L1 =dict()\n L2 =defaultdict(list)\n \n for syn1 in s1:\n #s1_word = syn1.split('.')[0]\n L1[syn1.split('.')[0]] =list()\n \n for syn2 in s2:\n #s2_word = syn1.split('.')[0]\n# \n subsumer = wn.synset(syn1).lowest_common_hypernyms(wn.synset(syn2), simulate_root=True)[0]\n\n h =subsumer.max_depth() + 1 # as done on NLTK wordnet \n syn1_dist_subsumer = wn.synset(syn1).shortest_path_distance(subsumer,simulate_root =True)\n syn2_dist_subsumer = wn.synset(syn2).shortest_path_distance(subsumer,simulate_root =True)\n l =syn1_dist_subsumer + syn2_dist_subsumer\n f1 = np.exp(-alpha*l)\n a = np.exp(beta*h)\n b = np.exp(-beta*h)\n f2 = (a-b) /(a+b)\n sim = f1*f2\n L1[syn1.split('.')[0]].append(sim) \n L2[syn2.split('.')[0]].append(sim)\n return L1, L2\n\ndef getSimilarity(s1,s2):\n \n# s1_wsd = disambiguate(s1) # using default disambiguation\n# s2_wsd = disambiguate(s2)\n \n s1_wsd = disambiguate(s1, algorithm=max_similarity) # using max similarity\n s2_wsd = disambiguate(s2, algorithm=max_similarity)\n \n # remove None synsets\n s1_wsd = [syn for syn in s1_wsd if syn[1]]\n s2_wsd = [syn for syn in s2_wsd if syn[1]]\n \n s1_wsd_wn = [_sys[1].name() for _sys in s1_wsd]\n s2_wsd_wn = [_sys[1].name() for _sys in s2_wsd]\n pp.pprint(s1_wsd)\n pp.pprint(s2_wsd)\n print('s1 words to go in sysnet sim func: {}'.format(s1_wsd_wn))\n print('s2 words to go in sysnet sim func: {}'.format(s2_wsd_wn))\n #vector_length = max(len(s1_wsd), len(s2_wsd))\n L1,L2 = _synset_similarity(s1_wsd_wn,s2_wsd_wn)\n\n \n #L1,L2 = _synset_similarity(s1_wsd_wn,s2_wsd_wn)\n V1 =np.array( [max(L1[key]) for key in L1.keys()])\n V2 = np.array([max(L2[key]) for key in L2.keys()])\n S = np.linalg.norm(V1)*np.linalg.norm(V2)\n C1 = sum(V1>=benchmark_similarity)\n C2 = sum(V2>=benchmark_similarity)\n\n Xi = (C1+C2) / gamma\n\n if C1+C2 == 0:\n Xi = max(V1.size, V2.size) / 2\n return S/Xi\n\n\n# In[20]:\n\n\ns1 = 'An automobile is a car.'\ns2 = 'A car is a motor vehicle with room for a small number of passengers.'\n\n#s1= 'cyclone hit my home badly'\n#s2 ='hurricane striked my house severly'\n \nprint('similarity between '+'\\\"'+s1+'\\\"'+' and ' +'\\\"'+ s2+'\\\"'+ 'is: '+str(getSimilarity(s1,s2)))\n`", "1615154": "Interesting. I'm not sure what's going on given you seem to be using nearly the exact same code as was used in the original implementation. Have you check to make sure your vocab dictionary is setup correctly? That's the only thing I can think of at the moment. Other than that maybe try to do some additional print outs of V1 and V2 to make sure those values are consistent with what you expect. \n\nIf it doesn't work maybe you could consider using the tags to validate the captions i.e. do the captions relate to the tags at all or are they giving something completely different.", "1615879": "The vocab dict uses the version they have for pywsd which online was an issue for others. \nRegarding the image caption generator, if we are using this approach (https://machinelearningmastery.com/develop-a-deep-learning-caption-generation-model-in-python/)\n\nam i correct in assuming reducing the number of epochs to quickly train the data has no impact outside of under/over fitting?", "1615936": "If you reduce the number of epochs too far you'll end up severely under-fitting. For quick training what I would suggest is saving a copy of the model after each epoch if it improves the loss on the validation set. That way you can cut the training after you see things no longer improving without discarding your results. Once you've done this you can load the last best checkpoint and then use that model from there." }, "source": "meta" }