Hi All,\nI've started to take an interest in unsupervised learning lately and have found different clustering techniques really interesting for EDA and feature engineering.
\n\nI posted a kernel doing k-modes clustering and came up with some interesting observations on our categorical dataset:
\n\nhttps://www.kaggle.com/teejmahal20/clustering-categorical-data-k-modes-cat-ii
\n\nLet me know your thoughts/new ideas and if you find the analysis interesting and/or insightful. And upvote the kernel as well :)
\n\nOn the topic of clustering, I have a couple of questions to pose:\n1) Specific to categorical data, has anyone approached cluster analysis in ways other than k-modes?\n2) In general, what methods have you found worthwhile for cluster analysis?
", "messages": [ { "id": "725105", "postDate": "01/21/2020 19:58:57", "content": "Hi All,\nI've started to take an interest in unsupervised learning lately and have found different clustering techniques really interesting for EDA and feature engineering.
\n\nI posted a kernel doing k-modes clustering and came up with some interesting observations on our categorical dataset:
\n\nhttps://www.kaggle.com/teejmahal20/clustering-categorical-data-k-modes-cat-ii
\n\nLet me know your thoughts/new ideas and if you find the analysis interesting and/or insightful. And upvote the kernel as well :)
\n\nOn the topic of clustering, I have a couple of questions to pose:\n1) Specific to categorical data, has anyone approached cluster analysis in ways other than k-modes?\n2) In general, what methods have you found worthwhile for cluster analysis?
", "rawMarkdown": "Hi All,\nI've started to take an interest in unsupervised learning lately and have found different clustering techniques really interesting for EDA and feature engineering.\n\nI posted a kernel doing k-modes clustering and came up with some interesting observations on our categorical dataset:\n\n[https://www.kaggle.com/teejmahal20/clustering-categorical-data-k-modes-cat-ii](https://www.kaggle.com/teejmahal20/clustering-categorical-data-k-modes-cat-ii)\n\nLet me know your thoughts/new ideas and if you find the analysis interesting and/or insightful. And upvote the kernel as well :)\n\nOn the topic of clustering, I have a couple of questions to pose:\n1) Specific to categorical data, has anyone approached cluster analysis in ways other than k-modes?\n2) In general, what methods have you found worthwhile for cluster analysis?", "votes": null }, { "id": "725139", "postDate": "01/21/2020 20:45:20", "content": "I had done a clustering problem on sentence embeddings. For that, I used K-Means.\nFor a Financial Analytics course, I had also used K-Nearest Neighbour which helped me. \nGreat topic! Upvoted.
", "rawMarkdown": "I had done a clustering problem on sentence embeddings. For that, I used K-Means.\nFor a Financial Analytics course, I had also used K-Nearest Neighbour which helped me. \nGreat topic! Upvoted.", "votes": null }, { "id": "725238", "postDate": "01/21/2020 23:31:07", "content": "Thanks @rohanmahajan011! What kind of preprocessing or standardizing did you do on the data prior to applying kmeans?
", "rawMarkdown": "Thanks @rohanmahajan011! What kind of preprocessing or standardizing did you do on the data prior to applying kmeans?", "votes": null }, { "id": "725359", "postDate": "01/22/2020 02:54:53", "content": "Regarding the 2nd topic, a 'clustering' method that I've found really interesting is principal component analysis (P.C.A.) and kernel P.C.A. I know that they're usually described first as dimension reduction techniques, but they can also be thought of as used for clustering data. I do notice an interesting aspect is that they don't necessarily separate the data for you in the way that support vector machines (S.V.M.) may do so.
\n\nI learned about regular P.C.A. in a statistics course on multivariate data analysis and so I found the math quite interesting. The idea about transforming a data matrix into another one of identical dimensions was quite confusing to me at first. I think the concept is kind of strange, especially when you look at the analysis of the different eigenvectors.
\n\nLater on, I was able to learn about kernel P.C.A., and how the whole math shows that the data is actually being projected onto some sort of hyperplane which separates the data into different 'clusters.'
", "rawMarkdown": "Regarding the 2nd topic, a 'clustering' method that I've found really interesting is principal component analysis (P.C.A.) and kernel P.C.A. I know that they're usually described first as dimension reduction techniques, but they can also be thought of as used for clustering data. I do notice an interesting aspect is that they don't necessarily separate the data for you in the way that support vector machines (S.V.M.) may do so.\n\nI learned about regular P.C.A. in a statistics course on multivariate data analysis and so I found the math quite interesting. The idea about transforming a data matrix into another one of identical dimensions was quite confusing to me at first. I think the concept is kind of strange, especially when you look at the analysis of the different eigenvectors.\n\nLater on, I was able to learn about kernel P.C.A., and how the whole math shows that the data is actually being projected onto some sort of hyperplane which separates the data into different 'clusters.'", "votes": null }, { "id": "725842", "postDate": "01/22/2020 14:40:34", "content": "Thanks for the feedback @yuqizheng. PCA is definitely quite useful for visualizing clusters, particularly when you have a high-dimensional feature set.
", "rawMarkdown": "Thanks for the feedback @yuqizheng. PCA is definitely quite useful for visualizing clusters, particularly when you have a high-dimensional feature set.", "votes": null }, { "id": "726041", "postDate": "01/22/2020 18:36:47", "content": "I must admit though, my knowledge of algorithms still feels almost backwards. I'm a university student and I've studied an algorithms course prior to the machine learning courses. I hope to develop a greater understanding in due time. 😄
", "rawMarkdown": "I must admit though, my knowledge of algorithms still feels almost backwards. I'm a university student and I've studied an algorithms course prior to the machine learning courses. I hope to develop a greater understanding in due time. 😄", "votes": null }, { "id": "729772", "postDate": "01/26/2020 16:20:50", "content": "Hi @teejmahal20, I found your analysis pretty interesting, you're doing pretty well with visualizations, I would like to see some clustering methods with scatterplot, but I don't imagine how that can happen in your context. Keep going man!
", "rawMarkdown": "Hi @teejmahal20, I found your analysis pretty interesting, you're doing pretty well with visualizations, I would like to see some clustering methods with scatterplot, but I don't imagine how that can happen in your context. Keep going man!", "votes": null }, { "id": "730065", "postDate": "01/27/2020 03:53:13", "content": "Glad you found it interesting @brunhs. I’m definitely interested in other visualizations for categorical data that make sense. But like you said, scatter plots aren’t the best choice for this context. Looking for new ideas if you have them :)
", "rawMarkdown": "Glad you found it interesting @brunhs. I’m definitely interested in other visualizations for categorical data that make sense. But like you said, scatter plots aren’t the best choice for this context. Looking for new ideas if you have them :)", "votes": null }, { "id": "736968", "postDate": "02/04/2020 18:45:01", "content": "If categories are high-dimensional you can try embedding with some matrix factorization methods (PCA, ALS, SVD) or using simple word2vec. Once you have continuous vector features any clustering methods can be used.
", "rawMarkdown": "If categories are high-dimensional you can try embedding with some matrix factorization methods (PCA, ALS, SVD) or using simple word2vec. Once you have continuous vector features any clustering methods can be used.", "votes": null }, { "id": "739257", "postDate": "02/07/2020 16:03:32", "content": "That’s a good thought @ezamir. That would work really well with a few of the nominal and ordinal variables in this dataset that contain 200+ categories.
", "rawMarkdown": "That’s a good thought @ezamir. That would work really well with a few of the nominal and ordinal variables in this dataset that contain 200+ categories.", "votes": null } ], "comments": [ { "id": 725139, "author_name": "rohanmahajan011", "author_url": "", "post_date": "01/21/2020 20:45:20", "content": "I had done a clustering problem on sentence embeddings. For that, I used K-Means.\nFor a Financial Analytics course, I had also used K-Nearest Neighbour which helped me. \nGreat topic! Upvoted.
", "votes": null, "replies": [ { "id": 725238, "author_name": "teejmahal20", "author_url": "", "post_date": "01/21/2020 23:31:07", "content": "Thanks @rohanmahajan011! What kind of preprocessing or standardizing did you do on the data prior to applying kmeans?
", "votes": null, "replies": [] } ] }, { "id": 725359, "author_name": "yuqizheng", "author_url": "", "post_date": "01/22/2020 02:54:53", "content": "Regarding the 2nd topic, a 'clustering' method that I've found really interesting is principal component analysis (P.C.A.) and kernel P.C.A. I know that they're usually described first as dimension reduction techniques, but they can also be thought of as used for clustering data. I do notice an interesting aspect is that they don't necessarily separate the data for you in the way that support vector machines (S.V.M.) may do so.
\n\nI learned about regular P.C.A. in a statistics course on multivariate data analysis and so I found the math quite interesting. The idea about transforming a data matrix into another one of identical dimensions was quite confusing to me at first. I think the concept is kind of strange, especially when you look at the analysis of the different eigenvectors.
\n\nLater on, I was able to learn about kernel P.C.A., and how the whole math shows that the data is actually being projected onto some sort of hyperplane which separates the data into different 'clusters.'
", "votes": null, "replies": [ { "id": 725842, "author_name": "teejmahal20", "author_url": "", "post_date": "01/22/2020 14:40:34", "content": "Thanks for the feedback @yuqizheng. PCA is definitely quite useful for visualizing clusters, particularly when you have a high-dimensional feature set.
", "votes": null, "replies": [] }, { "id": 726041, "author_name": "yuqizheng", "author_url": "", "post_date": "01/22/2020 18:36:47", "content": "I must admit though, my knowledge of algorithms still feels almost backwards. I'm a university student and I've studied an algorithms course prior to the machine learning courses. I hope to develop a greater understanding in due time. 😄
", "votes": null, "replies": [] } ] }, { "id": 729772, "author_name": "brunhs", "author_url": "", "post_date": "01/26/2020 16:20:50", "content": "Hi @teejmahal20, I found your analysis pretty interesting, you're doing pretty well with visualizations, I would like to see some clustering methods with scatterplot, but I don't imagine how that can happen in your context. Keep going man!
", "votes": null, "replies": [ { "id": 730065, "author_name": "teejmahal20", "author_url": "", "post_date": "01/27/2020 03:53:13", "content": "Glad you found it interesting @brunhs. I’m definitely interested in other visualizations for categorical data that make sense. But like you said, scatter plots aren’t the best choice for this context. Looking for new ideas if you have them :)
", "votes": null, "replies": [] } ] }, { "id": 736968, "author_name": "ezamir", "author_url": "", "post_date": "02/04/2020 18:45:01", "content": "If categories are high-dimensional you can try embedding with some matrix factorization methods (PCA, ALS, SVD) or using simple word2vec. Once you have continuous vector features any clustering methods can be used.
", "votes": null, "replies": [ { "id": 739257, "author_name": "teejmahal20", "author_url": "", "post_date": "02/07/2020 16:03:32", "content": "That’s a good thought @ezamir. That would work really well with a few of the nominal and ordinal variables in this dataset that contain 200+ categories.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "725105": "Hi All,\nI've started to take an interest in unsupervised learning lately and have found different clustering techniques really interesting for EDA and feature engineering.\n\nI posted a kernel doing k-modes clustering and came up with some interesting observations on our categorical dataset:\n\n[https://www.kaggle.com/teejmahal20/clustering-categorical-data-k-modes-cat-ii](https://www.kaggle.com/teejmahal20/clustering-categorical-data-k-modes-cat-ii)\n\nLet me know your thoughts/new ideas and if you find the analysis interesting and/or insightful. And upvote the kernel as well :)\n\nOn the topic of clustering, I have a couple of questions to pose:\n1) Specific to categorical data, has anyone approached cluster analysis in ways other than k-modes?\n2) In general, what methods have you found worthwhile for cluster analysis?", "725139": "I had done a clustering problem on sentence embeddings. For that, I used K-Means.\nFor a Financial Analytics course, I had also used K-Nearest Neighbour which helped me. \nGreat topic! Upvoted.", "725238": "Thanks @rohanmahajan011! What kind of preprocessing or standardizing did you do on the data prior to applying kmeans?", "725359": "Regarding the 2nd topic, a 'clustering' method that I've found really interesting is principal component analysis (P.C.A.) and kernel P.C.A. I know that they're usually described first as dimension reduction techniques, but they can also be thought of as used for clustering data. I do notice an interesting aspect is that they don't necessarily separate the data for you in the way that support vector machines (S.V.M.) may do so.\n\nI learned about regular P.C.A. in a statistics course on multivariate data analysis and so I found the math quite interesting. The idea about transforming a data matrix into another one of identical dimensions was quite confusing to me at first. I think the concept is kind of strange, especially when you look at the analysis of the different eigenvectors.\n\nLater on, I was able to learn about kernel P.C.A., and how the whole math shows that the data is actually being projected onto some sort of hyperplane which separates the data into different 'clusters.'", "725842": "Thanks for the feedback @yuqizheng. PCA is definitely quite useful for visualizing clusters, particularly when you have a high-dimensional feature set.", "726041": "I must admit though, my knowledge of algorithms still feels almost backwards. I'm a university student and I've studied an algorithms course prior to the machine learning courses. I hope to develop a greater understanding in due time. 😄", "729772": "Hi @teejmahal20, I found your analysis pretty interesting, you're doing pretty well with visualizations, I would like to see some clustering methods with scatterplot, but I don't imagine how that can happen in your context. Keep going man!", "730065": "Glad you found it interesting @brunhs. I’m definitely interested in other visualizations for categorical data that make sense. But like you said, scatter plots aren’t the best choice for this context. Looking for new ideas if you have them :)", "736968": "If categories are high-dimensional you can try embedding with some matrix factorization methods (PCA, ALS, SVD) or using simple word2vec. Once you have continuous vector features any clustering methods can be used.", "739257": "That’s a good thought @ezamir. That would work really well with a few of the nominal and ordinal variables in this dataset that contain 200+ categories." }, "source": "meta" }