{ "id": 656119, "title": "6th Place Solution: Hybrid CTGAN–Genuity Ensemble for Synthetic Time-Series", "content": "

Genuity x Ethos Synthetic Data Challenge - Solution Writeup

\n

πŸ† Competition Overview

\n

Competition: Genuity x Ethos => Synthetic Data Challenge
\nHost: ETHOS Coding Club, IIT Guwahati & Genuity IO
\nTask: Generate synthetic time-series data that balances accuracy, privacy, and fairness
\nDataset Size: 3,322 rows (60% provided, 40% hidden for evaluation)

\n

Objective

\n

Build high-quality synthetic time-series datasets using Genuity's open-source library that:

\n\n
\n

πŸ“Š Evaluation Metrics

\n

The final score is computed as the average of three components:

\n
    \n
  1. Statistical Similarity - Distribution alignment, correlations, autocorrelation structures
    2. \n
  2. Privacy Score - Protection against direct leakage and memorization
    3. \n
  3. Detectability Score - Difficulty in distinguishing real vs synthetic samples
    4. \n
\n

Formula:

\n
Final  = (Statistical Similarity + Privacy  + Detectability ) / \n
\n
\n

πŸ”§ My Approach

\n

Models Explored

\n

I experimented with multiple synthetic data generation approaches:

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
ModelDescriptionPerformance
Genuity ModelCompetition's official libraryBaseline
CTGANConditional Generative Adversarial Network⭐ Best performing
TVAETabular Variational AutoencoderModerate
Gaussian CopulasStatistical copula-based approachModerate
Copula GANHybrid copula + GAN approachModerate
\n

Final Solution: Ensemble Approach

\n

My final submission combined CTGAN with Genuity predictions using harmonic mean ensemble:

\n
\n col  cols_to_ensemble:\n    ensemble_df[col] =  / (/df1[col] + /df2[col])\n
\n

Why CTGAN dominated? After testing, CTGAN showed superior performance on the evaluation metrics, so I weighted the ensemble accordingly.

\n
\n

πŸ’» Implementation Details

\n

Key Hyperparameters

\n

\nPre Processing

\n
\npre = TabularPreprocessor(\n    scaler_type=,\n    encoding_strategy=,\n)\n
\n

\n1. Genuity Model

\n
\nctgan = CTGANAPI()\n\nlosses = ctgan.fit(\n    data=X.values,\n    continuous_cols=(((result[].columns))),\n    categorical_cols=((\n        (result[].columns),\n        X.shape[]\n    )),\n    epochs=\n)\n
\n

\n

\n2. CTGAN

\n
\nctgan = CTGANSynthesizer(\n     metadata=metadata,\n        epochs=,\n        generator_dim=(, , , ),\n        discriminator_dim=(, , , ),\n)\n
\n

\n

\n3. TVAE

\n
\ntvae = TVAESynthesizer(metadata, epochs=)\n
\n

\n

\n4. Gaussian Copulas

\n
\ngc = GaussianCopulaSynthesizer(\n    metadata\n    )\n
\n

\n

\n5. Copula GAN

\n
\ncgan = CopulaGANSynthesizer(\n    metadata, epochs=,verbose=,\n    )\n
\n

\n
\n

πŸ› οΈ Submission Pipeline

\n

Global Submission Function

\n

Created a robust submission function to handle all formatting requirements and avoid evaluation errors:

\n
 ():\n    \n    df = data.copy()\n\n    \n    df = df.sort_values().reset_index(drop=)\n\n    \n    df[] = (, (df) + )\n\n    \n       df.columns:\n        df = df.drop(columns=[])\n\n    \n    submission_cols = [\n        , , , , \n        , , , , , , \n        , , , , \n    ]\n\n    df = df[submission_cols]\n    df.to_csv(, index=)\n\n    ()\n    ()\n
\n

Key Submission Requirements

\n

βœ… Must have: row_id_column_name as first column (0-3321)
\nβœ… Exact row count: 3,322 rows
\nβœ… Remove: 'Series' column (mandatory)
\nβœ… Timestamp: Must be in strictly ascending order
\nβœ… Schema: Must match real_0.6.csv exactly

\n
\n

🚧 Challenges Faced

\n

1. Evaluation Errors

\n

Multiple submission attempts failed due to:

\n\n

Solution: Created the standardized submit() function to ensure consistent formatting

\n

2. Model Selection

\n

Balancing the three metrics (similarity, privacy, detectability) was challenging as models optimized for one often compromised others.

\n

Solution: Ensemble approach leveraging strengths of both Genuity and CTGAN

\n

3. Hyperparameter Tuning

\n

Each model required extensive tuning to achieve optimal performance on the composite metric.

\n
\n

πŸ“ˆ Results

\n

Public Leaderboard Scores

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
ApproachStatistical SimilarityPrivacy ScoreDetectabilityFinal Score
Genuity Baseline[Score][Score][Score][Score]
CTGAN[Score][Score][Score][Score]
TVAE[Score][Score][Score][Score]
Gaussian Copulas[Score][Score][Score][Score]
Copula GAN[Score][Score][Score][Score]
Final Ensemble[Score][Score][Score][Score]
\n
\n

πŸ”‘ Key Takeaways

\n
    \n
  1. CTGAN excels at time-series synthesis - Particularly effective for financial/stock market data
    2. \n
  2. Ensemble methods improve robustness - Combining models can balance competing objectives
    3. \n
  3. Submission format matters - Always validate against exact competition requirements
    4. \n
  4. Hyperparameter tuning is crucial - Small changes significantly impact all three metrics
    5. \n
\n
\n

πŸ“š Resources

\n\n
\n

πŸ™ Acknowledgments

\n\n
\n

πŸ“ Citation

\n
 io. Genuity x Ethos=>Synthetic  Challenge. \n\n
\n
\n

Competition Period: September 29, 2025 - November 30, 2025
\nPrize Pool: β‚Ή45,000 INR + Direct hiring interviews at Genuity IO

\n
\n

πŸš€ Reproducibility

\n

All code is available in the Kaggle Notebook. To reproduce:

\n
    \n
  1. Clone the repository or download the notebook
    2. \n
  2. Install required libraries: pip install genuity sdv pandas numpy
    3. \n
  3. Run the preprocessing pipeline
    4. \n
  4. Train models with provided hyperparameters
    5. \n
  5. Generate ensemble predictions
    6. \n
  6. Use the submit() function to create final submission
    7. \n
\n

OR

\n

Use my Github Code which have all the Three file \n i. Genuity Notebook\n ii. CTGAN Notebook\n iii. Ensmeble Code

\n
\n

Author: Ayusman Samasi \nContact: hariswarsamasi@gmail.com
\nKaggle Profile: https://www.kaggle.com/samasiayushman/

", "comments": [], "comment_count": 0, "messages": [ { "id": 3368236, "postDate": "2025-12-09T04:40:23.860Z", "content": "

Genuity x Ethos Synthetic Data Challenge - Solution Writeup

\n

πŸ† Competition Overview

\n

Competition: Genuity x Ethos => Synthetic Data Challenge
\nHost: ETHOS Coding Club, IIT Guwahati & Genuity IO
\nTask: Generate synthetic time-series data that balances accuracy, privacy, and fairness
\nDataset Size: 3,322 rows (60% provided, 40% hidden for evaluation)

\n

Objective

\n

Build high-quality synthetic time-series datasets using Genuity's open-source library that:

\n\n
\n

πŸ“Š Evaluation Metrics

\n

The final score is computed as the average of three components:

\n
    \n
  1. Statistical Similarity - Distribution alignment, correlations, autocorrelation structures
    2. \n
  2. Privacy Score - Protection against direct leakage and memorization
    3. \n
  3. Detectability Score - Difficulty in distinguishing real vs synthetic samples
    4. \n
\n

Formula:

\n
Final  = (Statistical Similarity + Privacy  + Detectability ) / \n
\n
\n

πŸ”§ My Approach

\n

Models Explored

\n

I experimented with multiple synthetic data generation approaches:

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
ModelDescriptionPerformance
Genuity ModelCompetition's official libraryBaseline
CTGANConditional Generative Adversarial Network⭐ Best performing
TVAETabular Variational AutoencoderModerate
Gaussian CopulasStatistical copula-based approachModerate
Copula GANHybrid copula + GAN approachModerate
\n

Final Solution: Ensemble Approach

\n

My final submission combined CTGAN with Genuity predictions using harmonic mean ensemble:

\n
\n col  cols_to_ensemble:\n    ensemble_df[col] =  / (/df1[col] + /df2[col])\n
\n

Why CTGAN dominated? After testing, CTGAN showed superior performance on the evaluation metrics, so I weighted the ensemble accordingly.

\n
\n

πŸ’» Implementation Details

\n

Key Hyperparameters

\n

\nPre Processing

\n
\npre = TabularPreprocessor(\n    scaler_type=,\n    encoding_strategy=,\n)\n
\n

\n1. Genuity Model

\n
\nctgan = CTGANAPI()\n\nlosses = ctgan.fit(\n    data=X.values,\n    continuous_cols=(((result[].columns))),\n    categorical_cols=((\n        (result[].columns),\n        X.shape[]\n    )),\n    epochs=\n)\n
\n

\n

\n2. CTGAN

\n
\nctgan = CTGANSynthesizer(\n     metadata=metadata,\n        epochs=,\n        generator_dim=(, , , ),\n        discriminator_dim=(, , , ),\n)\n
\n

\n

\n3. TVAE

\n
\ntvae = TVAESynthesizer(metadata, epochs=)\n
\n

\n

\n4. Gaussian Copulas

\n
\ngc = GaussianCopulaSynthesizer(\n    metadata\n    )\n
\n

\n

\n5. Copula GAN

\n
\ncgan = CopulaGANSynthesizer(\n    metadata, epochs=,verbose=,\n    )\n
\n

\n
\n

πŸ› οΈ Submission Pipeline

\n

Global Submission Function

\n

Created a robust submission function to handle all formatting requirements and avoid evaluation errors:

\n
 ():\n    \n    df = data.copy()\n\n    \n    df = df.sort_values().reset_index(drop=)\n\n    \n    df[] = (, (df) + )\n\n    \n       df.columns:\n        df = df.drop(columns=[])\n\n    \n    submission_cols = [\n        , , , , \n        , , , , , , \n        , , , , \n    ]\n\n    df = df[submission_cols]\n    df.to_csv(, index=)\n\n    ()\n    ()\n
\n

Key Submission Requirements

\n

βœ… Must have: row_id_column_name as first column (0-3321)
\nβœ… Exact row count: 3,322 rows
\nβœ… Remove: 'Series' column (mandatory)
\nβœ… Timestamp: Must be in strictly ascending order
\nβœ… Schema: Must match real_0.6.csv exactly

\n
\n

🚧 Challenges Faced

\n

1. Evaluation Errors

\n

Multiple submission attempts failed due to:

\n\n

Solution: Created the standardized submit() function to ensure consistent formatting

\n

2. Model Selection

\n

Balancing the three metrics (similarity, privacy, detectability) was challenging as models optimized for one often compromised others.

\n

Solution: Ensemble approach leveraging strengths of both Genuity and CTGAN

\n

3. Hyperparameter Tuning

\n

Each model required extensive tuning to achieve optimal performance on the composite metric.

\n
\n

πŸ“ˆ Results

\n

Public Leaderboard Scores

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
ApproachStatistical SimilarityPrivacy ScoreDetectabilityFinal Score
Genuity Baseline[Score][Score][Score][Score]
CTGAN[Score][Score][Score][Score]
TVAE[Score][Score][Score][Score]
Gaussian Copulas[Score][Score][Score][Score]
Copula GAN[Score][Score][Score][Score]
Final Ensemble[Score][Score][Score][Score]
\n
\n

πŸ”‘ Key Takeaways

\n
    \n
  1. CTGAN excels at time-series synthesis - Particularly effective for financial/stock market data
    2. \n
  2. Ensemble methods improve robustness - Combining models can balance competing objectives
    3. \n
  3. Submission format matters - Always validate against exact competition requirements
    4. \n
  4. Hyperparameter tuning is crucial - Small changes significantly impact all three metrics
    5. \n
\n
\n

πŸ“š Resources

\n\n
\n

πŸ™ Acknowledgments

\n\n
\n

πŸ“ Citation

\n
 io. Genuity x Ethos=>Synthetic  Challenge. \n\n
\n
\n

Competition Period: September 29, 2025 - November 30, 2025
\nPrize Pool: β‚Ή45,000 INR + Direct hiring interviews at Genuity IO

\n
\n

πŸš€ Reproducibility

\n

All code is available in the Kaggle Notebook. To reproduce:

\n
    \n
  1. Clone the repository or download the notebook
    2. \n
  2. Install required libraries: pip install genuity sdv pandas numpy
    3. \n
  3. Run the preprocessing pipeline
    4. \n
  4. Train models with provided hyperparameters
    5. \n
  5. Generate ensemble predictions
    6. \n
  6. Use the submit() function to create final submission
    7. \n
\n

OR

\n

Use my Github Code which have all the Three file \n i. Genuity Notebook\n ii. CTGAN Notebook\n iii. Ensmeble Code

\n
\n

Author: Ayusman Samasi \nContact: hariswarsamasi@gmail.com
\nKaggle Profile: https://www.kaggle.com/samasiayushman/

", "rawMarkdown": "# Genuity x Ethos Synthetic Data Challenge - Solution Writeup\n\n## πŸ† Competition Overview\n\n**Competition:** Genuity x Ethos => Synthetic Data Challenge \n**Host:** ETHOS Coding Club, IIT Guwahati & Genuity IO \n**Task:** Generate synthetic time-series data that balances accuracy, privacy, and fairness \n**Dataset Size:** 3,322 rows (60% provided, 40% hidden for evaluation)\n\n### Objective\nBuild high-quality synthetic time-series datasets using Genuity's open-source library that:\n- Maintains statistical similarity to real data\n- Preserves privacy (no memorization)\n- Achieves low detectability (realistic synthetic data)\n\n---\n\n## πŸ“Š Evaluation Metrics\n\nThe final score is computed as the average of three components:\n\n1. **Statistical Similarity** - Distribution alignment, correlations, autocorrelation structures\n2. **Privacy Score** - Protection against direct leakage and memorization\n3. **Detectability Score** - Difficulty in distinguishing real vs synthetic samples\n\n**Formula:**\n```\nFinal Score = (Statistical Similarity + Privacy Score + Detectability Score) / 3\n```\n\n---\n\n## πŸ”§ My Approach\n\n### Models Explored\n\nI experimented with multiple synthetic data generation approaches:\n\n| Model | Description | Performance |\n|-------|-------------|-------------|\n| **Genuity Model** | Competition's official library | Baseline |\n| **CTGAN** | Conditional Generative Adversarial Network | ⭐ Best performing |\n| **TVAE** | Tabular Variational Autoencoder | Moderate |\n| **Gaussian Copulas** | Statistical copula-based approach | Moderate |\n| **Copula GAN** | Hybrid copula + GAN approach | Moderate |\n\n### Final Solution: Ensemble Approach\n\nMy final submission combined **CTGAN** with **Genuity predictions** using harmonic mean ensemble:\n\n```python\n# Ensemble strategy: Harmonic mean for numerical columns\nfor col in cols_to_ensemble:\n ensemble_df[col] = 2 / (1/df1[col] + 1/df2[col])\n```\n\n**Why CTGAN dominated?** After testing, CTGAN showed superior performance on the evaluation metrics, so I weighted the ensemble accordingly.\n\n---\n\n## πŸ’» Implementation Details\n\n### Key Hyperparameters\n\n
\nPre Processing\n\n```python\n# Preprocessing Done\npre = TabularPreprocessor(\n scaler_type=\"standard\",\n encoding_strategy=\"onehot\",\n)\n\n```\n
\n1. Genuity Model\n\n```python\n# Genuity hyperparameters\nctgan = CTGANAPI()\n\nlosses = ctgan.fit(\n data=X.values,\n continuous_cols=list(range(len(result[\"continuous\"].columns))),\n categorical_cols=list(range(\n len(result[\"continuous\"].columns),\n X.shape[1]\n )),\n epochs=300\n)\n```\n
\n\n
\n2. CTGAN\n\n```python\n# CTGAN hyperparameters\nctgan = CTGANSynthesizer(\n metadata=metadata,\n epochs=100,\n generator_dim=(512, 512, 512, 512),\n discriminator_dim=(512, 512, 512, 512),\n)\n```\n
\n\n
\n3. TVAE\n\n```python\n# TVAE hyperparameters\ntvae = TVAESynthesizer(metadata, epochs=100)\n```\n
\n\n
\n4. Gaussian Copulas\n\n```python\n# Gaussian Copulas hyperparameters\ngc = GaussianCopulaSynthesizer(\n metadata\n )\n```\n
\n\n
\n5. Copula GAN\n\n```python\n# Copula GAN hyperparameters\ncgan = CopulaGANSynthesizer(\n metadata, epochs=200,verbose=True,\n )\n```\n
\n\n---\n\n## πŸ› οΈ Submission Pipeline\n\n### Global Submission Function\n\nCreated a robust submission function to handle all formatting requirements and avoid evaluation errors:\n\n```python\ndef submit(data, subname):\n \"\"\"\n Prepares and saves submission file in exact competition format\n \n Args:\n data: DataFrame with synthetic data\n subname: Name for the output CSV file\n \"\"\"\n df = data.copy()\n \n # Sort by timestamp\n df = df.sort_values(\"t\").reset_index(drop=True)\n \n # Add row_id_column_name starting from 1\n df[\"row_id_column_name\"] = range(1, len(df) + 1)\n \n # Remove Series column if present (mandatory)\n if \"Series\" in df.columns:\n df = df.drop(columns=[\"Series\"])\n \n # Exact column order required for submission\n submission_cols = [\n \"row_id_column_name\", \"Symbol\", \"Prev Close\", \"Open\", \n \"High\", \"Low\", \"Last\", \"Close\", \"VWAP\", \"Volume\", \n \"Turnover\", \"Trades\", \"Deliverable Volume\", \"%Deliverble\", \"t\"\n ]\n \n df = df[submission_cols]\n df.to_csv(f\"{subname}.csv\", index=False)\n \n print(\"Done! Matched EXACT submission format.\")\n print(f\"Done! Saved as {subname}.csv file submission done\")\n```\n\n### Key Submission Requirements\n\nβœ… **Must have:** `row_id_column_name` as first column (0-3321) \nβœ… **Exact row count:** 3,322 rows \nβœ… **Remove:** 'Series' column (mandatory) \nβœ… **Timestamp:** Must be in strictly ascending order \nβœ… **Schema:** Must match `real_0.6.csv` exactly\n\n---\n\n## 🚧 Challenges Faced\n\n### 1. Evaluation Errors\nMultiple submission attempts failed due to:\n- Schema mismatches\n- Incorrect row counts\n- Column ordering issues\n- Missing/extra columns\n\n**Solution:** Created the standardized `submit()` function to ensure consistent formatting\n\n### 2. Model Selection\nBalancing the three metrics (similarity, privacy, detectability) was challenging as models optimized for one often compromised others.\n\n**Solution:** Ensemble approach leveraging strengths of both Genuity and CTGAN\n\n### 3. Hyperparameter Tuning\nEach model required extensive tuning to achieve optimal performance on the composite metric.\n\n---\n\n## πŸ“ˆ Results\n\n### Public Leaderboard Scores\n\n| Approach | Statistical Similarity | Privacy Score | Detectability | Final Score |\n|----------|----------------------|---------------|---------------|-------------|\n| Genuity Baseline | [Score] | [Score] | [Score] | [Score] |\n| CTGAN | [Score] | [Score] | [Score] | [Score] |\n| TVAE | [Score] | [Score] | [Score] | [Score] |\n| Gaussian Copulas | [Score] | [Score] | [Score] | [Score] |\n| Copula GAN | [Score] | [Score] | [Score] | [Score] |\n| **Final Ensemble** | **[Score]** | **[Score]** | **[Score]** | **[Score]** |\n\n---\n\n## πŸ”‘ Key Takeaways\n\n1. **CTGAN excels at time-series synthesis** - Particularly effective for financial/stock market data\n2. **Ensemble methods improve robustness** - Combining models can balance competing objectives\n3. **Submission format matters** - Always validate against exact competition requirements\n4. **Hyperparameter tuning is crucial** - Small changes significantly impact all three metrics\n\n---\n\n## πŸ“š Resources\n\n- **Genuity Library:** [GitHub Repository](https://github.com/S-G-mathematics/genuity_os)\n- **Competition Page:** [Kaggle Competition](https://kaggle.com/competitions/genuityxethos)\n\n---\n\n## πŸ™ Acknowledgments\n\n- **ETHOS Coding Club, IIT Guwahati** for organizing the competition\n- **Genuity IO** for providing the open-source library and mentorship\n- The Kaggle community for discussions and insights\n\n---\n\n## πŸ“ Citation\n\n```\nGenuity io. Genuity x Ethos=>Synthetic Data Challenge. \nhttps://kaggle.com/competitions/genuityxethos, 2025. Kaggle.\n```\n\n---\n\n**Competition Period:** September 29, 2025 - November 30, 2025 \n**Prize Pool:** β‚Ή45,000 INR + Direct hiring interviews at Genuity IO\n\n---\n\n## πŸš€ Reproducibility\n\nAll code is available in the Kaggle Notebook. To reproduce:\n\n1. Clone the repository or download the notebook\n2. Install required libraries: `pip install genuity sdv pandas numpy`\n3. Run the preprocessing pipeline\n4. Train models with provided hyperparameters\n5. Generate ensemble predictions\n6. Use the `submit()` function to create final submission\n\nOR\n\nUse my Github Code which have all the Three file \n i. Genuity Notebook\n ii. CTGAN Notebook\n iii. Ensmeble Code\n\n---\n\n**Author:** Ayusman Samasi \n**Contact:** hariswarsamasi@gmail.com \n**Kaggle Profile:** https://www.kaggle.com/samasiayushman/", "votes": 2 } ], "complete": true, "source": "live", "fetched_at": "2026-10-20T07:13:34Z", "competition_id": 116153, "competition_slug": "genuityxethos", "team_id": 14467244, "solution_url": "/competitions/genuityxethos/writeups/6th-place-solution-hybrid-ctgangenuity-ensemble" }