# Data

## Files
- **train.csv** (~45 million rows): Training data with features and target soil moisture values.
- **test.csv** (~5–10 million rows): Test data for predictions (target is hidden).
- **sample_submission.csv**: Example submission file format.

All files are in CSV format. Rows represent individual GNSS-R measurement samples (point observations over land).

**Note**: Data is large-scale tabular. Use efficient libraries like Pandas with chunking or Dask for loading. Expect some NaN values due to real-world measurement gaps (e.g., RFI flags or missing auxiliary data).

## Columns (Features and Target)

| Column Name              | Type    | Description                                                                 | Units / Notes |
|--------------------------|---------|-----------------------------------------------------------------------------|---------------|
| id                       | int     | Unique row identifier (for submission matching in test.csv)                | - |
| lat                      | float   | Latitude of the specular reflection point                                  | Degrees (-90 to 90) |
| lon                      | float   | Longitude of the specular reflection point                                 | Degrees (-180 to 180) |
| time                     | datetime| Observation timestamp (UTC)                                                 | YYYY-MM-DD HH:MM:SS |
| reflectivity             | float   | GNSS-R surface reflectivity (Γ, normalized power)                          | Unitless (typically 0–30 dB) |
| incidence_angle          | float   | Satellite incidence (elevation) angle                                       | Degrees (0–90) |
| azimuth_angle            | float   | Azimuth angle relative to track                                            | Degrees |
| coherence                | float   | Signal coherence (indicator of specular vs. diffuse reflection)            | Unitless (0–1) |
| surface_roughness        | float   | Estimated surface roughness parameter                                      | meters or unitless |
| vegetation_opacity       | float   | Vegetation optical depth (τ) at L-band                                     | Unitless (nepers) |
| ndvi                     | float   | Normalized Difference Vegetation Index (auxiliary, from MODIS/AVHRR)      | -1 to 1 |
| elevation                | float   | Surface elevation from DEM (e.g., SRTM)                                    | meters |
| land_cover               | int     | Land cover class (e.g., IGBP classification: 1=Evergreen Forest, etc.)     | Integer code |
| rfi_flag                 | bool/int| Radio Frequency Interference flag (1 = contaminated, recommend filtering) | 0/1 |
| ... (additional features)| ...     | Time-series stats (e.g., mean reflectivity last 7 days), etc.              | Varies |
| soil_moisture (target)   | float   | Volumetric soil moisture (0–5 cm depth) **Only in train.csv**              | cm³/cm³ (0–1 typical range) |

## Additional Notes
- **Target Distribution**: Soil moisture values range from ~0.0 (dry) to ~0.6 cm³/cm³ (saturated), with regional/seasonal variations. Expect skewness in arid vs. humid areas.
- **Spatial/Temporal Coverage**: Primarily ±38° latitude (CYGNSS orbit), 2017–2024, global land (filtered ocean/coast).
- **Challenges**: High heterogeneity (vegetation, topography, frozen soil in winter). NaNs common in auxiliary features—impute or mask as needed.
- **Ground Truth**: Target derived from co-located ISMN in-situ measurements and/or SMAP/SMOS products (nearest-neighbor matching with quality filters).
- **Recommended Baseline**: Start with simple models (e.g., LightGBM) on raw features, then add time-series engineering.

Download and explore in a Kaggle Notebook for fast iteration!

**Data Sources Acknowledgment**  
Based on NASA CYGNSS L3 products, ISMN, SMAP ancillary data. For full citations, see Overview page.