# DLchallengeWTold — Data

来源：https://www.kaggle.com/competitions/dlchanwt/data

采集时间：2026-10-19T08:23:44Z

## Files

*   **train.csv** - the training set
*   **test.csv**  - the test set
*   **sample_submission.csv** - a sample submission file in the correct format, with the header and the first rows

The train dataset, has 2 array X and Y, the samples and labels. It is in HDF5 format stored in a ‘.mat’ Matlab file

train.mat

The test dataset is in pickle format. It stores just the X array with the samples:

test.pickle


A function to load a *.mat file and shuffle the samples:

```
import scipy.io

def loadDataset(fname, numClasses, trainPer, 
                featuresName='X', labelsName='Y',
                xType='float32', yType='uint8'):

    #load dataset
    ds=scipy.io.loadmat(fname)
    X = ds[featuresName] # X is expected to be float32
    y = ds[labelsName]   # Y is expected to be one hot encoded uint8

    #Separate datasets equally by fault classes
    sinalLength  = int(X.shape[1])
    samplesClass = int(X.shape[0] / numClasses)
    splitPoint   = int(samplesClass*trainPer)
    
    xTrain = []
    yTrain = []
    xTest  = []
    yTest  = []

    for i in range(numClasses):       
        #slice by fault class
        st = i*samplesClass
        sp = st + splitPoint
        end = (i+1)*samplesClass
        
        #shuffle in place each fault data before slice train/set
        p = np.random.permutation(samplesClass)
        X[st:end, :] = X[st+p, :]
        y[st:end, :] = y[st+p, :]
               
        xTrain.append(X[st:sp, :])
        yTrain.append(y[st:sp, :])
        xTest.append (X[sp:end, :])
        yTest.append (y[sp:end, :])

    #Reshape matrices to proper sizes and define data types
    xTrain = np.array(xTrain, dtype=xType)
    yTrain = np.array(yTrain, dtype=yType)
    xTest  = np.array(xTest,  dtype=xType)
    yTest  = np.array(yTest,  dtype=yType)
    xTrain = xTrain.reshape(numClasses*splitPoint, sinalLength)
    yTrain = yTrain.reshape(numClasses*splitPoint, numClasses)
    xTest  = xTest.reshape (numClasses*(samplesClass-splitPoint), sinalLength)
    yTest  = yTest.reshape (numClasses*(samplesClass-splitPoint), numClasses)
 
    #Samples are shuffled inside class
    #but classes are in order
    #Need to shuffle in fit (model.fit(shuffle=True))
    #which is better since it shuffle before each epoch

    return xTrain, yTrain, xTest, yTest 
    
```