SGD NN Excercise

GjzCVCode

练习

功能：一层网络（无隐藏层网络），调整 epochs, mini_batch_size, eta，输出 log info

"""
Try creating a network with just two layers - an input and an output layer,
 no hidden layer - with 784 and 10 neurons, respectively. Train the network
 using stochastic gradient descent. What classification accuracy can you achieve?
"""
import network
import mnist_loader
import time
import itertools
# def train():
#     net = network.Network([784, 10])
#     training_data, validation_data, test_data = \
#         mnist_loader.load_data_wrapper()
#
#     epochs, mini_batch_size, eta = 30, 10, 1.0
#     return net.SGD(training_data, epochs, mini_batch_size,
#             eta, test_data=test_data)
def profile():
    # Load dataset
    training_data, validation_data, test_data = \
        mnist_loader.load_data_wrapper()
    # parameters
    sizes = [784, 10]
    epochs_ = (30, 60)
    mini_batch_size_ = (5, 10, 15, 20)
    eta_ = (0.1, 0.5, 1, 2, 3)
    # epochs, mini_batch_size, eta = 30, 10, 1.0
    comninations = []
    for epochs in epochs_:
        for mini_batch_size in mini_batch_size_:
            for eta in eta_:
                comninations.append([epochs, mini_batch_size, eta])
    for epochs, mini_batch_size, eta in comninations:
        # time the training
        time_begin = time.clock()
        average_times = 3
        accuracy = []
        for i in xrange(average_times):
            net = network.Network(sizes)
            accuracy.append(net.SGD(training_data, epochs, mini_batch_size,
                    eta, test_data=test_data))
        # awesome use of itertools
        accuracy = [sum(sublist) / float(average_times)
                    for sublist in itertools.izip(*accuracy)]
        time_end = time.clock()
        # print network information
        print "Network size: %s" % str(sizes)
        print "Epochs: %d" % epochs
        print "Mini batch size: %d" % mini_batch_size
        print "Eta: %f " % eta
        print "The max accuracy is %.4f" % max(accuracy)
        print "The final accuracy is %.4f" % accuracy[-1]
        print 'Spent time: %.1f seconds\n' % \
              ((time_end - time_begin) / float(average_times))
if __name__ == '__main__':
    # train()
    profile()

Log info

The best

Network size: [784, 10]
Epochs: 60
Mini batch size: 10
Eta: 0.500000 
The max accuracy is 0.8671
The final accuracy is 0.8671
Spent time: 261.6 seconds

All

Network size: [784, 10]
Epochs: 30
Mini batch size: 5
Eta: 0.100000 
The max accuracy is 0.7402
The final accuracy is 0.7402
Spent time: 135.9 seconds
Network size: [784, 10]
Epochs: 30
Mini batch size: 5
Eta: 0.500000 
The max accuracy is 0.6970
The final accuracy is 0.6970
Spent time: 135.8 seconds
Network size: [784, 10]
Epochs: 30
Mini batch size: 5
Eta: 1.000000 
The max accuracy is 0.0981
The final accuracy is 0.0981
Spent time: 136.4 seconds
Network size: [784, 10]
Epochs: 30
Mini batch size: 5
Eta: 2.000000 
The max accuracy is 0.1057
The final accuracy is 0.1057
Spent time: 135.7 seconds
Network size: [784, 10]
Epochs: 30
Mini batch size: 5
Eta: 3.000000 
The max accuracy is 0.0869
The final accuracy is 0.0869
Spent time: 135.8 seconds
Network size: [784, 10]
Epochs: 30
Mini batch size: 10
Eta: 0.100000 
The max accuracy is 0.7220
The final accuracy is 0.7220
Spent time: 131.2 seconds
Network size: [784, 10]
Epochs: 30
Mini batch size: 10
Eta: 0.500000 
The max accuracy is 0.7260
The final accuracy is 0.7260
Spent time: 131.2 seconds
Network size: [784, 10]
Epochs: 30
Mini batch size: 10
Eta: 1.000000 
The max accuracy is 0.0922
The final accuracy is 0.0922
Spent time: 131.3 seconds
Network size: [784, 10]
Epochs: 30
Mini batch size: 10
Eta: 2.000000 
The max accuracy is 0.1036
The final accuracy is 0.1036
Spent time: 131.1 seconds
Network size: [784, 10]
Epochs: 30
Mini batch size: 10
Eta: 3.000000 
The max accuracy is 0.1154
The final accuracy is 0.1154
Spent time: 131.2 seconds
Network size: [784, 10]
Epochs: 30
Mini batch size: 15
Eta: 0.100000 
The max accuracy is 0.5725
The final accuracy is 0.5725
Spent time: 129.5 seconds
Network size: [784, 10]
Epochs: 30
Mini batch size: 15
Eta: 0.500000 
The max accuracy is 0.6534
The final accuracy is 0.6534
Spent time: 129.5 seconds
Network size: [784, 10]
Epochs: 30
Mini batch size: 15
Eta: 1.000000 
The max accuracy is 0.0822
The final accuracy is 0.0822
Spent time: 129.7 seconds
Network size: [784, 10]
Epochs: 30
Mini batch size: 15
Eta: 2.000000 
The max accuracy is 0.0878
The final accuracy is 0.0878
Spent time: 129.5 seconds
Network size: [784, 10]
Epochs: 30
Mini batch size: 15
Eta: 3.000000 
The max accuracy is 0.1065
The final accuracy is 0.1065
Spent time: 129.9 seconds
Network size: [784, 10]
Epochs: 30
Mini batch size: 20
Eta: 0.100000 
The max accuracy is 0.5407
The final accuracy is 0.5407
Spent time: 128.8 seconds
Network size: [784, 10]
Epochs: 30
Mini batch size: 20
Eta: 0.500000 
The max accuracy is 0.7420
The final accuracy is 0.7420
Spent time: 128.6 seconds
Network size: [784, 10]
Epochs: 30
Mini batch size: 20
Eta: 1.000000 
The max accuracy is 0.1097
The final accuracy is 0.1097
Spent time: 128.8 seconds
Network size: [784, 10]
Epochs: 30
Mini batch size: 20
Eta: 2.000000 
The max accuracy is 0.0910
The final accuracy is 0.0910
Spent time: 128.9 seconds
Network size: [784, 10]
Epochs: 30
Mini batch size: 20
Eta: 3.000000 
The max accuracy is 0.1179
The final accuracy is 0.1179
Spent time: 128.8 seconds
Network size: [784, 10]
Epochs: 60
Mini batch size: 5
Eta: 0.100000 
The max accuracy is 0.7121
The final accuracy is 0.7121
Spent time: 270.7 seconds
Network size: [784, 10]
Epochs: 60
Mini batch size: 5
Eta: 0.500000 
The max accuracy is 0.7471
The final accuracy is 0.7471
Spent time: 270.5 seconds
Network size: [784, 10]
Epochs: 60
Mini batch size: 5
Eta: 1.000000 
The max accuracy is 0.1062
The final accuracy is 0.1062
Spent time: 271.4 seconds
Network size: [784, 10]
Epochs: 60
Mini batch size: 5
Eta: 2.000000 
The max accuracy is 0.0773
The final accuracy is 0.0773
Spent time: 271.2 seconds
Network size: [784, 10]
Epochs: 60
Mini batch size: 5
Eta: 3.000000 
The max accuracy is 0.1032
The final accuracy is 0.1032
Spent time: 271.3 seconds
Network size: [784, 10]
Epochs: 60
Mini batch size: 10
Eta: 0.100000 
The max accuracy is 0.7646
The final accuracy is 0.7646
Spent time: 261.7 seconds
Network size: [784, 10]
Epochs: 60
Mini batch size: 10
Eta: 0.500000 
The max accuracy is 0.8671
The final accuracy is 0.8671
Spent time: 261.6 seconds
Network size: [784, 10]
Epochs: 60
Mini batch size: 10
Eta: 1.000000 
The max accuracy is 0.1141
The final accuracy is 0.1141
Spent time: 262.2 seconds
Network size: [784, 10]
Epochs: 60
Mini batch size: 10
Eta: 2.000000 
The max accuracy is 0.1238
The final accuracy is 0.1238
Spent time: 262.2 seconds
Network size: [784, 10]
Epochs: 60
Mini batch size: 10
Eta: 3.000000 
The max accuracy is 0.1029
The final accuracy is 0.1029
Spent time: 262.2 seconds
Network size: [784, 10]
Epochs: 60
Mini batch size: 15
Eta: 0.100000 
The max accuracy is 0.6377
The final accuracy is 0.6377
Spent time: 259.7 seconds
Network size: [784, 10]
Epochs: 60
Mini batch size: 15
Eta: 0.500000 
The max accuracy is 0.8092
The final accuracy is 0.8092
Spent time: 259.5 seconds
Network size: [784, 10]
Epochs: 60
Mini batch size: 15
Eta: 1.000000 
The max accuracy is 0.0773
The final accuracy is 0.0773
Spent time: 259.5 seconds
Network size: [784, 10]
Epochs: 60
Mini batch size: 15
Eta: 2.000000 
The max accuracy is 0.0828
The final accuracy is 0.0828
Spent time: 259.2 seconds
Network size: [784, 10]
Epochs: 60
Mini batch size: 15
Eta: 3.000000 
The max accuracy is 0.1151
The final accuracy is 0.1151
Spent time: 259.3 seconds
Network size: [784, 10]
Epochs: 60
Mini batch size: 20
Eta: 0.100000 
The max accuracy is 0.7079
The final accuracy is 0.7079
Spent time: 257.7 seconds
Network size: [784, 10]
Epochs: 60
Mini batch size: 20
Eta: 0.500000 
The max accuracy is 0.8629
The final accuracy is 0.8629
Spent time: 257.5 seconds
Network size: [784, 10]
Epochs: 60
Mini batch size: 20
Eta: 1.000000 
The max accuracy is 0.1052
The final accuracy is 0.1052
Spent time: 257.5 seconds
Network size: [784, 10]
Epochs: 60
Mini batch size: 20
Eta: 2.000000 
The max accuracy is 0.1023
The final accuracy is 0.1023
Spent time: 257.6 seconds
Network size: [784, 10]
Epochs: 60
Mini batch size: 20
Eta: 3.000000 
The max accuracy is 0.1153
The final accuracy is 0.1153
Spent time: 258.1 seconds