[关闭]
@cleardusk 2015-11-20T11:14:54.000000Z 字数 6763 阅读 1304

SGD NN Excercise

GjzCVCode


练习

功能:一层网络(无隐藏层网络),调整 epochs, mini_batch_size, eta,输出 log info

  1. """
  2. Try creating a network with just two layers - an input and an output layer,
  3. no hidden layer - with 784 and 10 neurons, respectively. Train the network
  4. using stochastic gradient descent. What classification accuracy can you achieve?
  5. """
  6. import network
  7. import mnist_loader
  8. import time
  9. import itertools
  10. # def train():
  11. # net = network.Network([784, 10])
  12. # training_data, validation_data, test_data = \
  13. # mnist_loader.load_data_wrapper()
  14. #
  15. # epochs, mini_batch_size, eta = 30, 10, 1.0
  16. # return net.SGD(training_data, epochs, mini_batch_size,
  17. # eta, test_data=test_data)
  18. def profile():
  19. # Load dataset
  20. training_data, validation_data, test_data = \
  21. mnist_loader.load_data_wrapper()
  22. # parameters
  23. sizes = [784, 10]
  24. epochs_ = (30, 60)
  25. mini_batch_size_ = (5, 10, 15, 20)
  26. eta_ = (0.1, 0.5, 1, 2, 3)
  27. # epochs, mini_batch_size, eta = 30, 10, 1.0
  28. comninations = []
  29. for epochs in epochs_:
  30. for mini_batch_size in mini_batch_size_:
  31. for eta in eta_:
  32. comninations.append([epochs, mini_batch_size, eta])
  33. for epochs, mini_batch_size, eta in comninations:
  34. # time the training
  35. time_begin = time.clock()
  36. average_times = 3
  37. accuracy = []
  38. for i in xrange(average_times):
  39. net = network.Network(sizes)
  40. accuracy.append(net.SGD(training_data, epochs, mini_batch_size,
  41. eta, test_data=test_data))
  42. # awesome use of itertools
  43. accuracy = [sum(sublist) / float(average_times)
  44. for sublist in itertools.izip(*accuracy)]
  45. time_end = time.clock()
  46. # print network information
  47. print "Network size: %s" % str(sizes)
  48. print "Epochs: %d" % epochs
  49. print "Mini batch size: %d" % mini_batch_size
  50. print "Eta: %f " % eta
  51. print "The max accuracy is %.4f" % max(accuracy)
  52. print "The final accuracy is %.4f" % accuracy[-1]
  53. print 'Spent time: %.1f seconds\n' % \
  54. ((time_end - time_begin) / float(average_times))
  55. if __name__ == '__main__':
  56. # train()
  57. profile()

Log info

The best

  1. Network size: [784, 10]
  2. Epochs: 60
  3. Mini batch size: 10
  4. Eta: 0.500000
  5. The max accuracy is 0.8671
  6. The final accuracy is 0.8671
  7. Spent time: 261.6 seconds

All

  1. Network size: [784, 10]
  2. Epochs: 30
  3. Mini batch size: 5
  4. Eta: 0.100000
  5. The max accuracy is 0.7402
  6. The final accuracy is 0.7402
  7. Spent time: 135.9 seconds
  8. Network size: [784, 10]
  9. Epochs: 30
  10. Mini batch size: 5
  11. Eta: 0.500000
  12. The max accuracy is 0.6970
  13. The final accuracy is 0.6970
  14. Spent time: 135.8 seconds
  15. Network size: [784, 10]
  16. Epochs: 30
  17. Mini batch size: 5
  18. Eta: 1.000000
  19. The max accuracy is 0.0981
  20. The final accuracy is 0.0981
  21. Spent time: 136.4 seconds
  22. Network size: [784, 10]
  23. Epochs: 30
  24. Mini batch size: 5
  25. Eta: 2.000000
  26. The max accuracy is 0.1057
  27. The final accuracy is 0.1057
  28. Spent time: 135.7 seconds
  29. Network size: [784, 10]
  30. Epochs: 30
  31. Mini batch size: 5
  32. Eta: 3.000000
  33. The max accuracy is 0.0869
  34. The final accuracy is 0.0869
  35. Spent time: 135.8 seconds
  36. Network size: [784, 10]
  37. Epochs: 30
  38. Mini batch size: 10
  39. Eta: 0.100000
  40. The max accuracy is 0.7220
  41. The final accuracy is 0.7220
  42. Spent time: 131.2 seconds
  43. Network size: [784, 10]
  44. Epochs: 30
  45. Mini batch size: 10
  46. Eta: 0.500000
  47. The max accuracy is 0.7260
  48. The final accuracy is 0.7260
  49. Spent time: 131.2 seconds
  50. Network size: [784, 10]
  51. Epochs: 30
  52. Mini batch size: 10
  53. Eta: 1.000000
  54. The max accuracy is 0.0922
  55. The final accuracy is 0.0922
  56. Spent time: 131.3 seconds
  57. Network size: [784, 10]
  58. Epochs: 30
  59. Mini batch size: 10
  60. Eta: 2.000000
  61. The max accuracy is 0.1036
  62. The final accuracy is 0.1036
  63. Spent time: 131.1 seconds
  64. Network size: [784, 10]
  65. Epochs: 30
  66. Mini batch size: 10
  67. Eta: 3.000000
  68. The max accuracy is 0.1154
  69. The final accuracy is 0.1154
  70. Spent time: 131.2 seconds
  71. Network size: [784, 10]
  72. Epochs: 30
  73. Mini batch size: 15
  74. Eta: 0.100000
  75. The max accuracy is 0.5725
  76. The final accuracy is 0.5725
  77. Spent time: 129.5 seconds
  78. Network size: [784, 10]
  79. Epochs: 30
  80. Mini batch size: 15
  81. Eta: 0.500000
  82. The max accuracy is 0.6534
  83. The final accuracy is 0.6534
  84. Spent time: 129.5 seconds
  85. Network size: [784, 10]
  86. Epochs: 30
  87. Mini batch size: 15
  88. Eta: 1.000000
  89. The max accuracy is 0.0822
  90. The final accuracy is 0.0822
  91. Spent time: 129.7 seconds
  92. Network size: [784, 10]
  93. Epochs: 30
  94. Mini batch size: 15
  95. Eta: 2.000000
  96. The max accuracy is 0.0878
  97. The final accuracy is 0.0878
  98. Spent time: 129.5 seconds
  99. Network size: [784, 10]
  100. Epochs: 30
  101. Mini batch size: 15
  102. Eta: 3.000000
  103. The max accuracy is 0.1065
  104. The final accuracy is 0.1065
  105. Spent time: 129.9 seconds
  106. Network size: [784, 10]
  107. Epochs: 30
  108. Mini batch size: 20
  109. Eta: 0.100000
  110. The max accuracy is 0.5407
  111. The final accuracy is 0.5407
  112. Spent time: 128.8 seconds
  113. Network size: [784, 10]
  114. Epochs: 30
  115. Mini batch size: 20
  116. Eta: 0.500000
  117. The max accuracy is 0.7420
  118. The final accuracy is 0.7420
  119. Spent time: 128.6 seconds
  120. Network size: [784, 10]
  121. Epochs: 30
  122. Mini batch size: 20
  123. Eta: 1.000000
  124. The max accuracy is 0.1097
  125. The final accuracy is 0.1097
  126. Spent time: 128.8 seconds
  127. Network size: [784, 10]
  128. Epochs: 30
  129. Mini batch size: 20
  130. Eta: 2.000000
  131. The max accuracy is 0.0910
  132. The final accuracy is 0.0910
  133. Spent time: 128.9 seconds
  134. Network size: [784, 10]
  135. Epochs: 30
  136. Mini batch size: 20
  137. Eta: 3.000000
  138. The max accuracy is 0.1179
  139. The final accuracy is 0.1179
  140. Spent time: 128.8 seconds
  141. Network size: [784, 10]
  142. Epochs: 60
  143. Mini batch size: 5
  144. Eta: 0.100000
  145. The max accuracy is 0.7121
  146. The final accuracy is 0.7121
  147. Spent time: 270.7 seconds
  148. Network size: [784, 10]
  149. Epochs: 60
  150. Mini batch size: 5
  151. Eta: 0.500000
  152. The max accuracy is 0.7471
  153. The final accuracy is 0.7471
  154. Spent time: 270.5 seconds
  155. Network size: [784, 10]
  156. Epochs: 60
  157. Mini batch size: 5
  158. Eta: 1.000000
  159. The max accuracy is 0.1062
  160. The final accuracy is 0.1062
  161. Spent time: 271.4 seconds
  162. Network size: [784, 10]
  163. Epochs: 60
  164. Mini batch size: 5
  165. Eta: 2.000000
  166. The max accuracy is 0.0773
  167. The final accuracy is 0.0773
  168. Spent time: 271.2 seconds
  169. Network size: [784, 10]
  170. Epochs: 60
  171. Mini batch size: 5
  172. Eta: 3.000000
  173. The max accuracy is 0.1032
  174. The final accuracy is 0.1032
  175. Spent time: 271.3 seconds
  176. Network size: [784, 10]
  177. Epochs: 60
  178. Mini batch size: 10
  179. Eta: 0.100000
  180. The max accuracy is 0.7646
  181. The final accuracy is 0.7646
  182. Spent time: 261.7 seconds
  183. Network size: [784, 10]
  184. Epochs: 60
  185. Mini batch size: 10
  186. Eta: 0.500000
  187. The max accuracy is 0.8671
  188. The final accuracy is 0.8671
  189. Spent time: 261.6 seconds
  190. Network size: [784, 10]
  191. Epochs: 60
  192. Mini batch size: 10
  193. Eta: 1.000000
  194. The max accuracy is 0.1141
  195. The final accuracy is 0.1141
  196. Spent time: 262.2 seconds
  197. Network size: [784, 10]
  198. Epochs: 60
  199. Mini batch size: 10
  200. Eta: 2.000000
  201. The max accuracy is 0.1238
  202. The final accuracy is 0.1238
  203. Spent time: 262.2 seconds
  204. Network size: [784, 10]
  205. Epochs: 60
  206. Mini batch size: 10
  207. Eta: 3.000000
  208. The max accuracy is 0.1029
  209. The final accuracy is 0.1029
  210. Spent time: 262.2 seconds
  211. Network size: [784, 10]
  212. Epochs: 60
  213. Mini batch size: 15
  214. Eta: 0.100000
  215. The max accuracy is 0.6377
  216. The final accuracy is 0.6377
  217. Spent time: 259.7 seconds
  218. Network size: [784, 10]
  219. Epochs: 60
  220. Mini batch size: 15
  221. Eta: 0.500000
  222. The max accuracy is 0.8092
  223. The final accuracy is 0.8092
  224. Spent time: 259.5 seconds
  225. Network size: [784, 10]
  226. Epochs: 60
  227. Mini batch size: 15
  228. Eta: 1.000000
  229. The max accuracy is 0.0773
  230. The final accuracy is 0.0773
  231. Spent time: 259.5 seconds
  232. Network size: [784, 10]
  233. Epochs: 60
  234. Mini batch size: 15
  235. Eta: 2.000000
  236. The max accuracy is 0.0828
  237. The final accuracy is 0.0828
  238. Spent time: 259.2 seconds
  239. Network size: [784, 10]
  240. Epochs: 60
  241. Mini batch size: 15
  242. Eta: 3.000000
  243. The max accuracy is 0.1151
  244. The final accuracy is 0.1151
  245. Spent time: 259.3 seconds
  246. Network size: [784, 10]
  247. Epochs: 60
  248. Mini batch size: 20
  249. Eta: 0.100000
  250. The max accuracy is 0.7079
  251. The final accuracy is 0.7079
  252. Spent time: 257.7 seconds
  253. Network size: [784, 10]
  254. Epochs: 60
  255. Mini batch size: 20
  256. Eta: 0.500000
  257. The max accuracy is 0.8629
  258. The final accuracy is 0.8629
  259. Spent time: 257.5 seconds
  260. Network size: [784, 10]
  261. Epochs: 60
  262. Mini batch size: 20
  263. Eta: 1.000000
  264. The max accuracy is 0.1052
  265. The final accuracy is 0.1052
  266. Spent time: 257.5 seconds
  267. Network size: [784, 10]
  268. Epochs: 60
  269. Mini batch size: 20
  270. Eta: 2.000000
  271. The max accuracy is 0.1023
  272. The final accuracy is 0.1023
  273. Spent time: 257.6 seconds
  274. Network size: [784, 10]
  275. Epochs: 60
  276. Mini batch size: 20
  277. Eta: 3.000000
  278. The max accuracy is 0.1153
  279. The final accuracy is 0.1153
  280. Spent time: 258.1 seconds
添加新批注
在作者公开此批注前,只有你和作者可见。
回复批注